Fix scoring of explicit whitespace tokens

[tandede]

Description

Explicit whitespace tokens were skipped while collecting predicted token attributes and dependencies, but the corresponding reference loops still included them. As a result, even otherwise identical predicted and reference documents containing a whitespace token could receive 0.8 instead of 1.0 for token attributes and dependency scores.

This change applies the existing whitespace-token policy symmetrically to the reference side of:

• score_token_attr
• score_token_attr_per_feat
• score_deps

Alignment indices remain unchanged because both sides continue to use the original token positions. The regression deliberately assigns different tag, POS, morphology, and dependency values to the predicted whitespace token and verifies that all affected scores remain 1.0, while the non-whitespace annotations still match.

Closes #13739.

Types of change

Bug fix.

Validation

• Before the fix, the regression example produced 0.8 for tag_acc, pos_acc, morph_acc, morph_micro_f, dep_uas, and dep_las; after the fix all six are 1.0.
• python -m pytest -q spacy/tests/test_scorer.py — 11 passed
• python -m pytest -q spacy — 3,615 passed, 1,229 skipped, 24 xfailed, 7 xpassed, 5 subtests passed
• python -m ruff format spacy --check — 886 files already formatted
• python -m ruff check spacy --select I — passed
• git diff --check — passed

The repository-wide all-rule Ruff hook reports pre-existing complexity and legacy-style findings in spacy/scorer.py; the official CI format and import-order checks pass.

Checklist

• I confirm that I have the right to submit this contribution under the project MIT license.
• I ran the tests, and all new and existing tests passed.
• My changes do not require a documentation change.