Skip to content

Differentiate non-missing GT pairwise bias#673

Open
yliu2-sc wants to merge 2 commits into
mainfrom
yliu2/non-missing
Open

Differentiate non-missing GT pairwise bias#673
yliu2-sc wants to merge 2 commits into
mainfrom
yliu2/non-missing

Conversation

@yliu2-sc

@yliu2-sc yliu2-sc commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR differentiates non-missing pairwise GT attention-bias entries from missing entries.

Previously, sparse pairwise attention-bias values collapsed explicit zero-valued entries, diagonal self-pairs, and missing entries into the same 0.0 value. This makes it hard for the GT attention mechanism to distinguish self relationships from absent pairwise structure.

Changes

  • Adds pairwise_nonmissing_indices to Graph Transformer sequence auxiliary data.
  • Treats valid diagonal self-pairs as non-missing, even when their pairwise value is 0.0.
  • Treats explicit sparse CSR entries as non-missing, including explicit zero-valued entries.
  • Leaves missing non-self pairs as 0.0 without non-missing bias.
  • Adds a zero-initialized learned per-head non-missing attention bias in GraphTransformerEncoder.
  • Requires multiple pairwise attention-bias attributes to share identical non-missing support.

Tests

  • Added transform tests for:
    • diagonal self-pairs marked non-missing
    • explicit sparse entries marked non-missing
    • missing non-self pairs excluded
    • padding excluded
    • mismatched pairwise attr support raising ValueError
  • Added encoder test for:
    • head-specific non-missing bias
    • only indexed pairs receiving the bias
    • missing pairs remaining unchanged

Verification

Passed:

  • Custom smoke test for transform support coordinates and encoder non-missing bias
  • ruff check on touched files
  • ruff format --check on touched files
  • git diff --check

Where is the documentation for this feature?: N/A

Did you add automated tests or write a test plan?

Updated Changelog.md? NO

Ready for code review?: NO

@yliu2-sc yliu2-sc marked this pull request as ready for review June 18, 2026 16:02
@yliu2-sc yliu2-sc enabled auto-merge June 18, 2026 16:03
@yliu2-sc yliu2-sc added this pull request to the merge queue Jun 18, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 18, 2026
@yliu2-sc yliu2-sc added this pull request to the merge queue Jun 18, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants