Skip to content

feat: refactor four attention converters with TRT add_attention_v2()#4416

Open
zewenli98 wants to merge 2 commits into
mainfrom
evanli/refactor_attention_impl
Open

feat: refactor four attention converters with TRT add_attention_v2()#4416
zewenli98 wants to merge 2 commits into
mainfrom
evanli/refactor_attention_impl

Conversation

@zewenli98

Copy link
Copy Markdown
Collaborator

Description

This PR refactors four attention converters with TRT add_attention_v2(). Additionally, since newer torch rejects passing both is_causal=True and explicit attn_mask to SDPA, which aligns with TRT, some scenarios are gated in validators and the unreachable branches are pruned.

Fixes # (issue)

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • My code follows the style guidelines of this project (You can use the linters)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas and hacks
  • I have made corresponding changes to the documentation
  • I have added tests to verify my fix or my feature
  • New and existing unit tests pass locally with my changes
  • I have added the relevant labels to my PR in so that relevant reviewers are notified

@zewenli98
zewenli98 requested a review from narendasan July 17, 2026 18:09
@zewenli98 zewenli98 self-assigned this Jul 17, 2026
@meta-cla meta-cla Bot added the cla signed label Jul 17, 2026
@github-actions github-actions Bot added component: conversion Issues re: Conversion stage component: core Issues re: The core compiler component: converters Issues re: Specific op converters component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Jul 17, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/lowering/passes/eliminate_sym_min_int64_max.py	2026-07-17 18:09:43.975250+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/lowering/passes/eliminate_sym_min_int64_max.py	2026-07-17 18:10:04.835218+00:00
@@ -2,11 +2,10 @@

import torch
from torch.fx import GraphModule, Node

from .pass_utils import clean_up_graph_after_modifications
-

_INT64_MAX = 2**63 - 1
_SYM_MIN = getattr(torch, "sym_min", None)

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/lowering/passes/eliminate_sym_min_int64_max.py	2026-07-17 18:09:44.412631+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/lowering/passes/eliminate_sym_min_int64_max.py	2026-07-17 18:10:05.940612+00:00
@@ -2,11 +2,10 @@

import torch
from torch.fx import GraphModule, Node

from .pass_utils import clean_up_graph_after_modifications
-

_INT64_MAX = 2**63 - 1
_SYM_MIN = getattr(torch, "sym_min", None)

@github-actions github-actions Bot added the component: lowering Issues re: The lowering / preprocessing passes label Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: api [Python] Issues re: Python API component: conversion Issues re: Conversion stage component: converters Issues re: Specific op converters component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant