add torch-tensorrt-executorch-delegate wheel build by lanluo-nvidia · Pull Request #4398 · pytorch/TensorRT

lanluo-nvidia · 2026-07-11T09:49:14Z

Description

export EXECUTORCH_SOURCE_DIR=/home/lanl/git/executorch
export EXECUTORCH_ROOT=/home/lanl/git/executorch

build the torch-tensorrt-executorch-delegate.whl

python -m pip wheel
--no-build-isolation
--no-deps
--wheel-dir dist
packaging/executorch_delegate

install the wheel

python -m pip install --no-deps --force-reinstall
dist/torch_tensorrt_executorch_delegate-*.whl

export the pte

python /home/lanl/git/py312/TensorRT/examples/torchtrt_executorch_example/export_static_shape.py

load the pte

python examples/executorch_reference_runner/load_model.py
--model_path=/path/to/model.pte

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Checklist:

My code follows the style guidelines of this project (You can use the linters)
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas and hacks
I have made corresponding changes to the documentation
I have added tests to verify my fix or my feature
New and existing unit tests pass locally with my changes
I have added the relevant labels to my PR in so that relevant reviewers are notified

narendasan · 2026-07-13T15:47:38Z

@lanluo-nvidia the new package should be in

//py
-> torch-tensorrt
-> torch-tensorrt-executorch-delegate

See: https://packaging.python.org/en/latest/guides/packaging-namespace-packages/

shoumikhin

Nice work on this. The core idea is right. You build _portable_lib from source and pull TensorRTBackend into the same module that holds the backend registry, so the two always share one copy of the ExecuTorch core. That is the correct way to fix the undefined symbol crash people hit today. I checked the linking part and it holds up: one registry, no duplicate core, and the whole-archive flag is actually needed.

shoumikhin · 2026-07-15T03:56:02Z

+            )
+
+        output = pathlib.Path(self.get_ext_fullpath(ext.name)).resolve()
+        if output.exists():


Minor: if output.exists(): return skips the build when the .so is already there. On a clean CI run that is fine, but on a local rebuild it can ship an old .so from a previous build without telling you. Consider a build stamp, or just remove the early return.

shoumikhin · 2026-07-15T03:56:02Z

+export EXECUTORCH_SOURCE_DIR="$(dirname "${executorch_cmake_location%%:*}")"
+export TensorRT_ROOT=/path/to/TensorRT
+
+python -m pip install executorch==1.3.1


Minor: the README says pip install executorch==1.3.1, but MODULE.bazel now targets main for the 2.14 ABI. Please make the doc match the version you actually build against.

shoumikhin · 2026-07-15T03:56:04Z

One more note, on a file this PR does not change so I cannot leave it inline. In cpp/src/torch_tensorrt/executorch/TensorRTBackend.cpp around line 653, the result of register_backend is stored in kRegistered but never checked, so if registration fails you find out much later with a confusing "Backend TensorRTBackend is not registered" message. It would help to notice the failure at registration time. One thing to watch out for: this runs very early, before logging is set up, so do not call ET_LOG here. A safer option is to save the result and check it the first time the backend is used.

narendasan · 2026-07-16T20:40:23Z

+requires = [
+    "setuptools>=68",
+    "wheel>=0.40",
+    "torch",


Do we not depend on executorch for the delegate?

narendasan · 2026-07-16T20:40:55Z

+        build_dir = pathlib.Path(self.build_temp) / "executorch_delegate_native"
+        output.parent.mkdir(parents=True, exist_ok=True)
+        build_dir.mkdir(parents=True, exist_ok=True)
+        configure = [


Why arent we using bazel here like the main package?

I think it would be better to have one real preferred build system

narendasan · 2026-07-16T20:42:02Z

+    return runtime()
+
+
+class Program:


Should probably be in the delegate not this package

We can lift apis into the main package gated by import but any dependent code should be in the sub package

narendasan · 2026-07-16T20:42:23Z

+        return self.run(inputs, "forward")
+
+
+def load(path: Union[str, Path]) -> Program:


Also in the delegate and not this package. we should just expose via torch_tensorrt.load

narendasan · 2026-07-16T20:42:51Z

+EXECUTORCH_REQUIREMENT = "executorch>=1.3.1"
+EXECUTORCH_DELEGATE_REQUIREMENT = (
+    f"torch-tensorrt-executorch-delegate=={__version__}; "
+    "platform_system == 'Linux' and platform_machine == 'x86_64'"


We need to support ARM as well (Jetson Thor)

github-actions

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/py/torch-tensorrt-executorch-delegate/setup.py	2026-07-17 19:30:44.124103+00:00
+++ /home/runner/work/TensorRT/TensorRT/py/torch-tensorrt-executorch-delegate/setup.py	2026-07-17 19:31:08.985550+00:00
@@ -14,10 +14,11 @@
from setuptools import Extension, find_packages, setup
from setuptools.command.build_ext import build_ext

HERE = pathlib.Path(__file__).resolve().parent
REPO_ROOT = HERE.parents[1]
+

def torchtrt_version() -> str:
    if value := os.getenv("TORCH_TENSORRT_EXECUTORCH_DELEGATE_VERSION"):
        return value
    version_py = REPO_ROOT / "py/torch_tensorrt/_version.py"

add torch-tensorrt-executorch-delegate wheel

f215c0c

meta-cla Bot added the cla signed label Jul 11, 2026

github-actions Bot added component: tests Issues re: Tests component: build system Issues re: Build system component: api [Python] Issues re: Python API labels Jul 11, 2026

github-actions Bot requested a review from zewenli98 July 11, 2026 09:49

lanluo-nvidia added 2 commits July 11, 2026 16:41

fix the test error

f1339a0

test

e3a8f27

shoumikhin reviewed Jul 15, 2026

View reviewed changes

lanluo-nvidia marked this pull request as ready for review July 15, 2026 17:29

Merge branch 'main' into lluo/executorch_wheel_build

5ccbfcd

narendasan reviewed Jul 16, 2026

View reviewed changes

lanluo-nvidia added 2 commits July 17, 2026 12:30

resolve review comments

14c9c71

resolve review commits

e6a6f7a

github-actions Bot requested changes Jul 17, 2026

View reviewed changes

		return self.run(inputs, "forward")


		def load(path: Union[str, Path]) -> Program:

Uh oh!

Conversation

lanluo-nvidia commented Jul 11, 2026

Description

build the torch-tensorrt-executorch-delegate.whl

install the wheel

export the pte

load the pte

Type of change

Checklist:

Uh oh!

narendasan commented Jul 13, 2026

Uh oh!

shoumikhin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

shoumikhin commented Jul 15, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants