Skip to content

Add cel-expr-python backend#485

Open
anuraaga wants to merge 10 commits into
bufbuild:mainfrom
anuraaga:cel-python
Open

Add cel-expr-python backend#485
anuraaga wants to merge 10 commits into
bufbuild:mainfrom
anuraaga:cel-python

Conversation

@anuraaga

@anuraaga anuraaga commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

This adds an implementation using cel-expr-python. It is much faster, orders of magnitude, but has many gotchas. This is currently a rough draft PR to show the concept and list up what it means

  • We will need to keep the celpy fallback due to limited platform support for cel-expr-python. The engine glue code is large and duplicated by necessity
    • No musl wheels
    • No free-threaded Python support
    • Build requires Bazel so no sdist. Normally extensions like protobuf-py-ext can still be used with the relatively simple "install rust" prereq on less common platforms, but here it's impossible
  • We can't add an automatic dependency on it using markers so it is always opt-in by user
    • No marker for musl vs glibc
    • No marker for free-threaded Python
  • We have to copy protobuf-py messages to google.protobuf messages. There is some perf impact but we can consider it minimal. But our core engine becomes google.protobuf, even though we want our ecosystem to be built on protobuf-py
  • Our own code (i.e. extra_func.py) gets more brittle inheriting google.protobuf's weak type annotations. No docstrings on the API we use

@anuraaga anuraaga marked this pull request as draft July 3, 2026 05:32
@anuraaga anuraaga marked this pull request as ready for review July 7, 2026 13:00
@anuraaga anuraaga requested a review from ajeetdsouza July 7, 2026 13:00
@anuraaga

anuraaga commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

While I have some reservations, we have decided to go with this approach - we'll see if issues come up going forward. To make it easier to rethink, I have kept the implementations largely separate - anyways, the CEL APIs being quite different necessitates a lot of the general control flow duplication, and it would get quite surgical to try to extract common logic bits. We can reconsider in the future if things settle

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant