Skip to content

Only propagate newly-dirty vars in _mark_dirty_computed_vars#6743

Open
Alek99 wants to merge 5 commits into
mainfrom
claude/reflex-perf-optimizations-01l7a3-eng-10095
Open

Only propagate newly-dirty vars in _mark_dirty_computed_vars#6743
Alek99 wants to merge 5 commits into
mainfrom
claude/reflex-perf-optimizations-01l7a3-eng-10095

Conversation

@Alek99

@Alek99 Alek99 commented Jul 10, 2026

Copy link
Copy Markdown
Member

Linear: ENG-10095

Description

Every setattr and every proxied append/__setitem__ triggered a full dirty-propagation pass: a needs_update scan over all computed vars (interval expiry) plus a dependency re-walk of the entire accumulated dirty_vars set, not just the newly dirtied var.

  • Each class now precomputes which of its computed vars carry an update interval (_interval_computed_vars, refreshed whenever computed_vars changes), so the expiry scan is skipped entirely for the overwhelmingly common case of none.
  • Each instance tracks a transient "propagated frontier": vars whose dependency closure has already been walked. Subsequent mutations only process newly-dirty vars.

Correctness: re-walking the dirty set is only observably needed after a cached computed var is recomputed mid-cycle (the recompute re-materializes a cache that a later mutation of its dependencies must invalidate again — including across states). Recomputes therefore bump a process-wide generation counter, and a generation mismatch resets the frontier, forcing the same full re-walk as today. The frontier is cleared by _clean and excluded from pickling, so restored/copied states always start with a full walk.

Benchmarks (GitHub Actions runner, run, 2 passes each; state with 2 cached computed vars)

Case main this PR speedup
proxied list append 16.72 / 15.97 us 8.51 / 8.64 us ~1.9x
int var setattr 21.00 / 20.32 us 9.33 / 9.27 us ~2.2x

cProfile (2000 proxied appends): total function calls drop 216k -> 78k; _mark_dirty_computed_vars cumulative time drops 0.102s -> 0.019s and _expired_computed_vars disappears from the profile. The benchmark script also spot-checks that computed vars still recompute correctly after the mutation loop.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Changes To Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
    • test_computed_var_recompute_after_mid_cycle_read: mutate -> read (recompute) -> mutate again -> the cache is invalidated again (the exact staleness hazard a naive frontier would introduce).
    • test_computed_var_recompute_after_mid_cycle_read_across_states: same hazard across parent/child states.
    • test_interval_computed_vars_precomputed: per-class interval var precomputation.
  • Have you successfully ran tests with your changes locally?

Note: this touches the same ComputedVar.__get__ region as #6738; whichever merges second needs a trivial rebase.

🤖 Generated with Claude Code

https://claude.ai/code/session_01G8cXh3TUbNtbjm2ERqE62X


Generated by Claude Code

Every setattr and proxied mutation triggered a full dirty-propagation
pass: an expiry scan over all computed vars plus a dependency re-walk
of the entire accumulated dirty_vars set.

- Precompute per class which computed vars have an update interval, so
  the expiry scan is skipped entirely for the common case of none.
- Track a per-instance "propagated frontier" so each propagation only
  processes vars whose dependency closure has not been walked yet.
  Recomputing a cached var re-materializes its cache, which a later
  mutation of its dependencies must invalidate again, so recomputes
  bump a generation counter that resets the frontier. The frontier is
  transient: cleared by _clean and excluded from pickling.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G8cXh3TUbNtbjm2ERqE62X
@Alek99 Alek99 requested a review from a team as a code owner July 10, 2026 20:20
@linear-code

linear-code Bot commented Jul 10, 2026

Copy link
Copy Markdown

ENG-10095

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G8cXh3TUbNtbjm2ERqE62X
@greptile-apps

greptile-apps Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR makes computed-var dirty propagation do less repeated work. The main changes are:

  • Tracks already-propagated dirty vars during a mutation cycle.
  • Resets that tracking when cached computed vars are recomputed.
  • Precomputes interval-based computed vars per state class.
  • Adds tests for mid-cycle recompute behavior and interval-var tracking.
  • Adds release notes for the performance improvement.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
reflex/state.py Adds the dirty-propagation frontier, interval computed-var cache, cleanup handling, and serialization handling.
packages/reflex-base/src/reflex_base/vars/base.py Adds a shared recompute generation counter and bumps it when cached computed vars recompute.
packages/reflex-base/src/reflex_base/utils/types.py Marks the new transient propagation fields as reserved backend var names.
tests/units/test_state.py Adds tests for recomputation after mid-cycle reads, cross-state recomputation, and interval-var precomputation.
news/6743.performance.md Adds a release note for the dirty-propagation performance change.
packages/reflex-base/news/6743.performance.md Adds the package release note for the dirty-propagation performance change.

Reviews (4): Last reviewed commit: "Write propagation generation via __dict_..." | Re-trigger Greptile

@codspeed-hq

codspeed-hq Bot commented Jul 10, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 7.01%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 25 untouched benchmarks
⏩ 8 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
test_process_event 7.6 ms 7.1 ms +7.01%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/reflex-perf-optimizations-01l7a3-eng-10095 (0c44576) with main (32cc257)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Comment thread reflex/state.py Outdated
current_gen = reflex_base_vars_base._computed_var_recompute_generation
if propagated is None:
propagated = set()
object.__setattr__(self, "_propagated_dirty_vars", propagated)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we already have the instance_dict, why not just shove the value in there?

Comment thread reflex/state.py Outdated
self.dirty_vars = set()
self.dirty_substates = set()
# Discard the propagation frontier along with the dirty vars it tracked.
self.__dict__.pop("_propagated_dirty_vars", None)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we treating these new fields like this and not defining them as normal fields in a BaseState, like all the other fields

claude added 3 commits July 14, 2026 00:33
Match review feedback on the sibling PR: declare
_propagated_dirty_vars and _propagated_generation as real (non-var)
fields and reserve their names, so user vars cannot silently collide
with the framework's tracking machinery. Still transient: excluded
from pickling and recreated on unpickle.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G8cXh3TUbNtbjm2ERqE62X
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G8cXh3TUbNtbjm2ERqE62X
Background task handlers run state methods with self bound to a
StateProxy; wrapt's C ObjectProxy (used on Python <= 3.12) rejects
object.__setattr__ on the proxy, while __dict__ resolves to the
wrapped state's dict on both proxies and plain states. Fixes
test_background_task_* failures on 3.10-3.12.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G8cXh3TUbNtbjm2ERqE62X
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants