Skip to content

[6.x] Reset memoized Stache state between jobs instead of disabling it process-wide - #15126

Open
edalzell wants to merge 1 commit into
statamic:6.xfrom
edalzell:fix/isworker-job-memoization
Open

[6.x] Reset memoized Stache state between jobs instead of disabling it process-wide#15126
edalzell wants to merge 1 commit into
statamic:6.xfrom
edalzell:fix/isworker-job-memoization

Conversation

@edalzell

@edalzell edalzell commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Cause

Statamic::isWorker() disables in-process memoization for the entire life of a queue worker process whenever the running command is queue:*/horizon:*, instead of just resetting that memoization between jobs. That means every single access within one job falls back to a cache-store round trip (or full disk listing) rather than reusing an already-computed value, making queued indexing dramatically slower per document than running the identical work synchronously.

Closes #15118.

Approach

  • Store/ContainerAssetsStore/Index are container singletons that persist for the whole worker process, so they genuinely need an explicit reset at job boundaries. Added a resetMemoizedState() method to each, invoked once per job via a new JobProcessing listener that's only active when Statamic::isWorker() is true (so synchronous dispatch is completely unaffected).
  • AssetContainer/AssetContainerContents are Blink-facade-backed, and Laravel's real queue:work daemon loop already clears resolved facade instances before every job it processes — so their isWorker() gates were redundant (and actively harmful, since they disabled memoization within a job too) and were simply removed.
  • A redundant eager cache-store write in AssetContainerContents::add() (made unnecessary by the caller always chaining ->save() right after) was also removed.

Real performance numbers

Measured on a sandbox app with a 17,000-file asset container, indexing a 100-entry chunk (chunk_size 100) where each entry references 4 assets:

per 100-doc job
Sync (QUEUE_CONNECTION=sync) ~29ms/doc (2.9s total)
Queue worker, before this fix ~2.0s/doc (3m 20s total)
Queue worker, after this fix ~10ms/doc (1s total)

That's a ~200x speedup for the identical job, with the fixed worker now on par with (slightly faster than) sync — confirming the fix removes the bug's overhead without reintroducing cross-job staleness (covered by new tests around resetMemoizedState()).

…ling it for the whole worker process

Statamic::isWorker() disabled in-process memoization for the entire life of a
queue worker whenever the running command was queue:*/horizon:*, instead of
just resetting it between jobs. That meant every single access within one
job fell back to a cache-store round trip (or full disk listing), making
queued indexing dramatically slower per document than the identical work run
synchronously.

Store/ContainerAssetsStore/Index are container singletons that persist for
the whole worker process, so they need an explicit reset at job boundaries -
added via a new JobProcessing listener (only active when isWorker() is true,
so sync dispatch is unaffected). AssetContainer/AssetContainerContents are
Blink-backed, and Laravel's real queue:work daemon loop already clears
resolved facade instances before every job, so their isWorker() gates were
redundant and simply removed.

Measured on a sandbox with a 17k-file asset container, indexing a 100-entry
chunk (chunk_size=100) referencing 4 assets each:
  - sync:                   ~29ms/doc  (2.9s total)
  - queue worker, before:   ~2.0s/doc  (3m20s total)
  - queue worker, after:    ~10ms/doc  (1s total)

~200x faster for the identical job, with no cross-job staleness reintroduced
(covered by new tests around resetMemoizedState()).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SRyCmr5vUE71mdgXVC3KCB
@edalzell edalzell changed the title Reset memoized Stache/asset state between queue jobs instead of disabling it process-wide [6.x] Reset memoized Stache state between jobs instead of disabling it process-wide Aug 5, 2026
@edalzell
edalzell marked this pull request as ready for review August 5, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Statamic::isWorker() disables memoisation for the whole process, making queued indexing ~30x slower than sync

1 participant