Skip to content

Consider memoizing grammar validation results per (grammar, value) #473

Description

@SJrX

Idea

Cache the result of validating a value against a grammar in memory, keyed by the grammar and the value string, so repeated validations of the same (grammar, value) pair don't re-run the matcher.

Context: the new list-of-successes engine (#467) runs Combinator.parse() / validate() during highlighting. The same value can be validated on every highlighting pass while it hasn't changed.

Design notes / gotchas (if/when we do this)

  • Cache the ParseOutcome, not the Sequence<ParseStep>. The outcome is small and immutable; the parse stream is lazy and stateful and must not be memoized.
  • Key = (grammar identity, value). Grammar validators are singletons with stable identity, so a per-grammar Map<String, ParseOutcome> is the natural shape (or a global Map<Pair<Combinator, String>, ParseOutcome>).
  • Bound it. Distinct value strings grow without limit as the user types — use an LRU / size cap, not an unbounded map (memory-leak risk).
  • Thread-safety. Highlighting/inspection runs on background threads concurrently, so the cache must be concurrent-safe.
  • Same applies to completion once that lands (Grammar Based Completion #343): a (grammar, prefix) -> completions cache has the same shape and trade-offs.

Open question: is it even needed?

Probably premature until profiling shows validation is a measurable cost:

  • option values are short, and
  • validate() already short-circuits on the first valid full parse, so the common (valid) case is cheap.

Also, IntelliJ likely already avoids re-running inspections on unchanged PSI (and CachedValuesManager ties cached results to PSI modification counts), so a hand-rolled cache may duplicate platform behavior. Measure first; only add if there's a real hotspot, and prefer a platform mechanism (CachedValuesManager) over a bespoke map if one fits.

Refs #467

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions