Skip to content

feat: add optional bounded LSP semantic providers - #2951

Open
kerberosmansour wants to merge 1 commit into
Graphify-Labs:v8from
kerberosmansour:feat/bounded-lsp-providers
Open

feat: add optional bounded LSP semantic providers#2951
kerberosmansour wants to merge 1 commit into
Graphify-Labs:v8from
kerberosmansour:feat/bounded-lsp-providers

Conversation

@kerberosmansour

Copy link
Copy Markdown

Summary

  • add an optional, local Language Server Protocol evidence layer without changing Graphify's native tree-sitter extraction path
  • provide one bounded runner and manifest-driven registry for Rust, TypeScript/JavaScript, Java, Kotlin, C#, Python, Go, PHP, and Ruby
  • preserve provider/run/time/range provenance on symbols and relationships
  • merge provider evidence additively into a separate graph output, reconciling with native nodes only on one unambiguous (source_file, label) match
  • expose explicit completed, unavailable, failed, and budget_exhausted outcomes through graphify-semantic

Closes #2948.

Related to #2941, which refreshes Graphify's existing semantic pipeline. This PR adds a distinct compiler/language-service evidence source and does not change that pipeline.

Static-analysis findings have different trust semantics and remain outside this PR; the bounded OpenGrep proposal is tracked separately in #2950.

Why

Tree-sitter is the right fast, deterministic baseline, but project-aware language services can resolve facts that syntax alone cannot reliably establish: configured module resolution, references, implementations, overloads, and call hierarchy.

The contribution is deliberately additive:

  • normal Graphify commands behave as before;
  • no language-server package is bundled or added as a runtime dependency;
  • no provider is installed, downloaded, or run by default;
  • missing or failed providers cannot remove native graph evidence;
  • custom provider manifests extend language coverage without adding language-specific runner code.

Safety and failure boundaries

  • subprocesses use argv execution with shell=False;
  • source files, symlinks, and returned locations are confined to the selected workspace;
  • file size/count, symbol count, relationship requests, RPC message size, per-request time, and retained notifications are bounded;
  • limits produce a visible budget_exhausted result and are never silently expanded;
  • subprocesses receive an allowlisted toolchain environment rather than unrelated cloud/API credentials;
  • the client advertises no workspace-edit capability and rejects edit requests;
  • persisted evidence contains locations and relationships, not source text, server stderr, environment data, or model reasoning;
  • merge writes a caller-selected output and leaves the input graph unchanged;
  • ambiguous native-node reconciliation keeps the provider node separate instead of guessing.

Language servers are trusted external executables and some may invoke project tooling. The guide therefore requires a trusted workspace or an isolated execution environment; this PR does not represent an LSP process as a sandbox.

Provider evidence matrix

Provider Languages Validation in this PR
rust-analyzer Rust real-tool smoke + protocol tests
typescript-language-server TypeScript, JavaScript, JSX/TSX real-tool smoke + protocol tests
Pyright Python real-tool smoke + protocol tests
Eclipse JDT LS Java protocol tests; optional binary not installed in CI
JetBrains Kotlin LSP Kotlin protocol tests; optional binary not installed in CI
csharp-ls C# protocol tests; optional binary not installed in CI
gopls Go protocol tests; optional binary not installed in CI
Phpactor PHP protocol tests; optional binary not installed in CI
Ruby LSP Ruby protocol tests; optional binary not installed in CI

The distinction is intentional: a registered/profile-tested provider is not presented as real-tool interoperability proof.

Verification

  • full upstream suite with extras: 4833 passed, 51 skipped
  • focused semantic-provider suite: 28 passed
  • ruff check and ruff format --check: pass
  • Pyright: 0 errors
  • Bandit (-ll): no medium/high findings
  • sdist and wheel build: pass; the new package and console entry point are present
  • real rust-analyzer run: completed, 12 nodes / 3 edges
  • real TypeScript Language Server run: 59 nodes / 99 edges; stopped visibly at the configured 100-relationship-request budget
  • real Pyright run: completed, 74 nodes / 105 edges
  • graphify update .: completed after the code changes

Review notes

This first PR intentionally does not wire providers into graphify extract, download language servers, run providers automatically, add scanner findings, or alter Graphify's LLM semantic behavior. Those decisions can be made separately after maintainers are comfortable with the evidence contract and execution boundary.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.


Graphify review — findings

Adds an optional graphify_semantic_providers package that collects bounded, provenance-tagged language-server evidence (symbols, references, call hierarchies) and merges it into a separate graph without touching native AST extraction. Wires up the graphify-semantic CLI (list/run/merge) with an LSP client, provider registry, custom-manifest loading, and env-allowlisted shell=False subprocesses, plus protocol/merge/CLI test suites. Extends CI to lint, type-check, run graphify-semantic list, and bandit-scan the new package, and documents it in docs/SEMANTIC-PROVIDERS.md and the README.

Worth a look

  • _run selects providers via registry.for_workspace but registry defines for_pathgraphify_semantic_providers/cli.py:79 · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • CLI accepts negative run budgetsgraphify_semantic_providers/cli.py:39 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Forwarded PYTHONPATH makes workspace code importable by providersgraphify_semantic_providers/lsp.py:53 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Concurrent JSON-RPC requests can consume and drop each other's responsesgraphify_semantic_providers/lsp.py:134 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • JSON-RPC server requests can be mistaken for matching client responsesgraphify_semantic_providers/lsp.py:160 · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 160 functions depend on the 160 functions this change touches.

Health — this change adds coupling hotspots:

  • new: run_provider() — 7 callers, 10 callees
  • new: main() — 3 callers, 4 callees
  • new: _append_symbols() — 1 callers, 8 callees
  • new: _run() — 1 callers, 7 callees
  • new: _append_calls() — 1 callers, 7 callees
  • new: _append_locations() — 1 callers, 7 callees
  • new: _merge() — 1 callers, 6 callees

Verification — 160 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 160 function(s) in the blast radius were not formally verified this run

· 7 grounded finding(s) anchored inline below.

from .registry import ProviderRegistry, builtin_registry


def main(argv: list[str] | None = None) -> int:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regressionmain()

high coupling complexity (Ca·Ce = 12).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return 0


def _run(args: argparse.Namespace, registry: ProviderRegistry) -> int:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression_run()

fans out to 7 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return 0 if completed or not runs else 1


def _merge(args: argparse.Namespace) -> int:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression_merge()

fans out to 6 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return result


def run_provider(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regressionrun_provider()

fans out to 10 callees (efferent coupling); 7 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return result


def _append_symbols(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression_append_symbols()

fans out to 8 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

)


def _append_locations(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression_append_locations()

fans out to 7 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

return exhausted


def _append_calls(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression_append_calls()

fans out to 7 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optional bounded LSP semantic-provider layer for compiler-resolved graph evidence

1 participant