BE-618: Rank semantic searches on a quantized embedding column with per-policy-branch reads - #9124
Conversation
Adds a generated bit(3072) column plus a partial HNSW index over the combined per-entity embeddings, and reworks both search endpoints to rank candidates on it under the permission and request filters, re-score them against the full vector, and hydrate the survivors. Replaces the `Filter::CosineDistance` special case, which scanned every embedding row unindexed; keys-first entity reads are unconditional now.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
…slow-top-nav-search-query-performance # Conflicts: # libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #9124 +/- ##
==========================================
+ Coverage 59.55% 59.61% +0.06%
==========================================
Files 1408 1409 +1
Lines 137734 138283 +549
Branches 6418 6424 +6
==========================================
+ Hits 82028 82438 +410
- Misses 54711 54843 +132
- Partials 995 1002 +7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
PR SummaryMedium Risk Overview The query layer drops
Reviewed by Cursor Bugbot for commit 16d8f47. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 16d8f47. Configure here.
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1002 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1527 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 108 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 27 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |

🌟 What is the purpose of this PR?
Top-nav semantic search took 16s+ locally (BE-618). The generic filter path sorted all entities by exact cosine distance before cutting to the limit, and the cross-table permission disjunction forced the planner to materialize the whole visible set.
This PR makes the search index-driven: entities are ranked on a new binary-quantized embedding column via a partial HNSW index, permission policies are decomposed into one conjunctive branch per permit (each individually plannable), and the candidates are re-scored against the full vector before hydration. Measurements and query plans are documented in BE-618.
🔗 Related links
🚫 Blocked by
main.🔍 What does this change?
Commit 1 — quantized ranking (
2a1ab997):V57: generatedembedding_bits bit(3072)column (binary_quantize, 392B inline vs 12kB TOASTed vector) onentity_embeddingsandentity_type_embeddings, plus a partial HNSW index (bit_hamming_ops,WHERE property IS NULL) on entities; shadow migrations updatedBinaryOperator::HammingDistance(<~>),Function::BinaryQuantize,PostgresType::BitSelectCompiler::rank_by_quantized_distance+restrict_embedding_property: ranks on the quantized column with anINNERembeddings join, forbids cursors, overfetcheslimit × 4candidatessearch_entities/search_entity_types: ranked key-read + exact<=>rerank with the distance threshold, then hydration with rank restore, in oneREPEATABLE READtransaction;SET LOCAL hnsw.ef_searchsized to the candidate pool withiterative_scan = relaxed_orderFilter::CosineDistanceremoved entirely (compile arm, error variants,Distancepseudo-columns, keys-first gate); snapshot restore switched to explicit column lists (generated columns rejectINSERT ... SELECT *)Commit 2 — policy branches (
HEAD):Filter::for_policy_branches: splits the permit disjunction at construction (no tree parsing), one branch per permit with all forbids conjoined; duplicate and unmatchable permits produce no branch;for_policiesre-expressed over the same parts extraction with byte-identical outputsearch_entities_implmoved toknowledge/entity/search.rsand restructured: one candidate key-read per branch, union-dedup, exact rerank overunnestarrays with deterministic tie-breaking, divergence warning if hydration disagrees with the ranking, branch count recorded on the tracing spanPre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
limit × 4positions away from its exact rank can differ. The binary-quantization recall has not been measured yet.limit × 4and miss a nearer neighbour. Deduplicating before the limit is part of BE-735, which rewrites the reads into one statement.search_entity_typeskeeps the single-statement shape:entity_type_embeddingshas no HNSW index yet and stays small. It also ranks and hydrates without a transaction, unlike the entity search — BE-738, which needs a trait change the compiler does not allow today.🐾 Next steps
UNION+ shared parameter registry, rerank composed from the AST, single statement per searchweb_id = ANY(...)🛡 What tests cover this?
tests/graph/integration/postgres/semantic_search.rs(new): 7 tests against constructed ground truth (exact cosine distances 0/0.5/1/2) — ranking with rank restore, distance threshold, limit, actor isolation, cross-branch deduplication via entity-scoped policies, draft handling, request filtersfilter::tests::policy_conversion: 15 unit tests incl. a property test comparingfor_policy_branchesagainstfor_policiesover a synthetic universelibs/@local/graph/postgres-store/tests/semantic_search/main.rs: ignored perf harness against a seeded database (measures, does not assert)❓ How to test this?
cargo nextest run --package hash-graph-integration --test postgres semantic_search(requires a freshly migrated database)cargo run --bin hash-graph -- server) andPOST /entities/searchwith anembedding,maximumSemanticDistanceandlimit— results are ordered by ascending distanceEXPLAIN ANALYZEon a branch statement shows the HNSW index driving broad branches and conventional index scans on selective ones📹 Demo
Plans and measurement details are documented in the Linear issues.