Skip to content

BE-618: Rank semantic searches on a quantized embedding column with per-policy-branch reads - #9124

Open
TimDiekmann wants to merge 5 commits into
mainfrom
t/be-618-investigate-slow-top-nav-search-query-performance
Open

BE-618: Rank semantic searches on a quantized embedding column with per-policy-branch reads#9124
TimDiekmann wants to merge 5 commits into
mainfrom
t/be-618-investigate-slow-top-nav-search-query-performance

Conversation

@TimDiekmann

@TimDiekmann TimDiekmann commented Jul 30, 2026

Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

Top-nav semantic search took 16s+ locally (BE-618). The generic filter path sorted all entities by exact cosine distance before cutting to the limit, and the cross-table permission disjunction forced the planner to materialize the whole visible set.

This PR makes the search index-driven: entities are ranked on a new binary-quantized embedding column via a partial HNSW index, permission policies are decomposed into one conjunctive branch per permit (each individually plannable), and the candidates are re-scored against the full vector before hydration. Measurements and query plans are documented in BE-618.

🔗 Related links

  • BE-618 — the investigation
  • BE-735 — follow-up merging the per-branch reads into one statement (stacked PR, to be merged together with this one)
  • BE-734 — follow-up collapsing structurally identical web permits
  • BE-732 — production embedding backfill (only 31k of 726k prod entities have embeddings)

🚫 Blocked by

  • Nothing. The BE-735 follow-up PR is stacked on this branch and follows it into main.

🔍 What does this change?

Commit 1 — quantized ranking (2a1ab997):

  • Migration V57: generated embedding_bits bit(3072) column (binary_quantize, 392B inline vs 12kB TOASTed vector) on entity_embeddings and entity_type_embeddings, plus a partial HNSW index (bit_hamming_ops, WHERE property IS NULL) on entities; shadow migrations updated
  • SQL-AST: BinaryOperator::HammingDistance (<~>), Function::BinaryQuantize, PostgresType::Bit
  • SelectCompiler::rank_by_quantized_distance + restrict_embedding_property: ranks on the quantized column with an INNER embeddings join, forbids cursors, overfetches limit × 4 candidates
  • search_entities / search_entity_types: ranked key-read + exact <=> rerank with the distance threshold, then hydration with rank restore, in one REPEATABLE READ transaction; SET LOCAL hnsw.ef_search sized to the candidate pool with iterative_scan = relaxed_order
  • Filter::CosineDistance removed entirely (compile arm, error variants, Distance pseudo-columns, keys-first gate); snapshot restore switched to explicit column lists (generated columns reject INSERT ... SELECT *)

Commit 2 — policy branches (HEAD):

  • Filter::for_policy_branches: splits the permit disjunction at construction (no tree parsing), one branch per permit with all forbids conjoined; duplicate and unmatchable permits produce no branch; for_policies re-expressed over the same parts extraction with byte-identical output
  • search_entities_impl moved to knowledge/entity/search.rs and restructured: one candidate key-read per branch, union-dedup, exact rerank over unnest arrays with deterministic tie-breaking, divergence warning if hydration disagrees with the ranking, branch count recorded on the tracing span
  • New compile goldens (branch shape, end-to-end branch split) and a property test asserting the branch union selects the same entities as the combined filter across 144 policy configurations

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

  • Behaviour change: results were previously exact over all entities; they are now approximate-then-exact — an entity whose quantized rank is more than limit × 4 positions away from its exact rank can differ. The binary-quantization recall has not been measured yet.
  • The per-branch reads run sequentially (N+3 round trips per search) until the stacked BE-735 PR lands.
  • The candidate pool counts rows duplicated by to-many filter joins, so the effective pool can fall below limit × 4 and miss a nearer neighbour. Deduplicating before the limit is part of BE-735, which rewrites the reads into one statement.
  • Actors with many webs get one branch per web role (BE-734); duplicate policies in prod (BE-696) would each cost a read — identical permits are deduplicated at branch assembly as a stopgap.
  • search_entity_types keeps the single-statement shape: entity_type_embeddings has no HNSW index yet and stays small. It also ranks and hydrates without a transaction, unlike the entity search — BE-738, which needs a trait change the compiler does not allow today.

🐾 Next steps

  • BE-735: statement-layer UNION + shared parameter registry, rerank composed from the AST, single statement per search
  • BE-734: collapse structurally identical web permits into web_id = ANY(...)
  • BE-732: production embedding backfill
  • Measure binary-quantization recall against exact ranking; the overfetch factor 4 is unmeasured

🛡 What tests cover this?

  • tests/graph/integration/postgres/semantic_search.rs (new): 7 tests against constructed ground truth (exact cosine distances 0/0.5/1/2) — ranking with rank restore, distance threshold, limit, actor isolation, cross-branch deduplication via entity-scoped policies, draft handling, request filters
  • filter::tests::policy_conversion: 15 unit tests incl. a property test comparing for_policy_branches against for_policies over a synthetic universe
  • Compile goldens pinning the ranked statement shapes, parameters, and the branch split end to end
  • libs/@local/graph/postgres-store/tests/semantic_search/main.rs: ignored perf harness against a seeded database (measures, does not assert)

❓ How to test this?

  1. Run the integration tests: cargo nextest run --package hash-graph-integration --test postgres semantic_search (requires a freshly migrated database)
  2. With a seeded database: start the graph (cargo run --bin hash-graph -- server) and POST /entities/search with an embedding, maximumSemanticDistance and limit — results are ordered by ascending distance
  3. EXPLAIN ANALYZE on a branch statement shows the HNSW index driving broad branches and conventional index scans on selective ones

📹 Demo

Plans and measurement details are documented in the Linear issues.

Adds a generated bit(3072) column plus a partial HNSW index over the
combined per-entity embeddings, and reworks both search endpoints to
rank candidates on it under the permission and request filters, re-score
them against the full vector, and hydrate the survivors. Replaces the
`Filter::CosineDistance` special case, which scanned every embedding
row unindexed; keys-first entity reads are unconditional now.
@TimDiekmann TimDiekmann self-assigned this Jul 30, 2026
@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview Jul 31, 2026 6:18pm
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Jul 31, 2026 6:18pm
petrinaut Skipped Skipped Jul 31, 2026 6:18pm

@github-actions github-actions Bot added area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/tests New or updated tests labels Jul 30, 2026
…slow-top-nav-search-query-performance

# Conflicts:
#	libs/@local/graph/postgres-store/src/store/postgres/knowledge/entity/mod.rs
@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 62.39669% with 273 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.61%. Comparing base (639beae) to head (16d8f47).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...tore/src/store/postgres/knowledge/entity/search.rs 0.00% 196 Missing ⚠️
.../graph/postgres-store/src/snapshot/entity/batch.rs 0.00% 24 Missing ⚠️
libs/@local/graph/store/src/filter/mod.rs 95.82% 15 Missing and 2 partials ⚠️
...s-store/src/snapshot/ontology/entity_type/batch.rs 0.00% 15 Missing ⚠️
...s-store/src/store/postgres/ontology/entity_type.rs 0.00% 11 Missing ⚠️
...rc/store/postgres/query/ast/expression/function.rs 50.00% 4 Missing ⚠️
...h/postgres-store/src/store/postgres/query/table.rs 33.33% 4 Missing ⚠️
...gres-store/src/store/postgres/query/compile/mod.rs 97.95% 0 Missing and 1 partial ⚠️
...es-store/src/store/postgres/query/postgres_type.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9124      +/-   ##
==========================================
+ Coverage   59.55%   59.61%   +0.06%     
==========================================
  Files        1408     1409       +1     
  Lines      137734   138283     +549     
  Branches     6418     6424       +6     
==========================================
+ Hits        82028    82438     +410     
- Misses      54711    54843     +132     
- Partials      995     1002       +7     
Flag Coverage Δ
apps.hash-ai-worker-ts 1.99% <ø> (ø)
apps.hash-api 12.09% <ø> (ø)
local.hash-backend-utils 2.55% <ø> (ø)
local.hash-graph-sdk 10.02% <ø> (ø)
local.hash-isomorphic-utils 6.37% <ø> (+0.62%) ⬆️
rust.hash-graph-api 7.37% <ø> (ø)
rust.hash-graph-postgres-store 29.33% <19.74%> (-0.34%) ⬇️
rust.hashql-compiletest 28.39% <ø> (ø)
rust.hashql-eval 79.82% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vercel
vercel Bot temporarily deployed to Preview – petrinaut July 30, 2026 16:19 Inactive
@TimDiekmann
TimDiekmann marked this pull request as ready for review July 31, 2026 17:39
@TimDiekmann
TimDiekmann requested a review from a team as a code owner July 31, 2026 17:39
Copilot AI review requested due to automatic review settings July 31, 2026 17:39
@cursor

cursor Bot commented Jul 31, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes core semantic search and authorization filtering paths with approximate-then-exact ranking; behaviour can differ from prior full scans, and search cost scales with policy branch count until follow-up work lands.

Overview
Semantic entity search is reworked so ranking can use Postgres HNSW instead of scanning all embeddings under a single permission disjunction. Migration V57 (and shadow migrations) add a stored embedding_bits column (binary_quantize on 3072-d vectors) on entity_embeddings and entity_type_embeddings, plus a partial HNSW index on combined entity embeddings (property IS NULL). Snapshot restore omits generated columns and the HNSW index on staging tables.

The query layer drops Filter::CosineDistance and distance pseudo-columns. SelectCompiler::rank_by_quantized_distance and restrict_embedding_property rank on Hamming distance over embedding_bits, with a 4× candidate overfetch before exact cosine rerank. Entity search_entities runs in a read-only transaction: one quantized candidate read per Filter::for_policy_branches permit branch, union/dedup, full-vector rerank, then hydration with rank restored; HNSW scan settings are tuned per transaction. search_entity_types uses the same quantize-then-rerank pattern without branch splitting.

Filter::for_policy_branches splits permits into conjunctive branches (each permit plus all forbids); for_policies is refactored to share the same parts extraction. Integration and compile tests cover ranking, policies, and snapshot inserts.

Reviewed by Cursor Bugbot for commit 16d8f47. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Comment thread libs/@local/graph/postgres-store/src/snapshot/entity/batch.rs
Copilot AI review requested due to automatic review settings July 31, 2026 18:04
@vercel
vercel Bot temporarily deployed to Preview – petrinaut July 31, 2026 18:04 Inactive

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 16d8f47. Configure here.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$26.4 \mathrm{ms} \pm 217 \mathrm{μs}\left({\color{gray}0.609 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.84 \mathrm{ms} \pm 16.4 \mathrm{μs}\left({\color{gray}0.979 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1002 $$11.9 \mathrm{ms} \pm 73.6 \mathrm{μs}\left({\color{gray}0.544 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$38.6 \mathrm{ms} \pm 314 \mathrm{μs}\left({\color{gray}-0.478 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$11.7 \mathrm{ms} \pm 76.3 \mathrm{μs}\left({\color{gray}-1.251 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1527 $$21.4 \mathrm{ms} \pm 214 \mathrm{μs}\left({\color{gray}0.050 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$27.1 \mathrm{ms} \pm 196 \mathrm{μs}\left({\color{gray}-1.757 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.14 \mathrm{ms} \pm 16.0 \mathrm{μs}\left({\color{gray}0.355 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$12.7 \mathrm{ms} \pm 92.2 \mathrm{μs}\left({\color{gray}-1.751 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.15 \mathrm{ms} \pm 21.9 \mathrm{μs}\left({\color{gray}-0.841 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.45 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.581 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 52 $$2.78 \mathrm{ms} \pm 18.6 \mathrm{μs}\left({\color{gray}1.61 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$4.46 \mathrm{ms} \pm 42.6 \mathrm{μs}\left({\color{gray}1.71 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$2.91 \mathrm{ms} \pm 17.5 \mathrm{μs}\left({\color{gray}-0.080 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 108 $$3.47 \mathrm{ms} \pm 19.8 \mathrm{μs}\left({\color{gray}1.18 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$3.75 \mathrm{ms} \pm 22.4 \mathrm{μs}\left({\color{gray}0.287 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.87 \mathrm{ms} \pm 18.5 \mathrm{μs}\left({\color{gray}2.07 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$3.38 \mathrm{ms} \pm 23.6 \mathrm{μs}\left({\color{gray}-0.206 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.15 \mathrm{ms} \pm 11.8 \mathrm{μs}\left({\color{gray}0.627 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.03 \mathrm{ms} \pm 11.3 \mathrm{μs}\left({\color{gray}0.044 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 2 $$2.11 \mathrm{ms} \pm 11.6 \mathrm{μs}\left({\color{gray}-0.709 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$2.36 \mathrm{ms} \pm 18.0 \mathrm{μs}\left({\color{gray}0.028 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.18 \mathrm{ms} \pm 12.9 \mathrm{μs}\left({\color{gray}-0.506 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$2.33 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.780 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$2.48 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.643 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.22 \mathrm{ms} \pm 11.0 \mathrm{μs}\left({\color{gray}-0.388 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 26 $$2.45 \mathrm{ms} \pm 19.7 \mathrm{μs}\left({\color{gray}4.59 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$2.79 \mathrm{ms} \pm 18.5 \mathrm{μs}\left({\color{gray}0.204 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$2.42 \mathrm{ms} \pm 16.5 \mathrm{μs}\left({\color{gray}0.640 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 27 $$2.64 \mathrm{ms} \pm 14.1 \mathrm{μs}\left({\color{gray}1.95 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$2.71 \mathrm{ms} \pm 16.3 \mathrm{μs}\left({\color{gray}-0.556 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.39 \mathrm{ms} \pm 13.3 \mathrm{μs}\left({\color{gray}1.39 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$2.70 \mathrm{ms} \pm 22.1 \mathrm{μs}\left({\color{gray}3.64 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$37.1 \mathrm{ms} \pm 208 \mathrm{μs}\left({\color{gray}-0.655 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$29.0 \mathrm{ms} \pm 246 \mathrm{μs}\left({\color{gray}-0.448 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$30.4 \mathrm{ms} \pm 156 \mathrm{μs}\left({\color{gray}-1.183 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$28.1 \mathrm{ms} \pm 183 \mathrm{μs}\left({\color{gray}3.03 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$37.1 \mathrm{ms} \pm 281 \mathrm{μs}\left({\color{gray}2.21 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$44.3 \mathrm{ms} \pm 179 \mathrm{μs}\left({\color{gray}-0.476 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$35.1 \mathrm{ms} \pm 188 \mathrm{μs}\left({\color{gray}0.309 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$81.4 \mathrm{ms} \pm 429 \mathrm{μs}\left({\color{gray}-1.023 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$29.3 \mathrm{ms} \pm 204 \mathrm{μs}\left({\color{lightgreen}-34.219 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$267 \mathrm{ms} \pm 1.35 \mathrm{ms}\left({\color{gray}-2.318 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$9.43 \mathrm{ms} \pm 63.0 \mathrm{μs}\left({\color{gray}0.726 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$9.35 \mathrm{ms} \pm 48.6 \mathrm{μs}\left({\color{gray}-1.035 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$9.41 \mathrm{ms} \pm 53.9 \mathrm{μs}\left({\color{gray}1.04 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$9.38 \mathrm{ms} \pm 62.9 \mathrm{μs}\left({\color{gray}0.859 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$9.17 \mathrm{ms} \pm 42.3 \mathrm{μs}\left({\color{gray}-1.953 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$9.25 \mathrm{ms} \pm 55.2 \mathrm{μs}\left({\color{gray}0.408 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$9.37 \mathrm{ms} \pm 47.8 \mathrm{μs}\left({\color{gray}-1.714 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$9.31 \mathrm{ms} \pm 54.5 \mathrm{μs}\left({\color{gray}0.259 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$9.38 \mathrm{ms} \pm 66.7 \mathrm{μs}\left({\color{gray}-3.611 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$9.80 \mathrm{ms} \pm 58.9 \mathrm{μs}\left({\color{gray}1.40 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$9.54 \mathrm{ms} \pm 63.7 \mathrm{μs}\left({\color{gray}-2.621 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$9.56 \mathrm{ms} \pm 50.6 \mathrm{μs}\left({\color{gray}-1.842 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$9.63 \mathrm{ms} \pm 58.5 \mathrm{μs}\left({\color{gray}-0.284 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$9.69 \mathrm{ms} \pm 55.4 \mathrm{μs}\left({\color{gray}-0.772 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$9.66 \mathrm{ms} \pm 50.0 \mathrm{μs}\left({\color{gray}-1.887 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$9.66 \mathrm{ms} \pm 47.1 \mathrm{μs}\left({\color{gray}-1.729 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$9.66 \mathrm{ms} \pm 53.1 \mathrm{μs}\left({\color{gray}0.577 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$9.66 \mathrm{ms} \pm 48.1 \mathrm{μs}\left({\color{gray}-4.144 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$9.84 \mathrm{ms} \pm 67.1 \mathrm{μs}\left({\color{gray}0.677 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$7.21 \mathrm{ms} \pm 37.6 \mathrm{μs}\left({\color{gray}3.16 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$52.0 \mathrm{ms} \pm 356 \mathrm{μs}\left({\color{lightgreen}-7.768 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$96.5 \mathrm{ms} \pm 503 \mathrm{μs}\left({\color{gray}-2.291 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$58.8 \mathrm{ms} \pm 349 \mathrm{μs}\left({\color{gray}-4.343 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$66.9 \mathrm{ms} \pm 461 \mathrm{μs}\left({\color{gray}-4.912 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$75.1 \mathrm{ms} \pm 394 \mathrm{μs}\left({\color{gray}-2.956 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$80.4 \mathrm{ms} \pm 469 \mathrm{μs}\left({\color{gray}-2.011 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$41.1 \mathrm{ms} \pm 245 \mathrm{μs}\left({\color{gray}-0.923 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$64.6 \mathrm{ms} \pm 319 \mathrm{μs}\left({\color{gray}-1.136 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$46.5 \mathrm{ms} \pm 226 \mathrm{μs}\left({\color{gray}-0.428 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$54.9 \mathrm{ms} \pm 515 \mathrm{μs}\left({\color{gray}0.288 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$56.2 \mathrm{ms} \pm 378 \mathrm{μs}\left({\color{gray}-0.609 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$56.0 \mathrm{ms} \pm 391 \mathrm{μs}\left({\color{gray}-0.620 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$108 \mathrm{ms} \pm 719 \mathrm{μs}\left({\color{red}5.76 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$118 \mathrm{ms} \pm 506 \mathrm{μs}\left({\color{red}5.16 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$16.3 \mathrm{ms} \pm 95.3 \mathrm{μs}\left({\color{gray}-2.372 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$477 \mathrm{ms} \pm 1.04 \mathrm{ms}\left({\color{gray}0.089 \mathrm{\%}}\right) $$ Flame Graph

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants