Add query-count perf harness + instrumentation#653
Merged
Conversation
Opt-in Kysely log hook gated behind EMDASH_QUERY_LOG=1 emits per-request NDJSON on stdout so a harness can count DB queries per route. Zero overhead when disabled. Exposed at emdash/database/instrumentation so @emdash-cms/cloudflare can wire the same hook into its per-request D1 session Kysely. Adds fixtures/perf-site (minimal blog-style fixture, dual sqlite/d1 config), scripts/query-counts.mjs (pnpm query-counts), committed snapshot files for both targets, and a CI job that runs both.
🦋 Changeset detectedLatest commit: bc3b69a The changes in this PR will be included in the next version bump. This PR includes changesets to release 11 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-playground | bc3b69a | Apr 19 2026, 06:40 AM |
Contributor
Scope checkThis PR changes 6,004 lines across 41 files. Large PRs are harder to review and more likely to be closed without review. If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs. See CONTRIBUTING.md for contribution guidelines. |
@emdash-cms/admin
@emdash-cms/auth
@emdash-cms/blocks
@emdash-cms/cloudflare
emdash
create-emdash
@emdash-cms/gutenberg-to-portable-text
@emdash-cms/x402
@emdash-cms/plugin-ai-moderation
@emdash-cms/plugin-atproto
@emdash-cms/plugin-audit-log
@emdash-cms/plugin-color
@emdash-cms/plugin-embeds
@emdash-cms/plugin-forms
@emdash-cms/plugin-webhook-notifier
commit: |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an opt-in, snapshot-based “queries per route” performance harness backed by new Kysely query instrumentation and a dedicated fixture site, and wires the harness into CI to catch query-count regressions across SQLite (node adapter) and D1 (Cloudflare).
Changes:
- Introduce core query instrumentation (
kyselyLogOption, ALS recorder, stdout NDJSON flush) and wire it into core/adapter Kysely instances. - Add a new perf fixture Astro site (
fixtures/perf-site) that can run on sqlite+node or d1+cloudflare. - Add the query-count harness + snapshots and a new CI job to run both targets on PRs.
Reviewed changes
Copilot reviewed 38 out of 41 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/query-counts.mjs | New harness to build/serve fixture, hit routes cold/warm, aggregate query-log events, and diff/update snapshots. |
| scripts/query-counts.snapshot.sqlite.json | Baseline expected query counts for sqlite target. |
| scripts/query-counts.snapshot.d1.json | Baseline expected query counts for d1 target. |
| package.json | Adds pnpm query-counts script entry. |
| .github/workflows/ci.yml | Adds query-counts CI job to run harness for sqlite + d1. |
| .gitignore | Ignores .perf-query-counts artifact. |
| pnpm-workspace.yaml | Includes fixtures/* as workspace packages. |
| pnpm-lock.yaml | Adds perf fixture importer deps; lockfile adjustments. |
| .changeset/query-counts-harness.md | Changeset documenting new instrumentation + exports. |
| packages/core/src/database/instrumentation.ts | New instrumentation module (recorder + flush + kyselyLogOption). |
| packages/core/package.json | Exports ./database/instrumentation. |
| packages/core/tsdown.config.ts | Ensures instrumentation is built/published from core. |
| packages/core/src/request-context.ts | Adds queryRecorder to ALS request context type. |
| packages/core/src/astro/middleware.ts | Attaches recorder per request when enabled and flushes at end of request. |
| packages/core/src/loader.ts | Adds log: kyselyLogOption() to loader-scoped Kysely construction. |
| packages/core/src/emdash-runtime.ts | Adds log: kyselyLogOption() to runtime-scoped Kysely construction. |
| packages/core/src/database/connection.ts | Adds log: kyselyLogOption() to direct DB factory Kysely construction. |
| packages/cloudflare/src/db/d1.ts | Wires kyselyLogOption() into request-scoped D1 session Kysely instance. |
| fixtures/perf-site/package.json | New fixture package with deps/scripts for sqlite + d1 runs. |
| fixtures/perf-site/astro.config.mjs | Conditional adapter/integration config via EMDASH_FIXTURE_TARGET. |
| fixtures/perf-site/wrangler.jsonc | Wrangler config for fixture D1/R2 + enables query log var. |
| fixtures/perf-site/tsconfig.json | Fixture TS config (node types). |
| fixtures/perf-site/src/worker.ts | Cloudflare worker entrypoint for Astro. |
| fixtures/perf-site/src/live.config.ts | Live collection config using emdashLoader(). |
| fixtures/perf-site/src/layouts/Base.astro | Base layout for fixture site; includes menus, search, theme styling. |
| fixtures/perf-site/src/styles/theme.css | Theme override CSS for fixture site. |
| fixtures/perf-site/src/utils/reading-time.ts | Reading-time utilities used by fixture pages. |
| fixtures/perf-site/src/utils/site-identity.ts | Helper to resolve site identity from settings. |
| fixtures/perf-site/src/components/PostCard.astro | Post card component for fixture pages. |
| fixtures/perf-site/src/components/TagList.astro | Tag list component for fixture pages. |
| fixtures/perf-site/src/pages/index.astro | Fixture homepage route (used by harness). |
| fixtures/perf-site/src/pages/posts/index.astro | Fixture posts listing route (used by harness). |
| fixtures/perf-site/src/pages/posts/[slug].astro | Fixture post detail route (used by harness). |
| fixtures/perf-site/src/pages/pages/[slug].astro | Fixture page detail route (used by harness). |
| fixtures/perf-site/src/pages/category/[slug].astro | Fixture category archive route (used by harness). |
| fixtures/perf-site/src/pages/tag/[slug].astro | Fixture tag archive route (used by harness). |
| fixtures/perf-site/src/pages/search.astro | Fixture search route (used by harness). |
| fixtures/perf-site/src/pages/rss.xml.ts | Fixture RSS route (used by harness). |
| fixtures/perf-site/src/pages/404.astro | Fixture 404 route. |
| fixtures/perf-site/seed/seed.json | Fixture seed data used by harness setup. |
| fixtures/perf-site/emdash-env.d.ts | Committed generated types for fixture collections. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+263
to
+271
| let resolveReady; | ||
| let rejectReady; | ||
| const ready = new Promise((res, rej) => { | ||
| resolveReady = res; | ||
| rejectReady = rej; | ||
| }); | ||
| const readyTimer = setTimeout(() => { | ||
| rejectReady(new Error(`server did not become ready within 120s (regex ${readyRe})`)); | ||
| }, 120_000); |
Comment on lines
+321
to
+329
| const r = await fetch(`${BASE}${path}`, { | ||
| method, | ||
| headers: { "x-perf-phase": phase }, | ||
| redirect: "manual", | ||
| }); | ||
| await r.arrayBuffer(); | ||
| process.stdout.write(` ${phase.padEnd(5)} ${method} ${path} -> ${r.status}\n`); | ||
| return r.status; | ||
| } catch (err) { |
Comment on lines
+16
to
+23
| // Fetch tags for each post (bylines are already hydrated by getEmDashCollection) | ||
| const postsWithTags = await Promise.all( | ||
| sortedPosts.map(async (post) => { | ||
| const tags = await getEntryTerms("posts", post.data.id, "tag"); | ||
| const bylines = post.data.bylines ?? []; | ||
| return { post, tags, bylines }; | ||
| }) | ||
| ); |
Comment on lines
+3
to
+44
| tags: Array<{ slug: string; label: string }>; | ||
| class?: string; | ||
| } | ||
|
|
||
| const { tags, class: className } = Astro.props; | ||
| --- | ||
|
|
||
| {tags.length > 0 && ( | ||
| <ul class:list={["tag-list", className]}> | ||
| {tags.map((tag) => ( | ||
| <li> | ||
| <a href={`/tag/${tag.slug}`} class="tag">{tag.label}</a> | ||
| </li> | ||
| ))} | ||
| </ul> | ||
| )} | ||
|
|
||
| <style> | ||
| .tag-list { | ||
| display: flex; | ||
| flex-wrap: wrap; | ||
| gap: var(--spacing-2); | ||
| list-style: none; | ||
| padding: 0; | ||
| margin: 0; | ||
| } | ||
|
|
||
| .tag { | ||
| display: inline-block; | ||
| padding: var(--tag-padding-y) var(--spacing-3); | ||
| font-size: var(--font-size-sm); | ||
| color: var(--color-text-secondary); | ||
| background: var(--color-surface); | ||
| border-radius: var(--radius); | ||
| text-decoration: none; | ||
| transition: color var(--transition-fast), background var(--transition-fast); | ||
| } | ||
|
|
||
| .tag:hover { | ||
| color: var(--color-text); | ||
| background: var(--color-border); | ||
| } |
| <div class="nav-links"> | ||
| { | ||
| menu?.items.map((item) => ( | ||
| <a href={item.url} target={item.target}> |
Comment on lines
+39
to
+49
| // Fetch tags for grid posts (bylines are already hydrated by getEmDashCollection) | ||
| const gridPostsWithTags = await Promise.all( | ||
| gridPosts.map(async (post) => { | ||
| const tags = await getEntryTerms("posts", post.data.id, "tag"); | ||
| const bylines = post.data.bylines ?? []; | ||
| return { | ||
| post, | ||
| tags: tags.map((t) => ({ slug: t.slug, label: t.label })), | ||
| bylines, | ||
| }; | ||
| }) |
Comment on lines
+19
to
+25
| // Fetch tags for display on each post card | ||
| const filteredPosts = await Promise.all( | ||
| posts.map(async (post) => { | ||
| const tags = await getEntryTerms("posts", post.data.id, "tag"); | ||
| return { post, tags }; | ||
| }) | ||
| ); |
Comment on lines
+19
to
+25
| // Fetch tags for display on each post card | ||
| const filteredPosts = await Promise.all( | ||
| posts.map(async (post) => { | ||
| const tags = await getEntryTerms("posts", post.data.id, "tag"); | ||
| return { post, tags }; | ||
| }) | ||
| ); |
Comment on lines
+51
to
+55
| /** | ||
| * Query recorder attached by middleware when EMDASH_QUERY_LOG_FILE is set. | ||
| * The Kysely `log` hook appends an event per query; middleware flushes | ||
| * to NDJSON after the response. | ||
| */ |
Comment on lines
+91
to
+95
| /** | ||
| * Returns a Kysely `log` option when instrumentation is enabled, or undefined. | ||
| * Pass as `new Kysely({ dialect, log: kyselyLogOption() })` so disabled mode | ||
| * has zero overhead — Kysely skips query timing entirely when `log` is absent. | ||
| */ |
Contributor
Overlapping PRsThis PR modifies files that are also changed by other open PRs:
This may cause merge conflicts or duplicated work. A maintainer will coordinate. |
pnpm exec emdash fails in CI because bin symlinks aren't linked for workspace-local packages (see scripts/relink-bins-if-needed.mjs, which early-exits under CI). Invoke the built CLI entry by absolute path instead so the harness works in both CI and local dev.
The fixture config imports from @emdash-cms/cloudflare for the d1 path, so `pnpm run --filter emdash... build` (which only walks emdash's deps, not its dependents) leaves cloudflare unbuilt and astro fails to resolve the import when loading the config.
The ready-regex approach was fragile — in CI, the cloudflare adapter's
dev mode wraps output in [vite] prefixes and the "ready in" line
sometimes never matches (observed on the D1 seed step: typegen POST
succeeded but ready timeout still fired).
TCP-connect is the real question anyway ("is the server accepting
connections?"). It also doesn't warm a fresh workerd isolate —
workerd defers isolate creation to the first HTTP request — so the
per-route cold-isolate measurement stays honest.
`astro dev` (the seed step) leaves .wrangler/deploy/ without the build-time config.json that cloudflare adapter's preview requires, so running `astro build` after the seed is what makes the subsequent `astro preview` spins work.
Merged
10 tasks
0aveRyan
pushed a commit
to 0aveRyan/emdash
that referenced
this pull request
Apr 27, 2026
* feat: add query-count perf harness + instrumentation
Opt-in Kysely log hook gated behind EMDASH_QUERY_LOG=1 emits per-request
NDJSON on stdout so a harness can count DB queries per route. Zero
overhead when disabled. Exposed at emdash/database/instrumentation so
@emdash-cms/cloudflare can wire the same hook into its per-request D1
session Kysely.
Adds fixtures/perf-site (minimal blog-style fixture, dual sqlite/d1
config), scripts/query-counts.mjs (pnpm query-counts), committed
snapshot files for both targets, and a CI job that runs both.
* fix(perf): invoke emdash CLI directly in query-counts harness
pnpm exec emdash fails in CI because bin symlinks aren't linked for
workspace-local packages (see scripts/relink-bins-if-needed.mjs, which
early-exits under CI). Invoke the built CLI entry by absolute path
instead so the harness works in both CI and local dev.
* ci(perf): build all packages for query-counts job
The fixture config imports from @emdash-cms/cloudflare for the d1 path,
so `pnpm run --filter emdash... build` (which only walks emdash's
deps, not its dependents) leaves cloudflare unbuilt and astro fails
to resolve the import when loading the config.
* fix(perf): wait for TCP port instead of parsing stdout for ready
The ready-regex approach was fragile — in CI, the cloudflare adapter's
dev mode wraps output in [vite] prefixes and the "ready in" line
sometimes never matches (observed on the D1 seed step: typegen POST
succeeded but ready timeout still fired).
TCP-connect is the real question anyway ("is the server accepting
connections?"). It also doesn't warm a fresh workerd isolate —
workerd defers isolate creation to the first HTTP request — so the
per-route cold-isolate measurement stays honest.
* fix(perf): seed D1 before building for preview
`astro dev` (the seed step) leaves .wrangler/deploy/ without the
build-time config.json that cloudflare adapter's preview requires, so
running `astro build` after the seed is what makes the subsequent
`astro preview` spins work.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds an opt-in query-count harness that measures DB queries per route against a fixture site, with separate snapshots for SQLite (node adapter) and D1 (Cloudflare via wrangler). Meant for catching per-route query regressions in PRs.
Three pieces:
Instrumentation (
emdash/database/instrumentation) — Kyselyloghook that reads a recorder from the ALS request context and emits[emdash-query-log]-prefixed NDJSON on stdout. Gated behindEMDASH_QUERY_LOG=1; zero overhead when unset (thelogoption is not passed to Kysely at all). The same helper is reused by@emdash-cms/cloudflare's per-request D1 session Kysely via a new export fromemdash.Fixture (
fixtures/perf-site) — minimal blog-style site based ontemplates/blogwith a conditionalastro.config.mjsthat switches betweennode+sqliteandcloudflare+d1viaEMDASH_FIXTURE_TARGET.Harness (
scripts/query-counts.mjs,pnpm query-counts) — builds the fixture withastro build, serves it via the prod adapter entry (node dist/server/entry.mjsfor SQLite,astro preview→ wrangler for D1), fetches each route with cold/warm phase headers, and diffs aggregate counts againstscripts/query-counts.snapshot.{sqlite,d1}.json. For D1 specifically, each route gets its own freshastro previewprocess so every cold hit lands on a genuinely fresh workerd isolate.--skip-seedand--skip-buildcompose for fast local iteration. Adist/.perf-targetmarker prevents reusing a build for the wrong target.New
query-countsjob in.github/workflows/ci.ymlruns both targets on every PR.The fixture adds `kysely` as a direct dep because Astro's node adapter externalizes it at build time — scoped to the fixture only, doesn't touch the main integration.
Type of change
Checklist
AI-generated code disclosure
Test plan