sync: standalone CLI engine with SQLite + bulk GraphQL

[AntonNiklasson]

Part of #72.

Adds packages/sync — a standalone sync engine for github-dashboard with both a library API and a CLI on top. The server, web, and desktop packages are untouched; phase B of #72 will replace the server's cache.ts + fetchers.ts by importing this package.

Also drops the @github-dashboard/ prefix from all workspace package names (server, web, desktop, sync) so pnpm --filter ... invocations get shorter.

Public API (what the server will use)

import {
  openCache,
  createSqliteRepository,
  createSyncEngine,
  printSummary,
} from "sync";

const { db } = openCache();              // raw SQLite + version check
const repo = createSqliteRepository(db); // Repository on top of the db
const engine = createSyncEngine({ repo });

await engine.runOnce();                  // single cycle

engine.start({
  intervalMs: 25_000,
  onCycle: printSummary,
  onError: (err) => { /* outer/structural failures */ },
});
// ...
await engine.stop();                     // wakes sleep, drains in-flight cycle
db.close();

createSyncEngine accepts an optional loadInstances for tests. reconcileInstances is exported as a standalone primitive. One entry point per concern: openCache for the file, createSqliteRepository for the storage contract, createSyncEngine for orchestration.

Engine internals

• SQLite cache at $XDG_CACHE_HOME/github-dashboard/cache.sqlite (default ~/.cache/...). WAL mode, integer schema_version in a meta table, wipe-and-recreate on mismatch in either direction. Pure regenerable state, so it lives under cache, not data.
• Repository interface — providers, orchestrator, and CLI depend on it, not the raw Database handle. createSqliteRepository(db) builds prepared statements once at factory time and closes over them (no re-preparing per call). Transactions for atomic replacePrs / replaceNotifications live inside.
• Bulk GraphQL for both PR categories — one round trip per category per instance, returning status checks, reviews, threads, auto-merge, merge queue, labels, etc. in a single response.
• Conditional notifications via REST with If-None-Match → cheap 304s when nothing changed.
• Canonical client-shape payload stored in a json_valid TEXT column, identical to what the existing web client renders today.
• Reconciliation runs at the top of each cycle: new instances in config.yml get inserted, removed ones cascade-delete from prs / notifications / sync_state.
• Rate-limit floor (200): the engine refuses to fetch when stored headroom for an (instance, kind) is below the floor and reset is still in the future.

CLI

pnpm sync once                                  # one cycle, all categories
pnpm sync once --kind prs                       # one cycle, prs only
pnpm sync once --instance github-com            # one cycle, one instance
pnpm sync loop                                  # background loop, 25s sleep between cycles
pnpm sync loop --interval 60                    # background loop, 60s sleep
pnpm sync status                                # cache path, schema version, row counts, last sync, rate state
pnpm sync wipe                                  # delete cache.sqlite + sidecars

loop calls engine.start + waits for SIGINT + calls engine.stop — stop interrupts sleep immediately and drains the in-flight cycle.

Validation

Against a local config with two instances (github.com + a GHES instance):

sync cycle 2026-05-27T07:00:57Z (9.6s)
  instance: github-com
    authored           3 rows (rate 4707)
    review_requested   3 rows (rate 4699)
    notifications      304 not modified
  instance: <ghes>
    authored           4 rows (rate 4782)
    review_requested  29 rows (rate 4781)
    notifications      304 not modified

• GraphQL rateLimit.cost observed: 4 on github.com per PR query, lower on the GHES instance. At a 25s-after-finished cadence (~144 cycles/h) that's ~12% of the 5000/h GraphQL bucket on github.com.
• ETag round-trip confirmed on notifications: second sync returns 304 with no body.
• Idempotent across runs (same row counts).
• Payload shape verified by JSON-dumping a row — fields and casing match what the server emits today.
• pnpm sync loop --interval 2 + SIGINT cleanly stops the engine mid-sleep and drains in-flight.

Known limitations

• Team-based auto-assigned detection is coarse. Team.slug requires read:org scope, which most PATs don't have. The query drops Team identification and falls back to "any auto-actor event in the timeline" for team-attached PRs. Direct user-level auto-assignment is exact.
• No pagination beyond first 100 results per search. If anyone has >100 open authored or review-requested PRs, the tail gets truncated.
• No Retry-After parsing or exponential backoff yet. Only the rate-limit floor guard.

Test plan

• pnpm typecheck clean across all 4 packages
• pnpm test — 14 (sync) + 145 (server) + 39 (web) = 198 passing
• pnpm lint clean
• pnpm fmt:check clean
• pnpm sync once, pnpm sync once --kind <each>, pnpm sync once --instance <id>, pnpm sync loop --interval <n> with SIGINT, pnpm sync status, pnpm sync wipe
• Engine lifecycle tests cover start/stop idempotency, stop interrupting sleep, reconcile add/remove/edit
• Public-surface smoke test (src/index.test.ts) imports only from ./index.js and exercises the full server-adoption flow
• Cache version-check unit tests cover first-open, reuse, wipe-on-mismatch, file-removal

[Copilot]

Pull request overview

This PR introduces a new standalone sync engine (packages/sync) that persists state to a local SQLite cache and fetches PRs/reviews in bulk via GraphQL, while also renaming workspace packages to drop the @github-dashboard/ prefix and updating scripts/workflows accordingly.

Changes:

• Added packages/sync: CLI (ghd-sync), SQLite cache bootstrap/versioning, and GitHub provider fetchers (GraphQL for PRs/reviews; REST+ETag for notifications).
• Renamed workspace package names (server, web, desktop, sync) and updated root scripts + CI workflows to use shorter pnpm --filter targets.
• Updated lockfile and build config to support better-sqlite3 (including pnpm.onlyBuiltDependencies).

Reviewed changes

Copilot reviewed 26 out of 27 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
pnpm-lock.yaml	Adds deps for sync package (incl. better-sqlite3) and workspace name updates.
package.json	Updates pnpm --filter scripts and adds sync script + onlyBuiltDependencies entry for better-sqlite3.
packages/web/package.json	Renames workspace package to web.
packages/server/package.json	Renames workspace package to server.
packages/desktop/package.json	Renames workspace package to desktop and updates workspace dependency name.
packages/desktop/src/main.ts	Updates comment to reflect renamed desktop package.
.github/workflows/desktop-ci.yml	Updates pnpm filters to desktop.
.github/workflows/desktop-release.yml	Updates pnpm filters to desktop.
packages/sync/package.json	Adds new sync workspace package metadata, scripts, bin, and exports.
packages/sync/tsconfig.json	TypeScript config for the sync package.
packages/sync/vitest.config.ts	Vitest config for sync tests.
packages/sync/src/index.ts	Placeholder entrypoint for package exports.
packages/sync/src/cli.ts	Implements ghd-sync CLI commands: run/status/wipe.
packages/sync/src/sync.ts	Orchestrates reconciliation + per-instance fetches with a rate-limit floor guard and watch mode.
packages/sync/src/config.ts	Loads config.yml instances and derives API base URL + username.
packages/sync/src/cache/path.ts	Resolves SQLite cache path using XDG cache conventions.
packages/sync/src/cache/schema.ts	Declares schema version and DDL (WAL, tables, indexes, json_valid payload).
packages/sync/src/cache/open.ts	Opens SQLite DB, checks schema version, and wipes/recreates on mismatch.
packages/sync/src/cache/open.test.ts	Unit tests for cache open/wipe/version behavior.
packages/sync/src/cache/store.ts	DB store helpers for instances, PRs, notifications, and sync_state.
packages/sync/src/providers/github/client.ts	Shared Octokit client cache for GitHub API calls.
packages/sync/src/providers/github/queries.ts	GraphQL fragments/queries and response typings for PR search + review search.
packages/sync/src/providers/github/normalize.ts	Normalizes GraphQL PR nodes into the canonical client payload shape.
packages/sync/src/providers/github/fetchPrs.ts	Fetches authored PRs via bulk GraphQL and persists rows/payloads.
packages/sync/src/providers/github/fetchReviews.ts	Fetches review-requested PRs via bulk GraphQL incl. timeline heuristic and persists rows/payloads.
packages/sync/src/providers/github/fetchNotifications.ts	Fetches notifications via REST with ETag, filters redundancies, and persists rows/state.
packages/sync/src/providers/github/notificationUrl.ts	Converts notification API subject URLs into HTML URLs.

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 28 out of 29 changed files in this pull request and generated 6 comments.

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

[Copilot]

Pull request overview

Copilot reviewed 29 out of 30 changed files in this pull request and generated 6 comments.

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

[Copilot]

Pull request overview

Copilot reviewed 29 out of 30 changed files in this pull request and generated 6 comments.

Files not reviewed (1)

• pnpm-lock.yaml: Language not supported

[Copilot]

Pull request overview

Copilot reviewed 29 out of 30 changed files in this pull request and generated 2 comments.

Files not reviewed (1)

• pnpm-lock.yaml: Generated file

[Copilot]

Copilot reviewed 29 out of 30 changed files in this pull request and generated no new comments.

Files not reviewed (1)

• pnpm-lock.yaml: Generated file