Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Development contracts:
| Read-only database queries | `cdidx status --db /artifacts/codeindex.db --read-only --json`; `cdidx search AuthService --db /artifacts/codeindex.db --immutable` | Query commands accept `--read-only` (alias `--immutable`) to open an existing CodeIndex database through SQLite's immutable read-only URI mode. Use this for CI artifacts, mounted caches, and sandboxes where creating or updating `codeindex.db-wal` / `codeindex.db-shm` sidecars is not allowed. |
| Mutating commands | `index`, `backfill-fold`, `optimize`, `vacuum` | These require writable storage and reject read-only database opens. |
| Reusable index artifact | `cdidx export codeindex.cdidx.zip`; `cdidx import codeindex.cdidx.zip --db <path>`; `cdidx import codeindex.cdidx.zip --dry-run --json` | Run export after indexing and upload the archive. Export refuses an existing destination unless `--overwrite` is explicit, publishes atomically from an owner-only temporary file, and verifies POSIX mode `0600`. Successful export JSON adds final archive byte size and SHA-256 plus the complete immutable manifest while retaining the prior result fields. Consumers import before query commands, or use `--dry-run` / `--check` to validate the archive without replacing the destination DB. Use `--prune-paths` when the archive comes from another checkout and the restored DB should advertise the import target project root; imports targeting `.../.cdidx/codeindex.db` use the sibling project directory, while other DB paths fall back to the process current directory. The archive contains only `manifest.json` plus `codeindex.db`; import validates ZIP entry names through `ZipArchiveSafetyPolicy` and rejects absolute, parent-directory, backslash, NUL, non-canonical, duplicate, and extra entries before extraction. The manifest carries bounded summary/readiness metadata including row counts, readiness bits, writer/indexed-head metadata, schema contract stamps, and unknown-extension summary when available. Import validates manifest format, manifest `user_version`, `database_sha256`, present summary counts, and the embedded SQLite file as a CodeIndex database before replacing the destination DB. Import rejects archive `codeindex.db` entries whose compressed or uncompressed metadata exceeds 8 GiB, and the extraction stream is also capped at 8 GiB. |
| Maintenance checkpoint | `cdidx db checkpoint <name> [--dry-run]`; `cdidx db checkpoints --list|--delete <name>|--prune --keep <n> [--dry-run]`; `cdidx db restore <name> [--dry-run]`; `cdidx db restore-backups --list|--prune --keep <n> [--dry-run]` | Checkpoint snapshots `codeindex.db` plus existing WAL/SHM sidecars before risky maintenance. Checkpoint creation `--dry-run` reports the source files and byte count without creating a directory. Checkpoint delete/prune and restore-backup prune require an explicit mutation action; their `--dry-run` forms make no filesystem changes and report the exact paths that would be deleted and retained. Checkpoint prune retains the newest `<n>` valid snapshots, and skips all deletion if its bounded 1,000-directory scan is truncated. Restore `--dry-run` validates the bounded manifest, regular-file payload paths, required staging bytes, and destination free space without replacing the DB; a real restore runs the same validation first. Checkpoint manifests record only the database file name, not the local absolute DB path. Restore rolls back and keeps pre-restore files under `<db>.restore-backup-<timestamp>/`. Checkpoints live next to the DB under `<db>.checkpoints/<name>/`. `backfill-fold` creates an automatic checkpoint before it mutates rows unless `--no-checkpoint` is passed; automatic checkpoint names include a UTC timestamp plus random suffix to avoid same-millisecond collisions. |
| Maintenance checkpoint and managed rollback | `cdidx db checkpoint <name> [--dry-run]`; `cdidx db checkpoints --list|--delete <name>|--prune --keep <n> [--dry-run]`; `cdidx db restore <name> [--dry-run] [--no-backup]`; `cdidx db restore-backups --list|--prune --keep <n>|--restore <id> [--dry-run] [--no-backup]` | Checkpoint snapshots `codeindex.db` plus existing WAL/SHM sidecars before risky maintenance. Import and both restore forms create a consistent, verified managed SQLite rollback snapshot before replacing an existing DB unless `--no-backup` is explicit. Managed directories use `<db>.restore-backup-<id>/` and contain a bounded manifest plus one standalone database payload; the manifest records SHA-256, byte count, supported `user_version`, provenance, and an optional source identifier, but no local absolute source path. `restore-backups --list` exposes the ID and provenance while retaining legacy directory metadata; existing prune retention remains compatible. `restore-backups --restore <id>` revalidates the directory boundary, manifest, payload hash, schema, and combined staging/rollback free space, then performs an atomic replacement with transient rollback-on-failure. Its `--dry-run` reports every validation and planned backup without mutation. Checkpoint delete/prune and restore-backup prune require an explicit mutation action, and checkpoint prune skips all deletion if its bounded 1,000-directory scan is truncated. Checkpoints live under `<db>.checkpoints/<name>/`. `backfill-fold` creates an automatic checkpoint before row mutation unless `--no-checkpoint` is passed. |
| Binary compatibility | [COMPATIBILITY.md](COMPATIBILITY.md) | Database compatibility across `cdidx` binary upgrades and downgrades is documented there. Keep that policy updated whenever readiness bits, `codeindex_meta` contract stamps, or rebuild requirements change. |
| Fold backfill preview and recovery | `backfill-fold --dry-run`; MCP `backfill_fold` with `dry_run: true` or `force: true` | Dry-run previews folded-key rows without mutating the DB or stamping FoldReady. MCP accepts the same preview and can force rewriting all folded keys when an operator needs to recover from suspicious fold metadata or row state even though the stored version/fingerprint appears current. Non-dry-run row rewrites are resumable after interruption: completed row updates remain durable, and final FoldReady metadata is stamped only after verification succeeds. MCP responses include `progress.rows_done`, `progress.rows_total`, and `progress.fraction` so clients can report and retry long backfills. |

Expand Down Expand Up @@ -3273,7 +3273,7 @@ net9 CI lane に合わせる場合は `FRAMEWORK=net9.0 make test` を使いま
| read-only database query | `cdidx status --db /artifacts/codeindex.db --read-only --json`; `cdidx search AuthService --db /artifacts/codeindex.db --immutable` | query コマンドは `--read-only`(alias: `--immutable`)を受け付け、既存の CodeIndex database を SQLite の immutable read-only URI mode で開けます。CI artifact、mounted cache、`codeindex.db-wal` / `codeindex.db-shm` sidecar を作成・更新できない sandbox で使います。 |
| 変更系コマンド | `index`、`backfill-fold`、`optimize`、`vacuum` | 書き込み可能な storage を必要とし、read-only database open を拒否します。 |
| 再利用可能な index artifact | `cdidx export codeindex.cdidx.zip`; `cdidx import codeindex.cdidx.zip --db <path>`; `cdidx import codeindex.cdidx.zip --dry-run --json` | CI job では index 後に export して archive を upload します。export は `--overwrite` を明示しない限り既存 destination を拒否し、owner-only temporary file から atomic に publish して POSIX mode `0600` を検証します。export 成功時の JSON は従来 field を維持し、最終 archive の byte 数と SHA-256、完全で immutable な manifest を追加します。利用側は query コマンドの前に import でき、`--dry-run` / `--check` で destination DB を置き換えず archive を検証できます。別 checkout 由来の archive を import 先 project root として扱いたい場合は `--prune-paths` を使います。`.../.cdidx/codeindex.db` を import 先にした場合は sibling の project directory を使い、それ以外の DB path では process current directory に fallback します。archive は `manifest.json` と `codeindex.db` だけを含みます。import は ZIP entry 名を `ZipArchiveSafetyPolicy` で検証し、absolute path、parent-directory segment、backslash、NUL、non-canonical name、duplicate entry、extra entry を extraction 前に拒否します。manifest は row count、readiness bit、writer / indexed-head metadata、schema contract stamp、利用可能な unknown-extension summary などの bounded summary/readiness metadata を持ちます。import は manifest format、manifest `user_version`、`database_sha256`、存在する summary count、embedded SQLite file が CodeIndex database であることを検証してから destination DB を置き換えます。archive の `codeindex.db` entry は compressed / uncompressed metadata と extraction stream の双方で 8 GiB を上限に拒否されます。 |
| maintenance checkpoint | `cdidx db checkpoint <name> [--dry-run]`; `cdidx db checkpoints --list|--delete <name>|--prune --keep <n> [--dry-run]`; `cdidx db restore <name> [--dry-run]`; `cdidx db restore-backups --list|--prune --keep <n> [--dry-run]` | 危険な maintenance の前に `codeindex.db` と既存 WAL/SHM sidecar の filesystem snapshot を作成し、restore で戻します。checkpoint 作成の `--dry-run` は directory を作らず、コピー元 file と byte 数を報告します。checkpoint の delete / prune と restore-backup の prune は明示的な変更 action を必要とし、それぞれの `--dry-run` は filesystem を変更せず、削除予定 path と保持予定 path を正確に報告します。checkpoint prune は新しい有効 snapshot を `<n>` 件保持し、上限 1,000 directory の bounded scan が truncated になった場合は削除をすべて skip します。restore の `--dry-run` は DB を置き換えずに bounded manifest、regular-file payload path、staging に必要な byte 数、destination free space を検証し、実際の restore も先に同じ検証を実行します。checkpoint manifest は database file name だけを記録し、local absolute DB path は記録しません。checkpoint は DB の隣の `<db>.checkpoints/<name>/` に置かれ、restore は pre-restore file を `<db>.restore-backup-<timestamp>/` に保持します。`backfill-fold` は `--no-checkpoint` を渡さない限り、row mutation 前に automatic checkpoint を作ります。automatic checkpoint 名には UTC timestamp と random suffix が含まれ、同一 millisecond の衝突を避けます。 |
| maintenance checkpoint と managed rollback | `cdidx db checkpoint <name> [--dry-run]`; `cdidx db checkpoints --list|--delete <name>|--prune --keep <n> [--dry-run]`; `cdidx db restore <name> [--dry-run] [--no-backup]`; `cdidx db restore-backups --list|--prune --keep <n>|--restore <id> [--dry-run] [--no-backup]` | 危険な maintenance の前に `codeindex.db` と既存 WAL/SHM sidecar の checkpoint を作成できます。import と2種類の restore は、既存 DB を置き換える前に consistent かつ検証済みの managed SQLite rollback snapshot を既定で作成し、`--no-backup` を明示した場合だけ省略します。managed directory は `<db>.restore-backup-<id>/` で、bounded manifest と standalone database payload 1個を含みます。manifest は SHA-256、byte 数、対応する `user_version`、provenance、任意の source identifier を記録しますが、local absolute source path は記録しません。`restore-backups --list` は従来 directory metadata との互換性を維持しつつ ID と provenance を表示し、既存 prune retention もそのまま利用できます。`restore-backups --restore <id>` は directory 境界、manifest、payload hash、schema、staging と rollback を合わせた free space を再検証してから、失敗時の transient rollback を伴う atomic replacement を実行します。`--dry-run` は変更せず、すべての検証と作成予定 backup を報告します。checkpoint の delete / prune と restore-backup の prune は明示的な変更 action を必要とし、checkpoint prune の bounded scan が truncated の場合は削除をすべて skip します。checkpoint は `<db>.checkpoints/<name>/` に置かれ、`backfill-fold` は `--no-checkpoint` がなければ row mutation 前に automatic checkpoint を作ります。 |
| binary compatibility | [COMPATIBILITY.md](COMPATIBILITY.md) | `cdidx` binary の upgrade / downgrade をまたぐ database compatibility を記載します。readiness bit、`codeindex_meta` contract stamp、rebuild requirement を変える場合は、この policy も更新してください。 |
| Fold backfill の preview / recovery | `backfill-fold --dry-run`; MCP `backfill_fold` の `dry_run: true` または `force: true` | dry-run は DB を変更せず FoldReady stamp も書かずに、rewrite 対象の folded-key row をプレビューします。MCP も同じ preview を受け付け、stored version / fingerprint が current に見える場合でも suspicious な fold metadata や row state を復旧するため `force: true` を受け付けます。non-dry-run rewrite は中断後に resume でき、完了済み row update は durable に残り、最終 FoldReady metadata は verification 成功後にだけ stamp されます。MCP response は `progress.rows_done`、`progress.rows_total`、`progress.fraction` を含みます。 |

Expand Down
4 changes: 4 additions & 0 deletions TESTING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -652,6 +652,8 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding
- The `dotnet.yml` SDK setup has one conditional retry for transient SDK download failures. Keep the first attempt marked `continue-on-error` only while the retry is guarded by its failed outcome, so a second failure still fails the job.
- `DbRecoveryTests.cs`
Database corruption recovery and graceful degradation behavior. Filesystem setup failures for `cdidx index` (read-only DB files and unwritable DB parent directories) are covered in `IndexCommandRunnerTests.cs` so they exercise the same CLI JSON/stderr boundary users see.
- `Issue4857ManagedRestoreBackupTests.cs`
End-to-end regression coverage for automatic verified backups before import replacement, managed backup list/restore by ID, mutation-free dry runs, rollback after an injected post-move failure, corrupt payload and insufficient-space rejection, older supported schema stamps, corrupt import archives, and Windows exclusive-lock behavior. The fixture uses the SQLite-pool-sensitive collection because it resets process-wide free-space and replacement hooks in `finally`.
- `JsonOutputSnapshotTests.cs`, `JsonOutputSnapshotHelper.cs`
Golden-file regression fixtures for the CLI `--json` output contracts (issue #1548). Each test runs one command (`status`, `search`, `references`, `impact`, `excerpt`) against a deterministic in-memory fixture, normalizes volatile fields (timestamps, absolute paths, commit SHAs, FTS5 scores, SQLite page counts), and diffs against the matching file under `tests/CodeIndex.Tests/golden/`. Renames, removals, reordered arrays, or new keys fail the snapshot so the contract change is forced to land alongside an intentional golden update. See "JSON `--json` output snapshots" below for the update procedure.
- `QueryCommandRunnerBatchIssue4723Tests.cs`
Expand Down Expand Up @@ -1560,6 +1562,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
`dotnet.yml` と `release.yml` の Windows lane は、temp 固定と Defender 除外 setup をこのスクリプトで共有します。通常の `TMP` / `TEMP` は runner の高速な `RUNNER_TEMP\cdidx-temp` を使います。実行可能な plugin / hook / Git fixture だけは `USERPROFILE\cdidx-trusted-test-temp` を使い、current-user 限定の protected ACL と trusted な祖先 chain で production の executable-boundary contract を満たします。この専用 root は `CDIDX_TEST_TRUSTED_TEMP_ROOT` として helper へ渡します。Windows の実行時間を大きく増やすため、通常の SQLite / filesystem fixture を protected root へ移してはいけません。スクリプトは両 root を含む候補 path を正規化・重複排除し、残した各 path と reason を console および利用可能な場合は job summary へ監査表示してから、生成した string array を1回の `Add-MpPreference` 呼び出しで登録し、最後に Defender preference を読み戻して欠けた path があれば失敗します。この split、batching、audit、verification、または workflow 呼び出し contract を変更するときは `CiWorkflowTests` も更新してください。
- `DbRecoveryTests.cs`
DB破損からの復旧とグレースフル劣化のテスト。`cdidx index` の filesystem setup failure(read-only DB file や書き込み不可の DB 親ディレクトリ)は、ユーザーが見る CLI JSON/stderr 境界を通すため `IndexCommandRunnerTests.cs` で扱います。
- `Issue4857ManagedRestoreBackupTests.cs`
import の置換前に作成する自動検証済み backup、managed backup の list / ID 指定 restore、変更を伴わない dry-run、post-move failure 注入後の rollback、破損 payload と容量不足の拒否、対応する旧 schema stamp、破損 import archive、Windows の exclusive lock を end-to-end で検証します。process-wide の free-space hook と replacement hook を `finally` で戻すため、SQLite-pool-sensitive collection を使用します。
- `JsonOutputSnapshotTests.cs`、`JsonOutputSnapshotHelper.cs`
CLI の `--json` 出力契約に対するゴールデンファイル回帰フィクスチャ (issue #1548)。各テストは `status` / `search` / `references` / `impact` / `excerpt` を決定的なインメモリ fixture に対して実行し、揺らぐフィールド(timestamp、絶対パス、commit SHA、FTS5 score、SQLite page count など)を正規化したうえで `tests/CodeIndex.Tests/golden/` 配下のファイルと差分比較します。フィールドの rename / 削除 / 並び替え / 新規追加が起きると snapshot が失敗するため、契約変更は意図的な golden 更新と同じ PR で揃えざるを得ません。更新手順は下記「JSON `--json` 出力 snapshot」を参照してください。
- `QueryCommandRunnerBatchIssue4723Tests.cs`
Expand Down
Loading
Loading