diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 201d055b9..65388158f 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -49,7 +49,7 @@ Development contracts: |---|---|---| | Read-only database queries | `cdidx status --db /artifacts/codeindex.db --read-only --json`; `cdidx search AuthService --db /artifacts/codeindex.db --immutable` | Query commands accept `--read-only` (alias `--immutable`) to open an existing CodeIndex database through SQLite's immutable read-only URI mode. Use this for CI artifacts, mounted caches, and sandboxes where creating or updating `codeindex.db-wal` / `codeindex.db-shm` sidecars is not allowed. | | Mutating commands | `index`, `backfill-fold`, `optimize`, `vacuum` | These require writable storage and reject read-only database opens. | -| Reusable index artifact | `cdidx export codeindex.cdidx.zip`; `cdidx import codeindex.cdidx.zip --db `; `cdidx import codeindex.cdidx.zip --dry-run --json` | Run export after indexing and upload the archive. Consumers import before query commands, or use `--dry-run` / `--check` to validate the archive without replacing the destination DB. Use `--prune-paths` when the archive comes from another checkout and the restored DB should advertise the import target project root; imports targeting `.../.cdidx/codeindex.db` use the sibling project directory, while other DB paths fall back to the process current directory. The archive contains only `manifest.json` plus `codeindex.db`; import validates ZIP entry names through `ZipArchiveSafetyPolicy` and rejects absolute, parent-directory, backslash, NUL, non-canonical, duplicate, and extra entries before extraction. The manifest carries bounded summary/readiness metadata including row counts, readiness bits, writer/indexed-head metadata, schema contract stamps, and unknown-extension summary when available. Import validates manifest format, manifest `user_version`, `database_sha256`, present summary counts, and the embedded SQLite file as a CodeIndex database before replacing the destination DB. Import rejects archive `codeindex.db` entries whose compressed or uncompressed metadata exceeds 8 GiB, and the extraction stream is also capped at 8 GiB. | +| Reusable index artifact | `cdidx export codeindex.cdidx.zip`; `cdidx import codeindex.cdidx.zip --db `; `cdidx import codeindex.cdidx.zip --dry-run --json` | Run export after indexing and upload the archive. Export refuses an existing destination unless `--overwrite` is explicit, publishes atomically from an owner-only temporary file, and verifies POSIX mode `0600`. Successful export JSON adds final archive byte size and SHA-256 plus the complete immutable manifest while retaining the prior result fields. Consumers import before query commands, or use `--dry-run` / `--check` to validate the archive without replacing the destination DB. Use `--prune-paths` when the archive comes from another checkout and the restored DB should advertise the import target project root; imports targeting `.../.cdidx/codeindex.db` use the sibling project directory, while other DB paths fall back to the process current directory. The archive contains only `manifest.json` plus `codeindex.db`; import validates ZIP entry names through `ZipArchiveSafetyPolicy` and rejects absolute, parent-directory, backslash, NUL, non-canonical, duplicate, and extra entries before extraction. The manifest carries bounded summary/readiness metadata including row counts, readiness bits, writer/indexed-head metadata, schema contract stamps, and unknown-extension summary when available. Import validates manifest format, manifest `user_version`, `database_sha256`, present summary counts, and the embedded SQLite file as a CodeIndex database before replacing the destination DB. Import rejects archive `codeindex.db` entries whose compressed or uncompressed metadata exceeds 8 GiB, and the extraction stream is also capped at 8 GiB. | | Maintenance checkpoint | `cdidx db checkpoint [--dry-run]`; `cdidx db checkpoints --list|--delete |--prune --keep [--dry-run]`; `cdidx db restore [--dry-run]`; `cdidx db restore-backups --list|--prune --keep [--dry-run]` | Checkpoint snapshots `codeindex.db` plus existing WAL/SHM sidecars before risky maintenance. Checkpoint creation `--dry-run` reports the source files and byte count without creating a directory. Checkpoint delete/prune and restore-backup prune require an explicit mutation action; their `--dry-run` forms make no filesystem changes and report the exact paths that would be deleted and retained. Checkpoint prune retains the newest `` valid snapshots, and skips all deletion if its bounded 1,000-directory scan is truncated. Restore `--dry-run` validates the bounded manifest, regular-file payload paths, required staging bytes, and destination free space without replacing the DB; a real restore runs the same validation first. Checkpoint manifests record only the database file name, not the local absolute DB path. Restore rolls back and keeps pre-restore files under `.restore-backup-/`. Checkpoints live next to the DB under `.checkpoints//`. `backfill-fold` creates an automatic checkpoint before it mutates rows unless `--no-checkpoint` is passed; automatic checkpoint names include a UTC timestamp plus random suffix to avoid same-millisecond collisions. | | Binary compatibility | [COMPATIBILITY.md](COMPATIBILITY.md) | Database compatibility across `cdidx` binary upgrades and downgrades is documented there. Keep that policy updated whenever readiness bits, `codeindex_meta` contract stamps, or rebuild requirements change. | | Fold backfill preview and recovery | `backfill-fold --dry-run`; MCP `backfill_fold` with `dry_run: true` or `force: true` | Dry-run previews folded-key rows without mutating the DB or stamping FoldReady. MCP accepts the same preview and can force rewriting all folded keys when an operator needs to recover from suspicious fold metadata or row state even though the stored version/fingerprint appears current. Non-dry-run row rewrites are resumable after interruption: completed row updates remain durable, and final FoldReady metadata is stamped only after verification succeeds. MCP responses include `progress.rows_done`, `progress.rows_total`, and `progress.fraction` so clients can report and retry long backfills. | @@ -61,8 +61,9 @@ Development contracts: | `.cdidx/` | Created with mode `0700`. | | `codeindex.db` plus WAL/SHM sidecars | Mode `0600` is applied when the files exist. Enforcement defaults to best-effort and can be made strict with `CDIDX_DB_PERMISSION_POLICY=strict`. | | `suggestions-*.json` suggestion stores | Written atomically with owner-only mode `0600` on POSIX. | +| Portable export archives | Written atomically through the `Sensitive` profile and verified as owner-only mode `0600` on POSIX because they contain indexed source text. Existing destinations require explicit `--overwrite`. | | Indexed workspace source reads | Source-file content and checksum reads use `FileShare.ReadWrite | FileShare.Delete`, long-path normalization, the configured max-file byte cap, and modified-time retry checks so indexing can inspect files that build tools keep open without allowing unbounded growth. | -| Atomic file writes | `AtomicFileWriter` writes to a sibling temp file, applies the requested POSIX mode before replacement, flushes file contents, renames over the target, and fsyncs the parent directory on Unix. Callers must use the `Sensitive` write profile for local state, caches, suggestions, checkpoints, and other private payloads; user-requested exports and reports use the default `Public` profile unless their content is explicitly private. If the parent directory flush fails after replacement, the command fails explicitly so callers know the file was replaced but directory durability was not confirmed. Windows skips directory fsync because the helper only promises it on supported Unix platforms. | +| Atomic file writes | `AtomicFileWriter` writes to a sibling temp file, applies the requested POSIX mode before replacement, flushes file contents, renames over the target, and fsyncs the parent directory on Unix. Callers must use the `Sensitive` write profile for local state, caches, suggestions, checkpoints, portable archives, and other private payloads; user-requested reports use the default `Public` profile unless their content is explicitly private. If the parent directory flush fails after replacement, the command fails explicitly so callers know the file was replaced but directory durability was not confirmed. Windows skips directory fsync because the helper only promises it on supported Unix platforms. | | Index locks, watch sub-run spools, staged hook scripts, lock metadata sidecars, and active workspace `active.json` | Created or written as owner-only files (`0600`) before contents are exposed, and read through small bounded buffers where applicable so stale or corrupted diagnostics cannot expose local paths more broadly or force unbounded allocation. | | Checkpoint roots, snapshot directories, manifest files, copied DB/WAL/SHM snapshots, and restore staging/backup directories | Forced owner-only on POSIX. | | `status --json` | Reports `data_dir_mode`, `db_file_mode`, the effective `database_permission_policy`, and support-safe `database_permission_diagnostics` when Unix mode operations fail. | @@ -3119,7 +3120,7 @@ net9 CI lane に合わせる場合は `FRAMEWORK=net9.0 make test` を使いま |---|---|---| | read-only database query | `cdidx status --db /artifacts/codeindex.db --read-only --json`; `cdidx search AuthService --db /artifacts/codeindex.db --immutable` | query コマンドは `--read-only`（alias: `--immutable`）を受け付け、既存の CodeIndex database を SQLite の immutable read-only URI mode で開けます。CI artifact、mounted cache、`codeindex.db-wal` / `codeindex.db-shm` sidecar を作成・更新できない sandbox で使います。 | | 変更系コマンド | `index`、`backfill-fold`、`optimize`、`vacuum` | 書き込み可能な storage を必要とし、read-only database open を拒否します。 | -| 再利用可能な index artifact | `cdidx export codeindex.cdidx.zip`; `cdidx import codeindex.cdidx.zip --db `; `cdidx import codeindex.cdidx.zip --dry-run --json` | CI job では index 後に export して archive を upload します。利用側は query コマンドの前に import でき、`--dry-run` / `--check` で destination DB を置き換えず archive を検証できます。別 checkout 由来の archive を import 先 project root として扱いたい場合は `--prune-paths` を使います。`.../.cdidx/codeindex.db` を import 先にした場合は sibling の project directory を使い、それ以外の DB path では process current directory に fallback します。archive は `manifest.json` と `codeindex.db` だけを含みます。import は ZIP entry 名を `ZipArchiveSafetyPolicy` で検証し、absolute path、parent-directory segment、backslash、NUL、non-canonical name、duplicate entry、extra entry を extraction 前に拒否します。manifest は row count、readiness bit、writer / indexed-head metadata、schema contract stamp、利用可能な unknown-extension summary などの bounded summary/readiness metadata を持ちます。import は manifest format、manifest `user_version`、`database_sha256`、存在する summary count、embedded SQLite file が CodeIndex database であることを検証してから destination DB を置き換えます。archive の `codeindex.db` entry は compressed / uncompressed metadata と extraction stream の双方で 8 GiB を上限に拒否されます。 | +| 再利用可能な index artifact | `cdidx export codeindex.cdidx.zip`; `cdidx import codeindex.cdidx.zip --db `; `cdidx import codeindex.cdidx.zip --dry-run --json` | CI job では index 後に export して archive を upload します。export は `--overwrite` を明示しない限り既存 destination を拒否し、owner-only temporary file から atomic に publish して POSIX mode `0600` を検証します。export 成功時の JSON は従来 field を維持し、最終 archive の byte 数と SHA-256、完全で immutable な manifest を追加します。利用側は query コマンドの前に import でき、`--dry-run` / `--check` で destination DB を置き換えず archive を検証できます。別 checkout 由来の archive を import 先 project root として扱いたい場合は `--prune-paths` を使います。`.../.cdidx/codeindex.db` を import 先にした場合は sibling の project directory を使い、それ以外の DB path では process current directory に fallback します。archive は `manifest.json` と `codeindex.db` だけを含みます。import は ZIP entry 名を `ZipArchiveSafetyPolicy` で検証し、absolute path、parent-directory segment、backslash、NUL、non-canonical name、duplicate entry、extra entry を extraction 前に拒否します。manifest は row count、readiness bit、writer / indexed-head metadata、schema contract stamp、利用可能な unknown-extension summary などの bounded summary/readiness metadata を持ちます。import は manifest format、manifest `user_version`、`database_sha256`、存在する summary count、embedded SQLite file が CodeIndex database であることを検証してから destination DB を置き換えます。archive の `codeindex.db` entry は compressed / uncompressed metadata と extraction stream の双方で 8 GiB を上限に拒否されます。 | | maintenance checkpoint | `cdidx db checkpoint [--dry-run]`; `cdidx db checkpoints --list|--delete |--prune --keep [--dry-run]`; `cdidx db restore [--dry-run]`; `cdidx db restore-backups --list|--prune --keep [--dry-run]` | 危険な maintenance の前に `codeindex.db` と既存 WAL/SHM sidecar の filesystem snapshot を作成し、restore で戻します。checkpoint 作成の `--dry-run` は directory を作らず、コピー元 file と byte 数を報告します。checkpoint の delete / prune と restore-backup の prune は明示的な変更 action を必要とし、それぞれの `--dry-run` は filesystem を変更せず、削除予定 path と保持予定 path を正確に報告します。checkpoint prune は新しい有効 snapshot を `` 件保持し、上限 1,000 directory の bounded scan が truncated になった場合は削除をすべて skip します。restore の `--dry-run` は DB を置き換えずに bounded manifest、regular-file payload path、staging に必要な byte 数、destination free space を検証し、実際の restore も先に同じ検証を実行します。checkpoint manifest は database file name だけを記録し、local absolute DB path は記録しません。checkpoint は DB の隣の `.checkpoints//` に置かれ、restore は pre-restore file を `.restore-backup-/` に保持します。`backfill-fold` は `--no-checkpoint` を渡さない限り、row mutation 前に automatic checkpoint を作ります。automatic checkpoint 名には UTC timestamp と random suffix が含まれ、同一 millisecond の衝突を避けます。 | | binary compatibility | [COMPATIBILITY.md](COMPATIBILITY.md) | `cdidx` binary の upgrade / downgrade をまたぐ database compatibility を記載します。readiness bit、`codeindex_meta` contract stamp、rebuild requirement を変える場合は、この policy も更新してください。 | | Fold backfill の preview / recovery | `backfill-fold --dry-run`; MCP `backfill_fold` の `dry_run: true` または `force: true` | dry-run は DB を変更せず FoldReady stamp も書かずに、rewrite 対象の folded-key row をプレビューします。MCP も同じ preview を受け付け、stored version / fingerprint が current に見える場合でも suspicious な fold metadata や row state を復旧するため `force: true` を受け付けます。non-dry-run rewrite は中断後に resume でき、完了済み row update は durable に残り、最終 FoldReady metadata は verification 成功後にだけ stamp されます。MCP response は `progress.rows_done`、`progress.rows_total`、`progress.fraction` を含みます。 | @@ -3131,8 +3132,9 @@ net9 CI lane に合わせる場合は `FRAMEWORK=net9.0 make test` を使いま | `.cdidx/` | mode `0700` で作成。 | | `codeindex.db` と WAL/SHM sidecar | ファイルが存在する場合は mode `0600` を適用。既定は best-effort で、`CDIDX_DB_PERMISSION_POLICY=strict` により strict enforcement にできます。 | | `suggestions-*.json` suggestion store | POSIX では owner-only の mode `0600` で atomic write します。 | +| portable export archive | indexed source text を含むため `Sensitive` profile で atomic write し、POSIX では owner-only の mode `0600` であることを検証します。既存 destination の置換には明示的な `--overwrite` が必要です。 | | インデックス対象ワークスペースソースの読み取り | ソースファイル本文とチェックサムの読み取りは `FileShare.ReadWrite | FileShare.Delete`、長いパスの正規化、設定された最大ファイルバイト数、更新時刻の再確認を使い、ビルドツールが開いたままのファイルも無制限な肥大化を許さず検査できるようにします。 | -| atomic file write | `AtomicFileWriter` は sibling temp file に書き込み、要求された POSIX mode を置換前に適用し、file content を flush してから target へ rename し、Unix では parent directory を fsync します。local state、cache、suggestion、checkpoint など private payload には `Sensitive` write profile を使い、user-requested export や report は内容が明示的に private でない限り既定の `Public` profile を使います。置換後に parent directory flush が失敗した場合、file は置換済みだが directory durability を確認できていないことが caller に分かるよう command は明示的に失敗します。Windows では、この helper の directory fsync 保証は supported Unix platform に限定されるため skip します。 | +| atomic file write | `AtomicFileWriter` は sibling temp file に書き込み、要求された POSIX mode を置換前に適用し、file content を flush してから target へ rename し、Unix では parent directory を fsync します。local state、cache、suggestion、checkpoint、portable archive など private payload には `Sensitive` write profile を使い、user-requested report は内容が明示的に private でない限り既定の `Public` profile を使います。置換後に parent directory flush が失敗した場合、file は置換済みだが directory durability を確認できていないことが caller に分かるよう command は明示的に失敗します。Windows では、この helper の directory fsync 保証は supported Unix platform に限定されるため skip します。 | | index lock、watch sub-run spool、staged hook script、lock metadata sidecar、active workspace の `active.json` | 内容が露出する前に owner-only file (`0600`) として作成または書き込み、該当するものは stale / corrupt diagnostic が local path を広く漏らしたり unbounded allocation を強制したりしないよう小さな bounded buffer で読みます。 | | database checkpoint root、snapshot directory、manifest file、copy された DB/WAL/SHM snapshot、restore staging/backup directory | POSIX では owner-only に固定。 | | `status --json` | `data_dir_mode`、`db_file_mode`、有効な `database_permission_policy`、Unix mode 操作失敗時の support-safe な `database_permission_diagnostics` を報告。 | diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 89cd9e80f..78d94d7f5 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -334,6 +334,8 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding Changelog limit tests resolve checked-in files through `RepositoryTestPaths` instead of performing another root walk. Whole-workflow policy audits should use `RepositoryTestPaths.ReadNormalizedWorkflows()` so enumeration order, extension filtering, file naming, and normalization are shared instead of being rebuilt inside individual test classes. When one policy test checks several step-level rules, parse workflow step blocks once and filter the retained blocks for each rule instead of rerunning the multiline step regex for every assertion family. +- `ExportImportCommandRunnerTests` issue-4827 portable-export coverage + keeps the overwrite boundary and artifact attestation deterministic. Preserve default refusal of regular and dangling-symlink destinations, explicit replacement with verifiable byte-size/SHA-256/manifest metadata and POSIX owner-only mode, failed-write temp cleanup, and a publish-boundary injection that creates a concurrent winner without using sleeps. - `IndexCommandRunnerTests.Run_CancelDuringFreshIndex_ReturnsInterruptedJson`, `Run_CancelDuringDryRunScan_ReturnsInterruptedJson`, and `Run_CancelBeforeFreshScan_ReturnsInterruptedJson` exercise the same in-process cancellation paths used after Ctrl-C/SIGINT wiring, including scan-time cancellation, so interrupted index runs keep returning the canonical JSON error contract. - `IndexCommandRunnerTests.RunOptimizeFts_DryRunPreviewsWithoutWritingThenOptimizeMutates_Issue4577`, `RunOptimizeFts_LockHeld_ReportsDbLocked`, and `RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable` @@ -1220,6 +1222,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" changelog limit test も別の root walk を行わず、`RepositoryTestPaths` 経由で checked-in file を解決します。 workflow 全体の policy audit には `RepositoryTestPaths.ReadNormalizedWorkflows()` を使い、列挙順、extension filter、file name、normalization を個別 test class 内で再構築せず共有します。 1つの policy test が複数の step-level rule を検証する場合は、workflow step block を一度だけ解析して保持し、assertion family ごとに multiline step regex を再実行せず保持済み block を絞り込みます。 +- `ExportImportCommandRunnerTests` の issue-4827 portable-export coverage + overwrite 境界と artifact attestation を deterministic に固定します。通常 file と dangling symlink destination の既定拒否、明示置換で検証可能な byte size / SHA-256 / manifest metadata と POSIX owner-only mode、書き込み失敗時の temp cleanup、sleep を使わず publish 境界へ concurrent winner を生成する injection を維持してください。 - `IndexCommandRunnerTests.Run_CancelDuringFreshIndex_ReturnsInterruptedJson`、`Run_CancelDuringDryRunScan_ReturnsInterruptedJson`、`Run_CancelBeforeFreshScan_ReturnsInterruptedJson` Ctrl-C/SIGINT 配線後に使われる in-process cancellation 経路を、scan 中のキャンセルも含めて検証し、interrupted index run が標準の JSON error contract を返し続けることを固定する。 - `IndexCommandRunnerTests.RunOptimizeFts_DryRunPreviewsWithoutWritingThenOptimizeMutates_Issue4577`、`RunOptimizeFts_LockHeld_ReportsDbLocked`、`RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable` diff --git a/USER_GUIDE.md b/USER_GUIDE.md index f3ffc711e..a11536d6b 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -450,6 +450,7 @@ job: ```bash cdidx export codeindex.cdidx.zip +cdidx export codeindex.cdidx.zip --overwrite --json cdidx export app.cdidx.zip --project App --lang csharp --exclude-tests cdidx export shared.cdidx.zip --path 'src/shared/*' --exclude-path 'src/shared/generated/*' cdidx import codeindex.cdidx.zip @@ -465,6 +466,14 @@ snapshot contains only the retained files and their dependent chunks, symbols, references, and diagnostics, and is vacuumed before packaging. JSON output and `manifest.json` include the requested scope, resolved project paths, and source and exported file counts. An export without scope flags remains a full archive. +Portable export refuses an existing destination by default; pass `--overwrite` +only when replacing it is intentional. The archive is built in an owner-only +sibling temporary file and atomically published, and POSIX archives are verified +as mode `0600`. Successful JSON keeps the existing fields and additionally +returns `archive_size_bytes`, the final archive's `archive_sha256`, and the +complete immutable `manifest` object. That manifest carries the database hash, +row counts, schema contract stamps, readiness state, unknown-extension summary, +and export scope needed to evaluate the artifact before import. The archive path is intended for trusted CodeIndex databases. Import validates that the embedded SQLite file is a CodeIndex DB before replacing the destination @@ -3548,6 +3557,7 @@ legacy database も query でき、generated-code policy は `unavailable` と ```bash cdidx export codeindex.cdidx.zip +cdidx export codeindex.cdidx.zip --overwrite --json cdidx export app.cdidx.zip --project App --lang csharp --exclude-tests cdidx export shared.cdidx.zip --path 'src/shared/*' --exclude-path 'src/shared/generated/*' cdidx import codeindex.cdidx.zip @@ -3562,6 +3572,13 @@ archive export では `--lang`、繰り返し指定できる `--path` / `--exclu chunk、symbol、reference、diagnostic だけを保持し、packaging 前に vacuum します。 JSON output と `manifest.json` には指定 scope、解決済み project path、元と出力後の file count が含まれます。scope flag を指定しなければ従来どおり full archive です。 +portable export は既存 destination を既定で拒否します。意図して置き換える場合だけ +`--overwrite` を指定してください。archive は owner-only の sibling temporary file に +構築して atomic に publish し、POSIX では mode `0600` であることも検証します。 +成功時の JSON は既存 field を維持したまま、`archive_size_bytes`、最終 archive の +`archive_sha256`、完全で immutable な `manifest` object を追加で返します。この manifest +には import 前に artifact を評価するための database hash、row count、schema contract +stamp、readiness state、unknown-extension summary、export scope が含まれます。 archive は信頼できる CodeIndex database の共有向けです。Import は埋め込まれた SQLite file が CodeIndex DB であることを検証してから destination database を置き換えます。 diff --git a/changelog.d/unreleased/4827.security.md b/changelog.d/unreleased/4827.security.md new file mode 100644 index 000000000..7ef599113 --- /dev/null +++ b/changelog.d/unreleased/4827.security.md @@ -0,0 +1,27 @@ +--- +category: security +issues: + - 4827 +affected: + - src/CodeIndex/Cli/AtomicFileWriter.cs + - src/CodeIndex/Cli/ExportImportCommandRunner.ExportArchive.cs + - src/CodeIndex/Cli/ExportImportCommandRunner.Manifest.cs + - src/CodeIndex/Cli/ExportImportCommandRunner.Contracts.cs + - src/CodeIndex/Cli/ExportImportCommandRunner.cs + - src/CodeIndex/Cli/CliFlagSchema.cs + - src/CodeIndex/Cli/ConsoleUi.Help.cs + - src/CodeIndex/Cli/ConsoleUi.cs + - tests/CodeIndex.Tests/ExportImportCommandRunnerTests.cs + - tests/CodeIndex.Tests/ConsoleUiTests.cs + - USER_GUIDE.md + - DEVELOPER_GUIDE.md + - TESTING_GUIDE.md +--- + +## English + +- **Portable archives now publish privately and require explicit replacement (#4827)** — `cdidx export` refuses an existing archive unless `--overwrite` is passed, atomically publishes through an owner-only temporary file, verifies POSIX mode `0600`, and returns final byte-size, SHA-256, and complete manifest metadata in successful JSON output. + +## 日本語 + +- **portable archive を非公開で publish し、置換を明示必須にしました (#4827)** — `cdidx export` は `--overwrite` を指定しない限り既存 archive を拒否し、owner-only temporary file を介して atomic に publish して POSIX mode `0600` を検証します。成功時の JSON には最終 byte size、SHA-256、完全な manifest metadata も返します。 diff --git a/src/CodeIndex/Cli/AtomicFileWriter.cs b/src/CodeIndex/Cli/AtomicFileWriter.cs index aecf50dc2..1f034bcb0 100644 --- a/src/CodeIndex/Cli/AtomicFileWriter.cs +++ b/src/CodeIndex/Cli/AtomicFileWriter.cs @@ -9,6 +9,9 @@ internal static class AtomicFileWriter { internal const int MaxTempFileNameChars = 120; private const int MaxTempStemChars = 48; + private const int AtCurrentWorkingDirectory = -100; + private const uint LinuxRenameNoReplace = 1; + private const uint MacRenameExclusiveFlag = 0x00000004; internal static Action? FlushParentDirectoryForTesting { get; set; } public enum WriteProfile @@ -87,7 +90,15 @@ public static void Write(string path, Action writeContents, WriteProfile => WriteCore(path, writeContents, ResolveProfileModeCallback(profile), profile, overwrite: true); public static void Write(string path, Action writeContents, WriteProfile profile, bool overwrite) - => WriteCore(path, writeContents, ResolveProfileModeCallback(profile), profile, overwrite); + => WriteCore(path, writeContents, ResolveProfileModeCallback(profile), profile, overwrite, validateBeforePublish: null); + + internal static void WriteWithPrePublishValidation( + string path, + Action writeContents, + WriteProfile profile, + bool overwrite, + Action validateBeforePublish) + => WriteCore(path, writeContents, ResolveProfileModeCallback(profile), profile, overwrite, validateBeforePublish); internal static void WritePreservingExistingOnFailure( string path, @@ -178,13 +189,13 @@ private static void WriteCore( Action writeContents, Action? applyFileMode, WriteProfile profile, - bool overwrite) + bool overwrite, + Action? validateBeforePublish = null) { ArgumentNullException.ThrowIfNull(writeContents); var tempPath = BuildTempPath(path); var ioTempPath = LongPath.EnsureWindowsPrefix(tempPath); - var moved = false; try { @@ -195,29 +206,38 @@ private static void WriteCore( stream.Flush(flushToDisk: true); } + validateBeforePublish?.Invoke(tempPath); if (overwrite) { MoveReplacing(tempPath, path); } else { + var deleteUnixTempAfterPublish = false; try { - MoveFileCore(tempPath, path, overwrite: false, applyDestinationMode: null); + if (OperatingSystem.IsWindows()) + { + MoveFileCore(tempPath, path, overwrite: false, applyDestinationMode: null); + } + else + { + deleteUnixTempAfterPublish = PublishUnixNoReplace(ioTempPath, path); + } } - catch (IOException ex) when (File.Exists(LongPath.EnsureWindowsPrefix(path))) + catch (IOException ex) when (PathEntryExists(path)) { throw new DestinationAlreadyExistsException(path, ex); } - moved = true; + + if (deleteUnixTempAfterPublish) + File.Delete(ioTempPath); FlushParentDirectoryAfterCreate(path); } - moved = true; } catch { - if (!moved) - TryDeleteFile(ioTempPath); + TryDeleteFile(ioTempPath); throw; } } @@ -230,9 +250,80 @@ private static FileStream CreateTempFile(string path, WriteProfile profile) return DataDirectorySecurity.OpenPrivateFileStream(path, FileMode.CreateNew, FileAccess.Write, FileShare.None); } + private static bool PublishUnixNoReplace(string existingPath, string destinationPath) + { + try + { + // Prefer the platform no-clobber rename so publication can work on + // mounts that implement rename exclusion but do not support hard links. + int result; + if (OperatingSystem.IsLinux()) + { + result = UnixRenameAt2( + AtCurrentWorkingDirectory, + existingPath, + AtCurrentWorkingDirectory, + destinationPath, + LinuxRenameNoReplace); + } + else if (OperatingSystem.IsMacOS()) + { + result = MacRenameWithFlags(existingPath, destinationPath, MacRenameExclusiveFlag); + } + else + { + result = -1; + } + + if (result == 0) + return false; + } + catch (EntryPointNotFoundException) + { + // Older libc implementations may not expose the no-clobber rename + // entry point. The hard-link fallback below retains atomic refusal. + } + + CreateUnixHardLink(existingPath, destinationPath); + return true; + } + + private static void CreateUnixHardLink(string existingPath, string linkPath) + { + if (UnixLink(existingPath, linkPath) == 0) + return; + + var errno = Marshal.GetLastPInvokeError(); + throw new IOException( + $"Could not publish the atomic file without replacing the destination (errno {errno})."); + } + private static Action? ResolveProfileModeCallback(WriteProfile profile) => profile == WriteProfile.Sensitive ? DataDirectorySecurity.ApplyPrivateFileMode : null; + internal static bool PathEntryExists(string path) + { + var ioPath = LongPath.EnsureWindowsPrefix(path); + if (File.Exists(ioPath) || Directory.Exists(ioPath)) + return true; + + try + { + if (!string.IsNullOrEmpty(new FileInfo(ioPath).LinkTarget)) + return true; + + return !string.IsNullOrEmpty(new DirectoryInfo(ioPath).LinkTarget); + } + catch (FileNotFoundException) + { + return false; + } + catch (DirectoryNotFoundException) + { + return false; + } + } + internal static void MoveReplacing(string sourcePath, string destinationPath) { MoveFileCore(sourcePath, destinationPath, overwrite: true, applyDestinationMode: null); @@ -419,6 +510,20 @@ private static void ReportCleanupFailure(Action? failureSink, Excepti [DllImport("libc", EntryPoint = "open", SetLastError = true)] private static extern int UnixOpen(string path, int flags); + [DllImport("libc", EntryPoint = "link", SetLastError = true)] + private static extern int UnixLink(string existingPath, string linkPath); + + [DllImport("libc", EntryPoint = "renameat2", SetLastError = true)] + private static extern int UnixRenameAt2( + int oldDirectoryFileDescriptor, + string oldPath, + int newDirectoryFileDescriptor, + string newPath, + uint flags); + + [DllImport("libc", EntryPoint = "renamex_np", SetLastError = true)] + private static extern int MacRenameWithFlags(string oldPath, string newPath, uint flags); + [DllImport("libc", EntryPoint = "fsync", SetLastError = true)] private static extern int UnixFsync(int fd); diff --git a/src/CodeIndex/Cli/CliFlagSchema.cs b/src/CodeIndex/Cli/CliFlagSchema.cs index 238b7d0f2..2be1d2d2f 100644 --- a/src/CodeIndex/Cli/CliFlagSchema.cs +++ b/src/CodeIndex/Cli/CliFlagSchema.cs @@ -340,7 +340,7 @@ private static IReadOnlyList BuildAll() new() { Name = "--description", ValuePlaceholder = "", Description = "Suggestions add: local suggestion description", PrimaryCommands = Set("suggestions") }, new() { Name = "--title", ValuePlaceholder = "", Description = "Suggestions add: optional issue-draft title source", PrimaryCommands = Set("suggestions") }, new() { Name = "--evidence-path", ValuePlaceholder = "<path>", Description = "Suggestions add: repository-relative evidence path; repeat for multiple paths", PrimaryCommands = Set("suggestions") }, - new() { Name = "--overwrite", Description = "Atomically replace an existing report bundle or suggestions export", PrimaryCommands = Set("report", "suggestions") }, + new() { Name = "--overwrite", Description = "Portable archive, report bundle, or suggestions export: atomically replace an existing output file", PrimaryCommands = Set("export", "report", "suggestions") }, new() { Name = "--body", Description = "Include definition body snippets in JSON-capable result rows", PrimaryCommands = Set(BodyCommands) }, new() { Name = "--body-start", ValuePlaceholder = "<line>", Description = "Inspect: start definition body slice at this 1-based source line", PrimaryCommands = Set(InspectFieldCommands) }, new() { Name = "--body-lines", ValuePlaceholder = "<n>", Description = "Inspect: return at most this many definition body lines", PrimaryCommands = Set(InspectFieldCommands) }, diff --git a/src/CodeIndex/Cli/ConsoleUi.Help.cs b/src/CodeIndex/Cli/ConsoleUi.Help.cs index 076d1a70b..506ee277a 100644 --- a/src/CodeIndex/Cli/ConsoleUi.Help.cs +++ b/src/CodeIndex/Cli/ConsoleUi.Help.cs @@ -297,6 +297,7 @@ private static void PrintExamples() Console.WriteLine(" cdidx index ./myproject --watch Run an initial scan, then keep the index live as files change (Ctrl+C to stop)"); Console.WriteLine(" cdidx export ctags --output tags Export editor tags for Vim, Emacs, and Sublime"); Console.WriteLine(" cdidx export codeindex.cdidx.zip Export a portable CodeIndex archive"); + Console.WriteLine(" cdidx export codeindex.cdidx.zip --overwrite Explicitly replace an existing portable archive"); Console.WriteLine(" cdidx import codeindex.cdidx.zip Import a portable CodeIndex archive"); Console.WriteLine(" cdidx import codeindex.cdidx.zip --dry-run Validate an archive without replacing the DB"); Console.WriteLine(" cdidx search \"authenticate\" Full-text search"); diff --git a/src/CodeIndex/Cli/ConsoleUi.cs b/src/CodeIndex/Cli/ConsoleUi.cs index fdd740293..ac5513365 100644 --- a/src/CodeIndex/Cli/ConsoleUi.cs +++ b/src/CodeIndex/Cli/ConsoleUi.cs @@ -141,7 +141,7 @@ private static readonly (string Command, string Usage)[] CommandUsageLines = ("suggestions-update", "cdidx suggestions update <id> [--description <text>] [--context <text>] [--title <text>] [--evidence-path <path>] [--category <value>] [--language <lang>] [--agent <name>] [--db <path>] [--json]"), ("suggestions-update", "cdidx suggestions update <id> --status <draft|open_in_upstream|resolved_in_upstream|wont_fix|duplicate|superseded> [--actor <name>] [--reason <text>] [--db <path>] [--json]"), ("suggestions-delete", "cdidx suggestions delete <id> [--db <path>] [--json]"), - ("export", "cdidx export <archive> [--db <path>] [--json] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--project <name|path>] [--solution <path>] [--exclude-tests]"), + ("export", "cdidx export <archive> [--db <path>] [--json] [--overwrite] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--project <name|path>] [--solution <path>] [--exclude-tests]"), ("export", "cdidx export ctags [--output <path>] [--db <path>] [--json] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--exclude-tests] [--include-generated]"), ("export-ctags", "cdidx export ctags [--output <path>] [--db <path>] [--json] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--exclude-tests] [--include-generated]"), ("import", "cdidx import <archive> [--db <path>] [--prune-paths] [--dry-run|--check] [--limit <n<=10000>] [--offset <n>] [--json]"), diff --git a/src/CodeIndex/Cli/ExportImportCommandRunner.Contracts.cs b/src/CodeIndex/Cli/ExportImportCommandRunner.Contracts.cs index 1a4c722cc..ae5e29749 100644 --- a/src/CodeIndex/Cli/ExportImportCommandRunner.Contracts.cs +++ b/src/CodeIndex/Cli/ExportImportCommandRunner.Contracts.cs @@ -224,6 +224,9 @@ internal sealed record ExportArchiveResult( [property: JsonPropertyName("api_version")] string ApiVersion, [property: JsonPropertyName("archive_path")] string ArchivePath, [property: JsonPropertyName("db_path")] string DbPath, + [property: JsonPropertyName("archive_size_bytes")] long ArchiveSizeBytes, + [property: JsonPropertyName("archive_sha256")] string ArchiveSha256, + [property: JsonPropertyName("manifest")] ExportManifest Manifest, [property: JsonPropertyName("scope")] ArchiveExportScopeResult Scope); private sealed record ArchiveExportOptions( string? Lang, diff --git a/src/CodeIndex/Cli/ExportImportCommandRunner.ExportArchive.cs b/src/CodeIndex/Cli/ExportImportCommandRunner.ExportArchive.cs index 2ead0acd0..f580c9887 100644 --- a/src/CodeIndex/Cli/ExportImportCommandRunner.ExportArchive.cs +++ b/src/CodeIndex/Cli/ExportImportCommandRunner.ExportArchive.cs @@ -24,6 +24,7 @@ private static int RunExportArchive(string[] args, JsonSerializerOptions jsonOpt var excludePathPatterns = new List<string>(); var projects = new List<string>(); var excludeTests = false; + var overwrite = false; var wantsJson = Array.Exists(args, arg => arg == "--json"); for (var i = 0; i < args.Length; i++) @@ -39,6 +40,11 @@ private static int RunExportArchive(string[] args, JsonSerializerOptions jsonOpt excludeTests = true; continue; } + if (arg == "--overwrite") + { + overwrite = true; + continue; + } if (TryReadValueOption(args, ref i, "--db", arg, out var dbValue, out var dbError)) { @@ -120,6 +126,17 @@ private static int RunExportArchive(string[] args, JsonSerializerOptions jsonOpt { return WriteExportError(wantsJson, jsonOptions, PhaseParseArgs, "export_archive_overlaps_database", "export archive path must not be the source database or a SQLite sidecar.", "choose a separate archive path, for example `codeindex.cdidx.zip`.", ArchiveExportUsage); } + if (!overwrite && AtomicFileWriter.PathEntryExists(fullOutputPath)) + { + return WriteExportError( + wantsJson, + jsonOptions, + PhaseWriteArchive, + "export_archive_exists", + "export archive destination already exists.", + "pass `--overwrite` to atomically replace the existing destination.", + ArchiveExportUsage); + } var scopeOptions = new ArchiveExportOptions( lang, @@ -171,16 +188,30 @@ private static int RunExportArchive(string[] args, JsonSerializerOptions jsonOpt phase = PhaseSha256; manifest = manifest with { DatabaseSha256 = ComputeSha256(snapshotPath, cancellationToken) }; phase = PhaseWriteArchive; - WriteExportArchiveFile(fullOutputPath, snapshotPath, manifest, jsonOptions, cancellationToken); + var artifact = WriteExportArchiveFile( + fullOutputPath, + snapshotPath, + manifest, + jsonOptions, + cancellationToken, + overwrite, + includeArtifactMetadata: wantsJson); if (wantsJson) + { + var attestation = artifact + ?? throw new InvalidDataException("export archive artifact metadata was not created"); Console.WriteLine(JsonSerializer.Serialize( new ExportArchiveResult( "1", fullOutputPath, fullSourceDbPath, + attestation.SizeBytes, + attestation.Sha256, + manifest, manifest.Scope ?? throw new InvalidDataException("export scope metadata was not created")), jsonOptions)); + } else Console.WriteLine($"Exported CodeIndex archive to {fullOutputPath}"); return CommandExitCodes.Success; @@ -197,6 +228,17 @@ private static int RunExportArchive(string[] args, JsonSerializerOptions jsonOpt ArchiveExportUsage, CommandExitCodes.CancelledBySignal); } + catch (AtomicFileWriter.DestinationAlreadyExistsException) + { + return WriteExportError( + wantsJson, + jsonOptions, + PhaseWriteArchive, + "export_archive_exists", + "export archive destination was created before the archive could be published.", + "pass `--overwrite` to atomically replace the destination, or choose another path.", + ArchiveExportUsage); + } catch (Exception ex) { return WriteExportError(wantsJson, jsonOptions, phase, "export_failed", $"export failed ({CommandErrorWriter.FormatSanitizedException(ex)}).", "check the database, scope, project, and output archive paths.", ArchiveExportUsage); diff --git a/src/CodeIndex/Cli/ExportImportCommandRunner.Manifest.cs b/src/CodeIndex/Cli/ExportImportCommandRunner.Manifest.cs index c52d8fcf9..a964d2549 100644 --- a/src/CodeIndex/Cli/ExportImportCommandRunner.Manifest.cs +++ b/src/CodeIndex/Cli/ExportImportCommandRunner.Manifest.cs @@ -65,22 +65,44 @@ private static void AddTextEntry(ZipArchive archive, string name, string content writer.Write(content); } - internal static void WriteExportArchiveFile(string outputPath, string snapshotPath, ExportManifest manifest, JsonSerializerOptions jsonOptions, CancellationToken cancellationToken) + internal static (long SizeBytes, string Sha256)? WriteExportArchiveFile( + string outputPath, + string snapshotPath, + ExportManifest manifest, + JsonSerializerOptions jsonOptions, + CancellationToken cancellationToken, + bool overwrite = true, + bool includeArtifactMetadata = true, + Action? beforePublishForTesting = null) { var fullOutputPath = Path.GetFullPath(outputPath); - AtomicFileWriter.Write( + void WriteContents(Stream stream) + { + cancellationToken.ThrowIfCancellationRequested(); + using var archive = new ZipArchive(stream, ZipArchiveMode.Create, leaveOpen: true); + AddTextEntry(archive, ManifestEntryName, JsonSerializer.Serialize(manifest, jsonOptions)); + var dbEntry = archive.CreateEntry(DatabaseEntryName, CompressionLevel.SmallestSize); + dbEntry.LastWriteTime = DeterministicZipTimestamp; + using var source = BoundedFile.OpenReadTrustedArchiveSource(snapshotPath); + using var target = dbEntry.Open(); + CopyToExactLength(source, target, source.Length, DatabaseEntryName, cancellationToken); + } + + (long SizeBytes, string Sha256)? artifact = null; + AtomicFileWriter.WriteWithPrePublishValidation( fullOutputPath, - stream => + WriteContents, + AtomicFileWriter.WriteProfile.Sensitive, + overwrite, + tempPath => { - cancellationToken.ThrowIfCancellationRequested(); - using var archive = new ZipArchive(stream, ZipArchiveMode.Create, leaveOpen: true); - AddTextEntry(archive, ManifestEntryName, JsonSerializer.Serialize(manifest, jsonOptions)); - var dbEntry = archive.CreateEntry(DatabaseEntryName, CompressionLevel.SmallestSize); - dbEntry.LastWriteTime = DeterministicZipTimestamp; - using var source = BoundedFile.OpenReadTrustedArchiveSource(snapshotPath); - using var target = dbEntry.Open(); - CopyToExactLength(source, target, source.Length, DatabaseEntryName, cancellationToken); + VerifyPrivateArchiveMode(tempPath); + if (includeArtifactMetadata) + artifact = ReadArchiveArtifactMetadata(tempPath, cancellationToken); + beforePublishForTesting?.Invoke(); }); + + return artifact; } internal static void WriteCtagsFile(string outputPath, Action<TextWriter> writeContents) @@ -108,4 +130,26 @@ private static string ComputeSha256(string path, CancellationToken cancellationT return Sha256StreamHasher.ComputeHex(stream, cancellationToken); } + private static (long SizeBytes, string Sha256) ReadArchiveArtifactMetadata( + string path, + CancellationToken cancellationToken) + { + cancellationToken.ThrowIfCancellationRequested(); + using var stream = BoundedFile.OpenReadForHash(path); + var sizeBytes = stream.Length; + var sha256 = Sha256StreamHasher.ComputeHex(stream, cancellationToken); + return (sizeBytes, sha256); + } + + private static void VerifyPrivateArchiveMode(string path) + { + if (OperatingSystem.IsWindows()) + return; + + DataDirectorySecurity.ApplyPrivateFileMode(path); + var mode = File.GetUnixFileMode(path) & DataDirectorySecurity.PermissionBits; + if (mode != DataDirectorySecurity.PrivateFileMode) + throw new UnauthorizedAccessException("export archive permissions could not be restricted to the current user"); + } + } diff --git a/src/CodeIndex/Cli/ExportImportCommandRunner.cs b/src/CodeIndex/Cli/ExportImportCommandRunner.cs index 825545fa2..cfad6b056 100644 --- a/src/CodeIndex/Cli/ExportImportCommandRunner.cs +++ b/src/CodeIndex/Cli/ExportImportCommandRunner.cs @@ -46,7 +46,7 @@ internal static partial class ExportImportCommandRunner private const string PhaseWriteArchive = "write_archive"; private const string PhaseWriteCtags = "write_ctags"; private const string ImportUsage = "cdidx import <archive> [--db <path>] [--prune-paths] [--dry-run|--check] [--limit <n<=10000>] [--offset <n>] [--json]"; - private const string ArchiveExportUsage = "cdidx export <archive> [--db <path>] [--json] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--project <name|path>] [--solution <path>] [--exclude-tests]"; + private const string ArchiveExportUsage = "cdidx export <archive> [--db <path>] [--json] [--overwrite] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--project <name|path>] [--solution <path>] [--exclude-tests]"; private const string CtagsExportUsage = "cdidx export ctags [--output <path>] [--db <path>] [--json] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--exclude-tests] [--include-generated]"; private const string CtagsSkipInvalidName = "invalid_name"; private const string CtagsSkipUnsupportedKind = "unsupported_kind"; diff --git a/tests/CodeIndex.Tests/ConsoleUiTests.cs b/tests/CodeIndex.Tests/ConsoleUiTests.cs index 25680db2b..39097c371 100644 --- a/tests/CodeIndex.Tests/ConsoleUiTests.cs +++ b/tests/CodeIndex.Tests/ConsoleUiTests.cs @@ -58,6 +58,22 @@ public void CompletionRenderer_UtilityCommandsExcludeQueryFlags_Issue4427() } } + [Fact] + public void ArchiveExport_HelpAndCompletionExposeExplicitOverwrite_Issue4827() + { + var overwrite = Assert.Single( + CliFlagSchema.GetCompletionFlagsForCommand("export"), + static flag => flag.Name == "--overwrite"); + Assert.Contains("atomically replace", overwrite.Description, StringComparison.OrdinalIgnoreCase); + + var (printed, stdout, stderr) = ConsoleCapture.Capture(() => + ConsoleUi.PrintCommandUsage("export") ? 1 : 0); + + Assert.Equal(1, printed); + Assert.Equal(string.Empty, stderr); + Assert.Contains("--overwrite", stdout, StringComparison.Ordinal); + } + [Fact] public void DoctorAndLicense_HelpAndCompletionExposeStructuredOutputControls_Issue4713() { diff --git a/tests/CodeIndex.Tests/ExportImportCommandRunnerTests.cs b/tests/CodeIndex.Tests/ExportImportCommandRunnerTests.cs index 2da82505a..c4350aaf3 100644 --- a/tests/CodeIndex.Tests/ExportImportCommandRunnerTests.cs +++ b/tests/CodeIndex.Tests/ExportImportCommandRunnerTests.cs @@ -757,7 +757,10 @@ public void RunExportArchive_FailureOmitsRawExceptionMessage() Directory.CreateDirectory(outputDirectory); var (exitCode, stdout, stderr) = ConsoleCapture.Capture(() => - ExportImportCommandRunner.RunExport([outputDirectory, "--db", dbPath], new JsonSerializerOptions(), "test")); + ExportImportCommandRunner.RunExport( + [outputDirectory, "--db", dbPath, "--overwrite"], + new JsonSerializerOptions(), + "test")); Assert.Equal(CommandExitCodes.UsageError, exitCode); Assert.Equal(string.Empty, stdout); @@ -846,6 +849,122 @@ public void RunExportArchive_RelativeOutputReportsAndWritesFullPath_Issue3138() } } + [Fact] + public void RunExportArchive_DefaultRefusesExistingDestination_Issue4827() + { + var projectRoot = TestProjectHelper.CreateTempProject("export_archive_existing_destination"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + var archivePath = Path.Combine(projectRoot, "codeindex.cdidx.zip"); + File.WriteAllText(archivePath, "existing archive"); + var jsonOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower }; + + var (exitCode, stdout, stderr) = ConsoleCapture.Capture(() => + ExportImportCommandRunner.RunExport([archivePath, "--db", dbPath, "--json"], jsonOptions, "test")); + + Assert.Equal(CommandExitCodes.UsageError, exitCode); + Assert.Equal(string.Empty, stderr); + using var document = JsonDocument.Parse(stdout); + Assert.Equal("write_archive", document.RootElement.GetProperty("phase").GetString()); + Assert.Equal("export_archive_exists", document.RootElement.GetProperty("error_code").GetString()); + Assert.Contains("--overwrite", document.RootElement.GetProperty("hint").GetString()); + Assert.Equal("existing archive", File.ReadAllText(archivePath)); + Assert.Empty(Directory.GetFiles(projectRoot, ".cdidx-*.tmp")); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void RunExportArchive_OverwriteReturnsVerifiablePrivateArtifactMetadata_Issue4827() + { + var projectRoot = TestProjectHelper.CreateTempProject("export_archive_artifact_metadata"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/App.cs", + "csharp", + "public class App { public void Run() { } }"); + var archivePath = Path.Combine(projectRoot, "codeindex.cdidx.zip"); + File.WriteAllText(archivePath, "replace me"); + var jsonOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower }; + + var (exitCode, stdout, stderr) = ConsoleCapture.Capture(() => + ExportImportCommandRunner.RunExport( + [archivePath, "--db", dbPath, "--overwrite", "--json"], + jsonOptions, + "test")); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + using var resultDocument = JsonDocument.Parse(stdout); + var result = resultDocument.RootElement; + Assert.Equal(new FileInfo(archivePath).Length, result.GetProperty("archive_size_bytes").GetInt64()); + Assert.Equal( + Convert.ToHexString(SHA256.HashData(File.ReadAllBytes(archivePath))).ToLowerInvariant(), + result.GetProperty("archive_sha256").GetString()); + + using var archive = ZipFile.OpenRead(archivePath); + using var manifestStream = archive.GetEntry("manifest.json")!.Open(); + using var manifestDocument = JsonDocument.Parse(manifestStream); + Assert.Equal( + manifestDocument.RootElement.GetRawText(), + result.GetProperty("manifest").GetRawText()); + Assert.True(result.GetProperty("manifest").GetProperty("file_count").GetInt64() > 0); + Assert.Equal( + result.GetProperty("manifest").GetProperty("scope").GetRawText(), + result.GetProperty("scope").GetRawText()); + + if (!OperatingSystem.IsWindows()) + { + Assert.Equal( + DataDirectorySecurity.PrivateFileMode, + File.GetUnixFileMode(archivePath) & DataDirectorySecurity.PermissionBits); + } + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void RunExportArchive_DefaultRefusesDanglingSymlinkDestination_Issue4827() + { + if (OperatingSystem.IsWindows()) + return; + + var projectRoot = TestProjectHelper.CreateTempProject("export_archive_symlink_destination"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + var archivePath = Path.Combine(projectRoot, "codeindex.cdidx.zip"); + var missingTargetPath = Path.Combine(projectRoot, "missing-target.cdidx.zip"); + File.CreateSymbolicLink(archivePath, missingTargetPath); + var jsonOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower }; + + var (exitCode, stdout, stderr) = ConsoleCapture.Capture(() => + ExportImportCommandRunner.RunExport([archivePath, "--db", dbPath, "--json"], jsonOptions, "test")); + + Assert.Equal(CommandExitCodes.UsageError, exitCode); + Assert.Equal(string.Empty, stderr); + using var document = JsonDocument.Parse(stdout); + Assert.Equal("export_archive_exists", document.RootElement.GetProperty("error_code").GetString()); + Assert.Equal(missingTargetPath, new FileInfo(archivePath).LinkTarget); + Assert.False(File.Exists(missingTargetPath)); + Assert.Empty(Directory.GetFiles(projectRoot, ".cdidx-*.tmp")); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + [Fact] public void RunExportArchive_AppliesProjectPathLanguageAndTestScope_Issue4714() { @@ -1530,6 +1649,112 @@ public void WriteExportArchiveFile_FailurePreservesExistingArchive() } } + [Fact] + public void WriteExportArchiveFile_FailureLeavesNoArtifactOrTemporaryFile_Issue4827() + { + var workDir = TestProjectHelper.CreateTempProject("cdidx_export_failed_cleanup"); + try + { + var outputPath = Path.Combine(workDir, "codeindex.cdidx.zip"); + var missingSnapshotPath = Path.Combine(workDir, "missing.db"); + var manifest = new ExportImportCommandRunner.ExportManifest( + "1", + "test", + 0, + null, + null, + new string('0', 64)); + + Assert.Throws<FileNotFoundException>(() => + ExportImportCommandRunner.WriteExportArchiveFile( + outputPath, + missingSnapshotPath, + manifest, + new JsonSerializerOptions(), + CancellationToken.None, + overwrite: false)); + + Assert.False(File.Exists(outputPath)); + Assert.Empty(Directory.GetFiles(workDir)); + } + finally + { + TestProjectHelper.DeleteDirectory(workDir); + } + } + + [Fact] + public void WriteExportArchiveFile_CanSkipArtifactMetadataForHumanOutput_Issue4827() + { + var workDir = TestProjectHelper.CreateTempProject("cdidx_export_without_attestation"); + try + { + var snapshotPath = TestProjectHelper.CreateProjectDb(workDir); + SqliteConnection.ClearAllPools(); + var outputPath = Path.Combine(workDir, "codeindex.cdidx.zip"); + var manifest = new ExportImportCommandRunner.ExportManifest( + "1", + "test", + 0, + null, + null, + new string('0', 64)); + + var artifact = ExportImportCommandRunner.WriteExportArchiveFile( + outputPath, + snapshotPath, + manifest, + new JsonSerializerOptions(), + CancellationToken.None, + overwrite: false, + includeArtifactMetadata: false); + + Assert.Null(artifact); + Assert.True(File.Exists(outputPath)); + Assert.Empty(Directory.GetFiles(workDir, ".cdidx-*.tmp")); + } + finally + { + TestProjectHelper.DeleteDirectory(workDir); + } + } + + [Fact] + public void WriteExportArchiveFile_ConcurrentDestinationCreationPreservesWinner_Issue4827() + { + var workDir = TestProjectHelper.CreateTempProject("cdidx_export_concurrent_destination"); + try + { + var snapshotPath = TestProjectHelper.CreateProjectDb(workDir); + SqliteConnection.ClearAllPools(); + var outputPath = Path.Combine(workDir, "codeindex.cdidx.zip"); + var manifest = new ExportImportCommandRunner.ExportManifest( + "1", + "test", + 0, + null, + null, + new string('0', 64)); + + Assert.Throws<AtomicFileWriter.DestinationAlreadyExistsException>(() => + ExportImportCommandRunner.WriteExportArchiveFile( + outputPath, + snapshotPath, + manifest, + new JsonSerializerOptions(), + CancellationToken.None, + overwrite: false, + beforePublishForTesting: () => File.WriteAllText(outputPath, "concurrent winner"))); + + Assert.Equal("concurrent winner", File.ReadAllText(outputPath)); + Assert.Empty(Directory.GetFiles(workDir, ".cdidx-*.tmp")); + } + finally + { + TestProjectHelper.DeleteDirectory(workDir); + } + } + [Fact] public void WriteCtagsFile_FailurePreservesExistingTagfile() {