Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions DEVELOPER_GUIDE.md

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions TESTING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,8 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding
Changelog limit tests resolve checked-in files through `RepositoryTestPaths` instead of performing another root walk.
Whole-workflow policy audits should use `RepositoryTestPaths.ReadNormalizedWorkflows()` so enumeration order, extension filtering, file naming, and normalization are shared instead of being rebuilt inside individual test classes.
When one policy test checks several step-level rules, parse workflow step blocks once and filter the retained blocks for each rule instead of rerunning the multiline step regex for every assertion family.
- `ExportImportCommandRunnerTests` issue-4827 portable-export coverage
keeps the overwrite boundary and artifact attestation deterministic. Preserve default refusal of regular and dangling-symlink destinations, explicit replacement with verifiable byte-size/SHA-256/manifest metadata and POSIX owner-only mode, failed-write temp cleanup, and a publish-boundary injection that creates a concurrent winner without using sleeps.
- `IndexCommandRunnerTests.Run_CancelDuringFreshIndex_ReturnsInterruptedJson`, `Run_CancelDuringDryRunScan_ReturnsInterruptedJson`, and `Run_CancelBeforeFreshScan_ReturnsInterruptedJson`
exercise the same in-process cancellation paths used after Ctrl-C/SIGINT wiring, including scan-time cancellation, so interrupted index runs keep returning the canonical JSON error contract.
- `IndexCommandRunnerTests.RunOptimizeFts_DryRunPreviewsWithoutWritingThenOptimizeMutates_Issue4577`, `RunOptimizeFts_LockHeld_ReportsDbLocked`, and `RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable`
Expand Down Expand Up @@ -1220,6 +1222,8 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
changelog limit test も別の root walk を行わず、`RepositoryTestPaths` 経由で checked-in file を解決します。
workflow 全体の policy audit には `RepositoryTestPaths.ReadNormalizedWorkflows()` を使い、列挙順、extension filter、file name、normalization を個別 test class 内で再構築せず共有します。
1つの policy test が複数の step-level rule を検証する場合は、workflow step block を一度だけ解析して保持し、assertion family ごとに multiline step regex を再実行せず保持済み block を絞り込みます。
- `ExportImportCommandRunnerTests` の issue-4827 portable-export coverage
overwrite 境界と artifact attestation を deterministic に固定します。通常 file と dangling symlink destination の既定拒否、明示置換で検証可能な byte size / SHA-256 / manifest metadata と POSIX owner-only mode、書き込み失敗時の temp cleanup、sleep を使わず publish 境界へ concurrent winner を生成する injection を維持してください。
- `IndexCommandRunnerTests.Run_CancelDuringFreshIndex_ReturnsInterruptedJson`、`Run_CancelDuringDryRunScan_ReturnsInterruptedJson`、`Run_CancelBeforeFreshScan_ReturnsInterruptedJson`
Ctrl-C/SIGINT 配線後に使われる in-process cancellation 経路を、scan 中のキャンセルも含めて検証し、interrupted index run が標準の JSON error contract を返し続けることを固定する。
- `IndexCommandRunnerTests.RunOptimizeFts_DryRunPreviewsWithoutWritingThenOptimizeMutates_Issue4577`、`RunOptimizeFts_LockHeld_ReportsDbLocked`、`RunOptimizeFts_ReadOnlyUri_ReturnsDbNotWritable`
Expand Down
17 changes: 17 additions & 0 deletions USER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -450,6 +450,7 @@ job:

```bash
cdidx export codeindex.cdidx.zip
cdidx export codeindex.cdidx.zip --overwrite --json
cdidx export app.cdidx.zip --project App --lang csharp --exclude-tests
cdidx export shared.cdidx.zip --path 'src/shared/*' --exclude-path 'src/shared/generated/*'
cdidx import codeindex.cdidx.zip
Expand All @@ -465,6 +466,14 @@ snapshot contains only the retained files and their dependent chunks, symbols,
references, and diagnostics, and is vacuumed before packaging. JSON output and
`manifest.json` include the requested scope, resolved project paths, and source
and exported file counts. An export without scope flags remains a full archive.
Portable export refuses an existing destination by default; pass `--overwrite`
only when replacing it is intentional. The archive is built in an owner-only
sibling temporary file and atomically published, and POSIX archives are verified
as mode `0600`. Successful JSON keeps the existing fields and additionally
returns `archive_size_bytes`, the final archive's `archive_sha256`, and the
complete immutable `manifest` object. That manifest carries the database hash,
row counts, schema contract stamps, readiness state, unknown-extension summary,
and export scope needed to evaluate the artifact before import.

The archive path is intended for trusted CodeIndex databases. Import validates
that the embedded SQLite file is a CodeIndex DB before replacing the destination
Expand Down Expand Up @@ -3548,6 +3557,7 @@ legacy database も query でき、generated-code policy は `unavailable` と

```bash
cdidx export codeindex.cdidx.zip
cdidx export codeindex.cdidx.zip --overwrite --json
cdidx export app.cdidx.zip --project App --lang csharp --exclude-tests
cdidx export shared.cdidx.zip --path 'src/shared/*' --exclude-path 'src/shared/generated/*'
cdidx import codeindex.cdidx.zip
Expand All @@ -3562,6 +3572,13 @@ archive export では `--lang`、繰り返し指定できる `--path` / `--exclu
chunk、symbol、reference、diagnostic だけを保持し、packaging 前に vacuum します。
JSON output と `manifest.json` には指定 scope、解決済み project path、元と出力後の
file count が含まれます。scope flag を指定しなければ従来どおり full archive です。
portable export は既存 destination を既定で拒否します。意図して置き換える場合だけ
`--overwrite` を指定してください。archive は owner-only の sibling temporary file に
構築して atomic に publish し、POSIX では mode `0600` であることも検証します。
成功時の JSON は既存 field を維持したまま、`archive_size_bytes`、最終 archive の
`archive_sha256`、完全で immutable な `manifest` object を追加で返します。この manifest
には import 前に artifact を評価するための database hash、row count、schema contract
stamp、readiness state、unknown-extension summary、export scope が含まれます。

archive は信頼できる CodeIndex database の共有向けです。Import は埋め込まれた
SQLite file が CodeIndex DB であることを検証してから destination database を置き換えます。
Expand Down
27 changes: 27 additions & 0 deletions changelog.d/unreleased/4827.security.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
category: security
issues:
- 4827
affected:
- src/CodeIndex/Cli/AtomicFileWriter.cs
- src/CodeIndex/Cli/ExportImportCommandRunner.ExportArchive.cs
- src/CodeIndex/Cli/ExportImportCommandRunner.Manifest.cs
- src/CodeIndex/Cli/ExportImportCommandRunner.Contracts.cs
- src/CodeIndex/Cli/ExportImportCommandRunner.cs
- src/CodeIndex/Cli/CliFlagSchema.cs
- src/CodeIndex/Cli/ConsoleUi.Help.cs
- src/CodeIndex/Cli/ConsoleUi.cs
- tests/CodeIndex.Tests/ExportImportCommandRunnerTests.cs
- tests/CodeIndex.Tests/ConsoleUiTests.cs
- USER_GUIDE.md
- DEVELOPER_GUIDE.md
- TESTING_GUIDE.md
---

## English

- **Portable archives now publish privately and require explicit replacement (#4827)** — `cdidx export` refuses an existing archive unless `--overwrite` is passed, atomically publishes through an owner-only temporary file, verifies POSIX mode `0600`, and returns final byte-size, SHA-256, and complete manifest metadata in successful JSON output.

## 日本語

- **portable archive を非公開で publish し、置換を明示必須にしました (#4827)** — `cdidx export` は `--overwrite` を指定しない限り既存 archive を拒否し、owner-only temporary file を介して atomic に publish して POSIX mode `0600` を検証します。成功時の JSON には最終 byte size、SHA-256、完全な manifest metadata も返します。
123 changes: 114 additions & 9 deletions src/CodeIndex/Cli/AtomicFileWriter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ internal static class AtomicFileWriter
{
internal const int MaxTempFileNameChars = 120;
private const int MaxTempStemChars = 48;
private const int AtCurrentWorkingDirectory = -100;
private const uint LinuxRenameNoReplace = 1;
private const uint MacRenameExclusiveFlag = 0x00000004;
internal static Action<string>? FlushParentDirectoryForTesting { get; set; }

public enum WriteProfile
Expand Down Expand Up @@ -87,7 +90,15 @@ public static void Write(string path, Action<Stream> writeContents, WriteProfile
=> WriteCore(path, writeContents, ResolveProfileModeCallback(profile), profile, overwrite: true);

public static void Write(string path, Action<Stream> writeContents, WriteProfile profile, bool overwrite)
=> WriteCore(path, writeContents, ResolveProfileModeCallback(profile), profile, overwrite);
=> WriteCore(path, writeContents, ResolveProfileModeCallback(profile), profile, overwrite, validateBeforePublish: null);

internal static void WriteWithPrePublishValidation(
string path,
Action<Stream> writeContents,
WriteProfile profile,
bool overwrite,
Action<string> validateBeforePublish)
=> WriteCore(path, writeContents, ResolveProfileModeCallback(profile), profile, overwrite, validateBeforePublish);

internal static void WritePreservingExistingOnFailure(
string path,
Expand Down Expand Up @@ -178,13 +189,13 @@ private static void WriteCore(
Action<Stream> writeContents,
Action<string>? applyFileMode,
WriteProfile profile,
bool overwrite)
bool overwrite,
Action<string>? validateBeforePublish = null)
{
ArgumentNullException.ThrowIfNull(writeContents);

var tempPath = BuildTempPath(path);
var ioTempPath = LongPath.EnsureWindowsPrefix(tempPath);
var moved = false;

try
{
Expand All @@ -195,29 +206,38 @@ private static void WriteCore(
stream.Flush(flushToDisk: true);
}

validateBeforePublish?.Invoke(tempPath);
if (overwrite)
{
MoveReplacing(tempPath, path);
}
else
{
var deleteUnixTempAfterPublish = false;
try
{
MoveFileCore(tempPath, path, overwrite: false, applyDestinationMode: null);
if (OperatingSystem.IsWindows())
{
MoveFileCore(tempPath, path, overwrite: false, applyDestinationMode: null);
}
else
{
deleteUnixTempAfterPublish = PublishUnixNoReplace(ioTempPath, path);
}
}
catch (IOException ex) when (File.Exists(LongPath.EnsureWindowsPrefix(path)))
catch (IOException ex) when (PathEntryExists(path))
{
throw new DestinationAlreadyExistsException(path, ex);
}
moved = true;

if (deleteUnixTempAfterPublish)
File.Delete(ioTempPath);
FlushParentDirectoryAfterCreate(path);
}
moved = true;
}
catch
{
if (!moved)
TryDeleteFile(ioTempPath);
TryDeleteFile(ioTempPath);
throw;
}
}
Expand All @@ -230,9 +250,80 @@ private static FileStream CreateTempFile(string path, WriteProfile profile)
return DataDirectorySecurity.OpenPrivateFileStream(path, FileMode.CreateNew, FileAccess.Write, FileShare.None);
}

private static bool PublishUnixNoReplace(string existingPath, string destinationPath)
{
try
{
// Prefer the platform no-clobber rename so publication can work on
// mounts that implement rename exclusion but do not support hard links.
int result;
if (OperatingSystem.IsLinux())
{
result = UnixRenameAt2(
AtCurrentWorkingDirectory,
existingPath,
AtCurrentWorkingDirectory,
destinationPath,
LinuxRenameNoReplace);
}
else if (OperatingSystem.IsMacOS())
{
result = MacRenameWithFlags(existingPath, destinationPath, MacRenameExclusiveFlag);
}
else
{
result = -1;
}

if (result == 0)
return false;
}
catch (EntryPointNotFoundException)
{
// Older libc implementations may not expose the no-clobber rename
// entry point. The hard-link fallback below retains atomic refusal.
}

CreateUnixHardLink(existingPath, destinationPath);
return true;
}

private static void CreateUnixHardLink(string existingPath, string linkPath)
{
if (UnixLink(existingPath, linkPath) == 0)
return;

var errno = Marshal.GetLastPInvokeError();
throw new IOException(
$"Could not publish the atomic file without replacing the destination (errno {errno}).");
}

private static Action<string>? ResolveProfileModeCallback(WriteProfile profile)
=> profile == WriteProfile.Sensitive ? DataDirectorySecurity.ApplyPrivateFileMode : null;

internal static bool PathEntryExists(string path)
{
var ioPath = LongPath.EnsureWindowsPrefix(path);
if (File.Exists(ioPath) || Directory.Exists(ioPath))
return true;

try
{
if (!string.IsNullOrEmpty(new FileInfo(ioPath).LinkTarget))
return true;

return !string.IsNullOrEmpty(new DirectoryInfo(ioPath).LinkTarget);
}
catch (FileNotFoundException)
{
return false;
}
catch (DirectoryNotFoundException)
{
return false;
}
}

internal static void MoveReplacing(string sourcePath, string destinationPath)
{
MoveFileCore(sourcePath, destinationPath, overwrite: true, applyDestinationMode: null);
Expand Down Expand Up @@ -419,6 +510,20 @@ private static void ReportCleanupFailure(Action<Exception>? failureSink, Excepti
[DllImport("libc", EntryPoint = "open", SetLastError = true)]
private static extern int UnixOpen(string path, int flags);

[DllImport("libc", EntryPoint = "link", SetLastError = true)]
private static extern int UnixLink(string existingPath, string linkPath);

[DllImport("libc", EntryPoint = "renameat2", SetLastError = true)]
private static extern int UnixRenameAt2(
int oldDirectoryFileDescriptor,
string oldPath,
int newDirectoryFileDescriptor,
string newPath,
uint flags);

[DllImport("libc", EntryPoint = "renamex_np", SetLastError = true)]
private static extern int MacRenameWithFlags(string oldPath, string newPath, uint flags);

[DllImport("libc", EntryPoint = "fsync", SetLastError = true)]
private static extern int UnixFsync(int fd);

Expand Down
2 changes: 1 addition & 1 deletion src/CodeIndex/Cli/CliFlagSchema.cs
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ private static IReadOnlyList<CliFlag> BuildAll()
new() { Name = "--description", ValuePlaceholder = "<text>", Description = "Suggestions add: local suggestion description", PrimaryCommands = Set("suggestions") },
new() { Name = "--title", ValuePlaceholder = "<title>", Description = "Suggestions add: optional issue-draft title source", PrimaryCommands = Set("suggestions") },
new() { Name = "--evidence-path", ValuePlaceholder = "<path>", Description = "Suggestions add: repository-relative evidence path; repeat for multiple paths", PrimaryCommands = Set("suggestions") },
new() { Name = "--overwrite", Description = "Atomically replace an existing report bundle or suggestions export", PrimaryCommands = Set("report", "suggestions") },
new() { Name = "--overwrite", Description = "Portable archive, report bundle, or suggestions export: atomically replace an existing output file", PrimaryCommands = Set("export", "report", "suggestions") },
new() { Name = "--body", Description = "Include definition body snippets in JSON-capable result rows", PrimaryCommands = Set(BodyCommands) },
new() { Name = "--body-start", ValuePlaceholder = "<line>", Description = "Inspect: start definition body slice at this 1-based source line", PrimaryCommands = Set(InspectFieldCommands) },
new() { Name = "--body-lines", ValuePlaceholder = "<n>", Description = "Inspect: return at most this many definition body lines", PrimaryCommands = Set(InspectFieldCommands) },
Expand Down
1 change: 1 addition & 0 deletions src/CodeIndex/Cli/ConsoleUi.Help.cs
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,7 @@ private static void PrintExamples()
Console.WriteLine(" cdidx index ./myproject --watch Run an initial scan, then keep the index live as files change (Ctrl+C to stop)");
Console.WriteLine(" cdidx export ctags --output tags Export editor tags for Vim, Emacs, and Sublime");
Console.WriteLine(" cdidx export codeindex.cdidx.zip Export a portable CodeIndex archive");
Console.WriteLine(" cdidx export codeindex.cdidx.zip --overwrite Explicitly replace an existing portable archive");
Console.WriteLine(" cdidx import codeindex.cdidx.zip Import a portable CodeIndex archive");
Console.WriteLine(" cdidx import codeindex.cdidx.zip --dry-run Validate an archive without replacing the DB");
Console.WriteLine(" cdidx search \"authenticate\" Full-text search");
Expand Down
2 changes: 1 addition & 1 deletion src/CodeIndex/Cli/ConsoleUi.cs
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ private static readonly (string Command, string Usage)[] CommandUsageLines =
("suggestions-update", "cdidx suggestions update <id> [--description <text>] [--context <text>] [--title <text>] [--evidence-path <path>] [--category <value>] [--language <lang>] [--agent <name>] [--db <path>] [--json]"),
("suggestions-update", "cdidx suggestions update <id> --status <draft|open_in_upstream|resolved_in_upstream|wont_fix|duplicate|superseded> [--actor <name>] [--reason <text>] [--db <path>] [--json]"),
("suggestions-delete", "cdidx suggestions delete <id> [--db <path>] [--json]"),
("export", "cdidx export <archive> [--db <path>] [--json] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--project <name|path>] [--solution <path>] [--exclude-tests]"),
("export", "cdidx export <archive> [--db <path>] [--json] [--overwrite] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--project <name|path>] [--solution <path>] [--exclude-tests]"),
("export", "cdidx export ctags [--output <path>] [--db <path>] [--json] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--exclude-tests] [--include-generated]"),
("export-ctags", "cdidx export ctags [--output <path>] [--db <path>] [--json] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--exclude-tests] [--include-generated]"),
("import", "cdidx import <archive> [--db <path>] [--prune-paths] [--dry-run|--check] [--limit <n<=10000>] [--offset <n>] [--json]"),
Expand Down
3 changes: 3 additions & 0 deletions src/CodeIndex/Cli/ExportImportCommandRunner.Contracts.cs
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,9 @@ internal sealed record ExportArchiveResult(
[property: JsonPropertyName("api_version")] string ApiVersion,
[property: JsonPropertyName("archive_path")] string ArchivePath,
[property: JsonPropertyName("db_path")] string DbPath,
[property: JsonPropertyName("archive_size_bytes")] long ArchiveSizeBytes,
[property: JsonPropertyName("archive_sha256")] string ArchiveSha256,
[property: JsonPropertyName("manifest")] ExportManifest Manifest,
[property: JsonPropertyName("scope")] ArchiveExportScopeResult Scope);
private sealed record ArchiveExportOptions(
string? Lang,
Expand Down
Loading
Loading