diff --git a/README.md b/README.md index dade669..ec8a157 100644 --- a/README.md +++ b/README.md @@ -45,6 +45,7 @@ aspects of the mirroring process such as; * whether the mirror is locked to a particular version of the external library. * whether the mirror is tracking a tag or a branch. * whether the mirror includes the entire external library or just a subdirectory. +* whether a subdirectory mirror uses Git partial clone to avoid unrelated blobs. ## Install @@ -73,7 +74,7 @@ unrelated staged, unstaged, and untracked work untouched and out of Braid's commits. `status`, `diff`, and `push` are the usual commands for deciding whether to pull, prepare a patch, or send local mirror changes upstream. -Use `--no-commit` with `add`, `pull`, or `remove` to stage Braid's changes +Use `--no-commit` with `add`, `pull`, `remove`, or `upgrade-config` to stage Braid's changes without creating the automatic commit. Braid stages only `.braids.json` and the selected mirror paths; unrelated staged files stay staged and will be included in your next `git commit` unless you unstage them first. @@ -110,6 +111,16 @@ braid add help braid add --help ``` +For an upstream with large blobs outside the mirrored subdirectory, opt into +Git partial clone with `braid add --path +--partial-clone`. This stores `"partial_clone": true` in config version 2 and +uses Git's `blob:none` filter for repository-local cache hydration and fetches. +The upstream server must support Git object filtering. The setting is ignored +when caching is disabled or a global cache is selected. + +Repositories with config version 1 must run `braid upgrade-config`. The command +commits the version 2 config by default; pass `--no-commit` to stage it instead. + ### Shell Completion Braid can print a Bash completion script to `stdout`: diff --git a/docs/migration-from-ruby-braid.md b/docs/migration-from-ruby-braid.md index 5e4bc36..0bf34d9 100644 --- a/docs/migration-from-ruby-braid.md +++ b/docs/migration-from-ruby-braid.md @@ -150,7 +150,7 @@ Migration impact: | Area | Ruby Braid | Current Go Braid | Migration impact | | --- | --- | --- | --- | -| Commands | `add`, `update`, `remove`, `diff`, `push`, `setup`, `version`, `status`, `upgrade-config` | `pull` is the documented mirror-update command; `update` and `up` are aliases; same other core commands, plus `sync`; no `upgrade-config` | Prefer `pull` in new docs and scripts. Scripts using `upgrade-config` must run Ruby Braid before migration or be removed. | +| Commands | `add`, `update`, `remove`, `diff`, `push`, `setup`, `version`, `status`, `upgrade-config` | `pull` is the documented mirror-update command; `update` and `up` are aliases; same other core commands, plus `sync`; `upgrade-config` migrates versioned JSON config | Prefer `pull` in new docs and scripts. Use Go Braid's `upgrade-config` only for `.braids.json` version 1 to 2 migration. | | Help form | `braid help`, `braid add help`, `braid add --help`; the old README also advertised `braid help add`, but the `v1.1.10` gem does not provide command-specific help through that form | `braid help`, `braid add help`, `braid add --help` | Prefer `braid help` or `braid --help`. | | Verbose flag | Per-command `--verbose`/`-v` | Global `--verbose`/`-v` before the command | Use `braid -v pull ...`, not `braid pull -v ...`. | | Quiet flag | No global quiet flag | Global `--quiet` before the command; incompatible with `--verbose` | Use `braid --quiet ...` in automation that wants data, warnings, errors, and recovery output without progress or informational chatter. | @@ -165,7 +165,8 @@ Migration impact: ## Configuration Compatibility The Go tool supports only the modern `.braids.json` shape with -`config_version: 1` and a `mirrors` object keyed by local mirror path. +`config_version: 2` and a `mirrors` object keyed by local mirror path. Version 1 +JSON config can be migrated with `braid upgrade-config`. Supported mirror attributes remain: @@ -174,12 +175,13 @@ Supported mirror attributes remain: - `tag` - `path` - `revision` +- `partial_clone` (optional; requires `path`) Important differences: - Legacy `.braids` YAML/PStore config is not read or upgraded. - Older `.braids.json` layouts without `config_version` are not upgraded. -- `upgrade-config` is not implemented. +- `upgrade-config` does not migrate legacy `.braids` YAML/PStore data. - Unknown root fields and unknown mirror fields fail validation. - Missing `config_version`, missing `mirrors`, missing `url`, missing `revision`, or a mirror with both `branch` and `tag` fails validation. diff --git a/integration/lifecycle_test.go b/integration/lifecycle_test.go index 6d7cea5..fe6e309 100644 --- a/integration/lifecycle_test.go +++ b/integration/lifecycle_test.go @@ -1,6 +1,7 @@ package integration import ( + "os" "path/filepath" "testing" ) @@ -135,3 +136,115 @@ func TestExecutablePrimaryLifecycle(t *testing.T) { assertNoRemote(t, env, downstream, remote) assertClean(t, env, downstream) } + +func TestUpgradeConfigCommitAndNoCommit(t *testing.T) { + for _, noCommit := range []bool{false, true} { + t.Run(map[bool]string{false: "commit", true: "no-commit"}[noCommit], func(t *testing.T) { + root := t.TempDir() + env := newProcessEnv(t, root) + braid := braidBinary(t) + repo := filepath.Join(root, "repo") + initRepo(t, env, repo) + if err := os.WriteFile(filepath.Join(repo, ".braids.json"), []byte("{\"config_version\":1,\"mirrors\":{}}\n"), 0o644); err != nil { + t.Fatal(err) + } + gitOK(t, env, repo, "add", ".braids.json") + gitOK(t, env, repo, "commit", "-m", "version one") + before := gitOutput(t, env, repo, "rev-parse", "HEAD") + args := []string{"upgrade-config"} + if noCommit { + args = append(args, "--no-commit") + } + result := runBraid(t, env, repo, braid, args...) + assertExit(t, result, 0) + if noCommit { + if got := gitOutput(t, env, repo, "rev-parse", "HEAD"); got != before { + t.Fatalf("HEAD moved: %s", got) + } + assertContains(t, gitOK(t, env, repo, "diff", "--cached").stdout, `"config_version": 2`) + } else { + assertLatestCommit(t, env, repo, defaultName+" <"+defaultEmail+">", "Upgrade Braid config to version 2") + assertClean(t, env, repo) + } + }) + } +} + +func TestPartialCloneSubdirectoryLifecycle(t *testing.T) { + root := t.TempDir() + env := newProcessEnv(t, root) + braid := braidBinary(t) + upstream := filepath.Join(root, "upstream") + initRepo(t, env, upstream) + gitOK(t, env, upstream, "config", "uploadpack.allowFilter", "true") + gitOK(t, env, upstream, "config", "receive.denyCurrentBranch", "updateInstead") + writeFile(t, upstream, "wanted/file.txt", "one\n") + writeFile(t, upstream, "other/large.txt", string(make([]byte, 1024*1024))) + first := commitAll(t, env, upstream, "initial") + unrelatedBlob := gitOutput(t, env, upstream, "rev-parse", "HEAD:other/large.txt") + + downstream := filepath.Join(root, "downstream") + initRepo(t, env, downstream) + writeFile(t, downstream, "README.md", "downstream\n") + commitAll(t, env, downstream, "initial") + + result := runBraid(t, env, downstream, braid, "add", upstream, "vendor/wanted", "--path", "wanted", "--partial-clone") + assertExit(t, result, 0) + assertFile(t, downstream, "vendor/wanted/file.txt", "one\n") + assertConfigRaw(t, downstream, map[string]configMirror{ + "vendor/wanted": {URL: upstream, Branch: "main", Path: "wanted", Revision: first, PartialClone: true}, + }) + cachePath := repositoryCachePath(t, downstream, "vendor/wanted", configMirror{URL: upstream, Branch: "main", Path: "wanted"}) + missing := runProcess(t, env.with("GIT_NO_LAZY_FETCH", "1"), cachePath, "git", "cat-file", "-e", unrelatedBlob) + if missing.exitCode == 0 { + t.Fatalf("unrelated blob %s unexpectedly present in partial cache", unrelatedBlob) + } + missing = runProcess(t, env.with("GIT_NO_LAZY_FETCH", "1"), downstream, "git", "cat-file", "-e", unrelatedBlob) + if missing.exitCode == 0 { + t.Fatalf("unrelated blob %s unexpectedly present downstream", unrelatedBlob) + } + + writeFile(t, downstream, "vendor/wanted/file.txt", "pushed\n") + commitAll(t, env, downstream, "local partial mirror change") + result = runBraid(t, env, downstream, braid, "push", "vendor/wanted", "--message", "Push partial mirror") + assertExit(t, result, 0) + assertFile(t, upstream, "wanted/file.txt", "pushed\n") + assertFile(t, upstream, "other/large.txt", string(make([]byte, 1024*1024))) + assertLatestCommit(t, env, upstream, defaultName+" <"+defaultEmail+">", "Push partial mirror") + missing = runProcess(t, env.with("GIT_NO_LAZY_FETCH", "1"), cachePath, "git", "cat-file", "-e", unrelatedBlob) + if missing.exitCode == 0 { + t.Fatalf("unrelated blob %s unexpectedly present in partial cache after push", unrelatedBlob) + } + result = runBraid(t, env, downstream, braid, "pull", "vendor/wanted") + assertExit(t, result, 0) + assertFile(t, downstream, "vendor/wanted/file.txt", "pushed\n") + + writeFile(t, upstream, "wanted/file.txt", "two\n") + second := commitAll(t, env, upstream, "update") + result = runBraid(t, env, downstream, braid, "pull", "vendor/wanted") + assertExit(t, result, 0) + assertFile(t, downstream, "vendor/wanted/file.txt", "two\n") + assertConfigRaw(t, downstream, map[string]configMirror{ + "vendor/wanted": {URL: upstream, Branch: "main", Path: "wanted", Revision: second, PartialClone: true}, + }) + assertClean(t, env, downstream) +} + +func TestPartialCloneRejectsUpstreamWithoutFilterSupport(t *testing.T) { + root := t.TempDir() + env := newProcessEnv(t, root) + braid := braidBinary(t) + upstream := filepath.Join(root, "upstream") + initRepo(t, env, upstream) + writeFile(t, upstream, "wanted/file.txt", "content\n") + commitAll(t, env, upstream, "initial") + downstream := filepath.Join(root, "downstream") + initRepo(t, env, downstream) + writeFile(t, downstream, "README.md", "downstream\n") + commitAll(t, env, downstream, "initial") + + result := runBraid(t, env, downstream, braid, "add", upstream, "vendor/wanted", "--path", "wanted", "--partial-clone") + assertExit(t, result, 1) + assertContains(t, result.stderr, "does not support partial clone filtering") + assertClean(t, env, downstream) +} diff --git a/integration/scoped_state_test.go b/integration/scoped_state_test.go index 6e35886..56230d3 100644 --- a/integration/scoped_state_test.go +++ b/integration/scoped_state_test.go @@ -113,7 +113,7 @@ func TestExecutableScopedAddRemovePrecheckBlockers(t *testing.T) { }, dirty: func(t *testing.T, downstream string) { t.Helper() - writeFile(t, downstream, ".braids.json", "{\"config_version\":1,\"mirrors\":{}}\n") + writeFile(t, downstream, ".braids.json", "{\"config_version\":2,\"mirrors\":{}}\n") }, args: []string{"add", "$upstream", "vendor/basic"}, wantErr: "local changes are present in .braids.json", diff --git a/integration/support.go b/integration/support.go index 1f87cb1..41482bf 100644 --- a/integration/support.go +++ b/integration/support.go @@ -280,16 +280,17 @@ type configFile struct { } type configMirror struct { - URL string `json:"url"` - Branch string `json:"branch,omitempty"` - Path string `json:"path,omitempty"` - Tag string `json:"tag,omitempty"` - Revision string `json:"revision"` + URL string `json:"url"` + Branch string `json:"branch,omitempty"` + Path string `json:"path,omitempty"` + Tag string `json:"tag,omitempty"` + Revision string `json:"revision"` + PartialClone bool `json:"partial_clone,omitempty"` } func expectedConfigRaw(t *testing.T, mirrors map[string]configMirror) string { t.Helper() - data, err := json.MarshalIndent(configFile{ConfigVersion: 1, Mirrors: mirrors}, "", " ") + data, err := json.MarshalIndent(configFile{ConfigVersion: 2, Mirrors: mirrors}, "", " ") if err != nil { t.Fatalf("marshal expected config: %v", err) } @@ -311,8 +312,8 @@ func assertConfigRaw(t *testing.T, repo string, mirrors map[string]configMirror) if err := json.Unmarshal(data, &parsed); err != nil { t.Fatalf("parse .braids.json: %v", err) } - if parsed.ConfigVersion != 1 || len(parsed.Mirrors) != len(mirrors) { - t.Fatalf(".braids.json semantic parse = %#v, want version 1 and %d mirrors", parsed, len(mirrors)) + if parsed.ConfigVersion != 2 || len(parsed.Mirrors) != len(mirrors) { + t.Fatalf(".braids.json semantic parse = %#v, want version 2 and %d mirrors", parsed, len(mirrors)) } } diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 6ae2a13..be9b068 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -11,17 +11,18 @@ var DefaultVersion = "0.0.0-dev" type Command string const ( - CommandAdd Command = "add" - CommandPull Command = "pull" - CommandRemove Command = "remove" - CommandDiff Command = "diff" - CommandPush Command = "push" - CommandSync Command = "sync" - CommandSetup Command = "setup" - CommandVersion Command = "version" - CommandStatus Command = "status" - CommandCompletion Command = "completion" - CommandComplete Command = "__complete" + CommandAdd Command = "add" + CommandPull Command = "pull" + CommandRemove Command = "remove" + CommandDiff Command = "diff" + CommandPush Command = "push" + CommandSync Command = "sync" + CommandSetup Command = "setup" + CommandVersion Command = "version" + CommandStatus Command = "status" + CommandCompletion Command = "completion" + CommandComplete Command = "__complete" + CommandUpgradeConfig Command = "upgrade-config" ) type GlobalOptions struct { @@ -33,15 +34,18 @@ type GlobalOptions struct { } type AddOptions struct { - URL string - LocalPath string - Branch string - Tag string - Revision string - RemotePath string - NoCommit bool + URL string + LocalPath string + Branch string + Tag string + Revision string + RemotePath string + NoCommit bool + PartialClone bool } +type UpgradeConfigOptions struct{ NoCommit bool } + type UpdateOptions struct { LocalPath string Branch string @@ -100,16 +104,17 @@ type Invocation struct { Command Command Help bool - Add AddOptions - Update UpdateOptions - Remove RemoveOptions - Diff DiffOptions - Push PushOptions - Sync SyncOptions - Setup SetupOptions - Status StatusOptions - Completion CompletionOptions - Complete CompleteOptions + Add AddOptions + Update UpdateOptions + Remove RemoveOptions + Diff DiffOptions + Push PushOptions + Sync SyncOptions + Setup SetupOptions + Status StatusOptions + Completion CompletionOptions + Complete CompleteOptions + UpgradeConfig UpgradeConfigOptions } type Handler interface { @@ -131,14 +136,15 @@ func New() App { return App{ Version: DefaultVersion, Handler: map[Command]Handler{ - CommandAdd: notImplemented(CommandAdd), - CommandPull: notImplemented(CommandPull), - CommandRemove: notImplemented(CommandRemove), - CommandDiff: notImplemented(CommandDiff), - CommandPush: notImplemented(CommandPush), - CommandSync: notImplemented(CommandSync), - CommandSetup: notImplemented(CommandSetup), - CommandStatus: notImplemented(CommandStatus), + CommandAdd: notImplemented(CommandAdd), + CommandPull: notImplemented(CommandPull), + CommandRemove: notImplemented(CommandRemove), + CommandDiff: notImplemented(CommandDiff), + CommandPush: notImplemented(CommandPush), + CommandSync: notImplemented(CommandSync), + CommandSetup: notImplemented(CommandSetup), + CommandStatus: notImplemented(CommandStatus), + CommandUpgradeConfig: notImplemented(CommandUpgradeConfig), }, } } @@ -272,6 +278,8 @@ func Parse(args []string) (Invocation, error) { return inv, parseCompletion(commandArgs, &inv.Completion) case CommandComplete: return inv, parseComplete(commandArgs, &inv.Complete) + case CommandUpgradeConfig: + return inv, parseUpgradeConfig(commandArgs, &inv.UpgradeConfig) default: return inv, usageError("unknown command %s", commandText) } @@ -326,6 +334,7 @@ func parseAdd(args []string, options *AddOptions) error { valueFlag("--revision", "-r", "revision", func(value string) { options.Revision = value }), valueFlag("--path", "-p", "path", func(value string) { options.RemotePath = value }), boolFlag("--no-commit", "", func() { options.NoCommit = true }), + boolFlag("--partial-clone", "", func() { options.PartialClone = true }), }, func(pos []string, _ []string) { positionals = pos }) @@ -341,6 +350,9 @@ func parseAdd(args []string, options *AddOptions) error { if options.Tag != "" && options.Revision != "" { return usageError("add cannot combine --tag and --revision") } + if options.PartialClone && options.RemotePath == "" { + return usageError("add --partial-clone requires --path") + } options.URL = positionals[0] if len(positionals) == 2 { options.LocalPath = normalizeLocalPathArg(positionals[1]) @@ -520,6 +532,19 @@ func parseComplete(args []string, options *CompleteOptions) error { return nil } +func parseUpgradeConfig(args []string, options *UpgradeConfigOptions) error { + var positionals []string + err := parseCommandArgs(CommandUpgradeConfig, args, []flagSpec{ + boolFlag("--no-commit", "", func() { options.NoCommit = true }), + }, func(pos []string, _ []string) { + positionals = pos + }) + if err != nil { + return err + } + return requireArgRange("upgrade-config", positionals, 0, 0) +} + func normalizeLocalPathArg(value string) string { return strings.ReplaceAll(value, `\`, "/") } @@ -620,6 +645,8 @@ func parseCommand(value string) (Command, bool) { return CommandCompletion, true case string(CommandComplete): return CommandComplete, true + case string(CommandUpgradeConfig): + return CommandUpgradeConfig, true default: return "", false } @@ -666,6 +693,8 @@ commands: version Show braid version completion Print Bash completion script + upgrade-config + Upgrade .braids.json to the current version Run "braid help" for command-specific usage. `, "\n") @@ -674,7 +703,7 @@ Run "braid help" for command-specific usage. func CommandUsage(command Command) string { switch command { case CommandAdd: - return "usage: braid add [local_path] [--branch|-b ] [--tag|-t ] [--revision|-r ] [--path|-p ] [--no-commit]\n" + return "usage: braid add [local_path] [--branch|-b ] [--tag|-t ] [--revision|-r ] [--path|-p ] [--no-commit] [--partial-clone]\n" case CommandPull: return "usage: braid pull [local_path] [--branch|-b ] [--tag|-t ] [--revision|-r ] [--keep] [--no-commit]\n" case CommandRemove: @@ -693,6 +722,8 @@ func CommandUsage(command Command) string { return "usage: braid status [local_path]\n" case CommandCompletion: return "usage: braid completion bash\n" + case CommandUpgradeConfig: + return "usage: braid upgrade-config [--no-commit]\n" default: return Usage() } diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go index 62934bd..1087a7f 100644 --- a/internal/cli/cli_test.go +++ b/internal/cli/cli_test.go @@ -316,7 +316,7 @@ func TestUsageDocumentsVerboseAsGlobalOnly(t *testing.T) { if !strings.Contains(Usage(), " pull Pull one mirror or every eligible mirror") { t.Fatalf("top-level usage missing pull command:\n%s", Usage()) } - if strings.Contains(Usage(), " update") || strings.Contains(Usage(), " up") { + if strings.Contains(Usage(), " update") || strings.Contains(Usage(), "\n up ") { t.Fatalf("top-level usage exposes update aliases:\n%s", Usage()) } if !strings.Contains(Usage(), " sync Push local mirror changes, then pull mirrors") { @@ -328,7 +328,7 @@ func TestUsageDocumentsVerboseAsGlobalOnly(t *testing.T) { if strings.Contains(Usage(), "__complete") { t.Fatalf("top-level usage exposes hidden completion callback:\n%s", Usage()) } - if got, want := CommandUsage(CommandAdd), "usage: braid add [local_path] [--branch|-b ] [--tag|-t ] [--revision|-r ] [--path|-p ] [--no-commit]\n"; got != want { + if got, want := CommandUsage(CommandAdd), "usage: braid add [local_path] [--branch|-b ] [--tag|-t ] [--revision|-r ] [--path|-p ] [--no-commit] [--partial-clone]\n"; got != want { t.Fatalf("CommandUsage(add) = %q, want %q", got, want) } if got, want := CommandUsage(CommandPull), "usage: braid pull [local_path] [--branch|-b ] [--tag|-t ] [--revision|-r ] [--keep] [--no-commit]\n"; got != want { diff --git a/internal/command/BUILD.bazel b/internal/command/BUILD.bazel index ce42c72..2812c55 100644 --- a/internal/command/BUILD.bazel +++ b/internal/command/BUILD.bazel @@ -19,6 +19,7 @@ go_library( "status.go", "sync.go", "update.go", + "upgrade_config.go", ], importpath = "braid/internal/command", visibility = ["//visibility:public"], diff --git a/internal/command/add.go b/internal/command/add.go index 60f1492..882351d 100644 --- a/internal/command/add.go +++ b/internal/command/add.go @@ -57,11 +57,12 @@ func (h AddHandler) add(ctx context.Context, repo RepoContext, git AddGit, inv c } m, err := mirror.NewFromOptions(addOptions.URL, mirror.Options{ - LocalPath: addOptions.LocalPath, - Branch: addOptions.Branch, - Tag: addOptions.Tag, - Revision: addOptions.Revision, - RemotePath: addOptions.RemotePath, + LocalPath: addOptions.LocalPath, + Branch: addOptions.Branch, + Tag: addOptions.Tag, + Revision: addOptions.Revision, + RemotePath: addOptions.RemotePath, + PartialClone: addOptions.PartialClone, }) if err != nil { return err diff --git a/internal/command/add_test.go b/internal/command/add_test.go index 59c30b9..b4001e1 100644 --- a/internal/command/add_test.go +++ b/internal/command/add_test.go @@ -317,7 +317,7 @@ func TestAddCommandScopedPrecheckBlocksDirtyConfig(t *testing.T) { } testutil.Git(t, repo, "add", config.FileName) testutil.Git(t, repo, "commit", "-m", "add empty braid config") - testutil.WriteFile(t, repo, config.FileName, "{\"config_version\":1,\"mirrors\":{}}\n") + testutil.WriteFile(t, repo, config.FileName, "{\"config_version\":2,\"mirrors\":{}}\n") stderr := runCommandError(t, repo, []string{"add", upstream, "vendor/basic"}) assertContains(t, stderr, "local changes are present in .braids.json") @@ -331,7 +331,7 @@ func TestAddCommandScopedPrecheckRunsBeforeDefaultBranchLookup(t *testing.T) { } testutil.Git(t, repo, "add", config.FileName) testutil.Git(t, repo, "commit", "-m", "add empty braid config") - testutil.WriteFile(t, repo, config.FileName, "{\"config_version\":1,\"mirrors\":{}}\n") + testutil.WriteFile(t, repo, config.FileName, "{\"config_version\":2,\"mirrors\":{}}\n") missingUpstream := filepath.Join(t.TempDir(), "missing-upstream") stderr := runCommandError(t, repo, []string{"add", missingUpstream, "vendor/basic"}) diff --git a/internal/command/cache.go b/internal/command/cache.go index 025ca55..29e8c43 100644 --- a/internal/command/cache.go +++ b/internal/command/cache.go @@ -170,6 +170,9 @@ func (cache CacheConfig) RemoteURL(m mirror.Mirror) string { case CacheModeGlobal: return CachePath(cache.Dir, m.URL) case CacheModeRepositoryLocal: + if m.PartialClone { + return m.URL + } return RepositoryCachePath(cache.Dir, m) default: return m.URL @@ -189,14 +192,14 @@ func (cache CacheConfig) TipRef(m mirror.Mirror) string { } func cacheResolveRecordedRevision(cache CacheConfig, m mirror.Mirror, requested string) string { - if requested != "" && cache.Enabled && cache.Mode == CacheModeRepositoryLocal { + if requested != "" && cache.Enabled && cache.Mode == CacheModeRepositoryLocal && !m.PartialClone { return cache.RecordedRef(m) } return requested } func cacheResolveRequestedRevision(cache CacheConfig, m mirror.Mirror, requested string) string { - if requested != "" && cache.Enabled && cache.Mode == CacheModeRepositoryLocal { + if requested != "" && cache.Enabled && cache.Mode == CacheModeRepositoryLocal && !m.PartialClone { return cache.RequestedRef(m) } return requested @@ -237,14 +240,20 @@ func fetchMirror(ctx context.Context, git fetchGit, cache CacheConfig, m mirror. fmt.Sprintf("Braid: fetched mirror %s", m.Path), func() error { if cache.Enabled && cache.Mode == CacheModeRepositoryLocal { - args := []string{"--update-shallow", "-n", m.Remote()} + args := []string{"--update-shallow", "-n"} + if m.PartialClone { + args = append(args, "--filter=blob:none") + } + args = append(args, m.Remote()) if m.Branch != "" { args = append(args, "+refs/heads/"+m.Branch+":refs/remotes/"+m.Remote()+"/"+m.Branch) } if m.Tag != "" { args = append(args, "+refs/tags/"+m.Tag+":"+m.LocalRef()) } - args = append(args, "+refs/braid/*:refs/braid/*") + if !m.PartialClone { + args = append(args, "+refs/braid/*:refs/braid/*") + } return git.Fetch(ctx, args...) } if err := git.Fetch(ctx, "-n", m.Remote()); err != nil { @@ -289,16 +298,35 @@ func (cache MirrorObjectCache) hydrateRepositoryLocal(ctx context.Context, m mir } defer release() - if err := cache.ensureRepositoryLocalCache(ctx, cachePath); err != nil { + backupPath, err := cache.ensureRepositoryLocalCache(ctx, cachePath, m) + if err != nil { + return err + } + hydrateErr := cache.hydrateRepositoryLocalReady(ctx, cachePath, m, extraRevisions...) + if backupPath == "" { + return hydrateErr + } + if hydrateErr != nil { + _ = os.RemoveAll(cachePath) + if restoreErr := os.Rename(backupPath, cachePath); restoreErr != nil { + return fmt.Errorf("%w; failed to restore previous cache: %w", hydrateErr, restoreErr) + } + return hydrateErr + } + if err := os.RemoveAll(backupPath); err != nil { return err } + return nil +} + +func (cache MirrorObjectCache) hydrateRepositoryLocalReady(ctx context.Context, cachePath string, m mirror.Mirror, extraRevisions ...string) error { cacheGit := gitexec.New(cachePath, cache.Verbose, cache.Trace) full := false recordedRevision := strings.TrimSpace(m.Revision) if recordedRevision != "" { if isFullObjectID(recordedRevision) { - if err := cache.fetchFullObjectID(ctx, cacheGit, m.URL, recordedRevision, cache.Config.RecordedRef(m)); err != nil { + if err := cache.fetchFullObjectID(ctx, cacheGit, m, recordedRevision, cache.Config.RecordedRef(m)); err != nil { full = true if err := cache.fetchFullMirror(ctx, cachePath, cacheGit, m); err != nil { return err @@ -325,7 +353,7 @@ func (cache MirrorObjectCache) hydrateRepositoryLocal(ctx context.Context, m mir continue } if isFullObjectID(revision) { - if err := cache.fetchFullObjectID(ctx, cacheGit, m.URL, revision, cache.Config.RequestedRef(m)); err != nil { + if err := cache.fetchFullObjectID(ctx, cacheGit, m, revision, cache.Config.RequestedRef(m)); err != nil { full = true if err := cache.fetchFullMirror(ctx, cachePath, cacheGit, m); err != nil { return err @@ -349,7 +377,7 @@ func (cache MirrorObjectCache) hydrateRepositoryLocal(ctx context.Context, m mir switch { case m.Branch != "": if !full { - if err := cacheGit.Fetch(ctx, "--depth=1", m.URL, "+refs/heads/"+m.Branch+":refs/heads/"+m.Branch); err != nil { + if err := cache.fetchRepositoryLocal(ctx, cacheGit, m, "--depth=1", "+refs/heads/"+m.Branch+":refs/heads/"+m.Branch); err != nil { return err } } @@ -357,10 +385,13 @@ func (cache MirrorObjectCache) hydrateRepositoryLocal(ctx context.Context, m mir if err != nil { return err } - return cacheGit.UpdateRef(ctx, cache.Config.TipRef(m), resolved) + if err := cacheGit.UpdateRef(ctx, cache.Config.TipRef(m), resolved); err != nil { + return err + } + return cache.materializeRemotePath(ctx, cacheGit, m, resolved) case m.Tag != "": if !full { - if err := cacheGit.Fetch(ctx, "--depth=1", m.URL, "+refs/tags/"+m.Tag+":refs/tags/"+m.Tag); err != nil { + if err := cache.fetchRepositoryLocal(ctx, cacheGit, m, "--depth=1", "+refs/tags/"+m.Tag+":refs/tags/"+m.Tag); err != nil { return err } } @@ -368,42 +399,83 @@ func (cache MirrorObjectCache) hydrateRepositoryLocal(ctx context.Context, m mir if err != nil { return err } - return cacheGit.UpdateRef(ctx, cache.Config.TipRef(m), resolved) + if err := cacheGit.UpdateRef(ctx, cache.Config.TipRef(m), resolved); err != nil { + return err + } + return cache.materializeRemotePath(ctx, cacheGit, m, resolved) } return nil } -func (cache MirrorObjectCache) ensureRepositoryLocalCache(ctx context.Context, cachePath string) error { +func (cache MirrorObjectCache) materializeRemotePath(ctx context.Context, git gitexec.Git, m mirror.Mirror, revision string) error { + if !m.PartialClone { + return nil + } + result, err := git.RunOK(ctx, "rev-list", "--objects", revision, "--", m.RemotePath) + if err != nil { + return err + } + for _, line := range strings.Split(result.Stdout, "\n") { + fields := strings.Fields(line) + if len(fields) == 0 { + continue + } + if _, err := git.RunOK(ctx, "cat-file", "-e", fields[0]); err != nil { + return err + } + } + return nil +} + +func (cache MirrorObjectCache) ensureRepositoryLocalCache(ctx context.Context, cachePath string, m mirror.Mirror) (string, error) { + backupPath := "" + recoveryPath := cachePath + ".backup" + if _, err := os.Stat(recoveryPath); err == nil { + _ = os.RemoveAll(cachePath) + if err := os.Rename(recoveryPath, cachePath); err != nil { + return "", err + } + } else if err != nil && !errors.Is(err, os.ErrNotExist) { + return "", err + } if info, err := os.Stat(cachePath); err == nil { replace := !info.IsDir() if !replace { if _, err := os.Stat(filepath.Join(cachePath, ".git")); err == nil { replace = true } else if err != nil && !errors.Is(err, os.ErrNotExist) { - return err + return "", err } } if !replace { bare, err := isBareRepository(ctx, cachePath, cache.Verbose, cache.Trace) if err == nil && bare { - return nil + cacheGit := gitexec.New(cachePath, cache.Verbose, cache.Trace) + matches, configErr := repositoryLocalCacheMatches(ctx, cacheGit, m) + if configErr != nil { + return "", configErr + } + if matches { + return "", cache.configureRepositoryLocalCache(ctx, cacheGit, m) + } } replace = true } if replace { - if err := os.RemoveAll(cachePath); err != nil { - return err + backupPath = recoveryPath + if err := os.Rename(cachePath, backupPath); err != nil { + return "", err } } } else if !errors.Is(err, os.ErrNotExist) { - return err + return "", err } if err := os.MkdirAll(filepath.Dir(cachePath), 0o755); err != nil { - return err + return "", err } tempPath, err := os.MkdirTemp(filepath.Dir(cachePath), ".tmp-"+filepath.Base(cachePath)+"-") if err != nil { - return err + return "", err } cleanup := true defer func() { @@ -412,12 +484,80 @@ func (cache MirrorObjectCache) ensureRepositoryLocalCache(ctx context.Context, c } }() if err := gitexec.New(".", cache.Verbose, cache.Trace).InitBare(ctx, tempPath); err != nil { - return err + return "", err } if err := os.Rename(tempPath, cachePath); err != nil { - return err + return "", err } cleanup = false + if err := cache.configureRepositoryLocalCache(ctx, gitexec.New(cachePath, cache.Verbose, cache.Trace), m); err != nil { + _ = os.RemoveAll(cachePath) + if backupPath != "" { + _ = os.Rename(backupPath, cachePath) + } + return "", err + } + return backupPath, nil +} + +func repositoryLocalCacheMatches(ctx context.Context, git gitexec.Git, m mirror.Mirror) (bool, error) { + mode, modeSet, err := git.ConfigGet(ctx, "--bool", "braid.partialClone") + if err != nil { + return false, err + } + if !modeSet { + _, versionSet, err := git.ConfigGet(ctx, "braid.cacheVersion") + if err != nil { + return false, err + } + return !versionSet && !m.PartialClone, nil + } + version, versionSet, err := git.ConfigGet(ctx, "braid.cacheVersion") + if err != nil { + return false, err + } + if !versionSet || version != "1" || mode != fmt.Sprint(m.PartialClone) { + return false, nil + } + if !m.PartialClone { + return true, nil + } + checks := []struct{ key, want string }{ + {"remote.braid-upstream.url", m.URL}, + {"remote.braid-upstream.promisor", "true"}, + {"remote.braid-upstream.partialclonefilter", "blob:none"}, + } + for _, check := range checks { + value, ok, err := git.ConfigGet(ctx, check.key) + if err != nil { + return false, err + } + if !ok || value != check.want { + return false, nil + } + } + return true, nil +} + +func (cache MirrorObjectCache) configureRepositoryLocalCache(ctx context.Context, git gitexec.Git, m mirror.Mirror) error { + if err := git.ConfigSet(ctx, "braid.cacheVersion", "1"); err != nil { + return err + } + if err := git.ConfigSet(ctx, "braid.partialClone", fmt.Sprint(m.PartialClone)); err != nil { + return err + } + if !m.PartialClone { + return nil + } + if err := git.ConfigSet(ctx, "remote.braid-upstream.url", m.URL); err != nil { + return err + } + if err := git.ConfigSet(ctx, "remote.braid-upstream.promisor", "true"); err != nil { + return err + } + if err := git.ConfigSet(ctx, "remote.braid-upstream.partialclonefilter", "blob:none"); err != nil { + return err + } return nil } @@ -429,8 +569,8 @@ func isBareRepository(ctx context.Context, path string, verbose bool, trace ioWr return out == "true", nil } -func (cache MirrorObjectCache) fetchFullObjectID(ctx context.Context, git gitexec.Git, url, objectID, ref string) error { - return git.Fetch(ctx, "--depth=1", url, objectID+":"+ref) +func (cache MirrorObjectCache) fetchFullObjectID(ctx context.Context, git gitexec.Git, m mirror.Mirror, objectID, ref string) error { + return cache.fetchRepositoryLocal(ctx, git, m, "--depth=1", objectID+":"+ref) } func (cache MirrorObjectCache) fetchFullMirror(ctx context.Context, cachePath string, git gitexec.Git, m mirror.Mirror) error { @@ -440,8 +580,27 @@ func (cache MirrorObjectCache) fetchFullMirror(ctx context.Context, cachePath st } else if err != nil && !errors.Is(err, os.ErrNotExist) { return err } - args = append(args, m.URL, "+refs/*:refs/*") - return git.Fetch(ctx, args...) + args = append(args, "+refs/*:refs/*") + return cache.fetchRepositoryLocal(ctx, git, m, args...) +} + +func (cache MirrorObjectCache) fetchRepositoryLocal(ctx context.Context, git gitexec.Git, m mirror.Mirror, args ...string) error { + refspec := args[len(args)-1] + options := args[:len(args)-1] + remote := m.URL + if m.PartialClone { + remote = "braid-upstream" + options = append([]string{"--filter=blob:none"}, options...) + } + args = append(append(options, remote), refspec) + result, err := git.FetchResult(ctx, args...) + if err != nil { + return err + } + if m.PartialClone && strings.Contains(result.Stderr, "filtering not recognized") { + return fmt.Errorf("upstream %s does not support partial clone filtering", m.URL) + } + return nil } func acquireCacheLock(ctx context.Context, lockPath, mirrorPath string) (func(), error) { diff --git a/internal/command/completion.go b/internal/command/completion.go index 7e7cc12..c3f8778 100644 --- a/internal/command/completion.go +++ b/internal/command/completion.go @@ -73,6 +73,7 @@ var commandCompletionFlags = map[string][]completionFlag{ {long: "--revision", short: "-r", value: true}, {long: "--path", short: "-p", value: true}, {long: "--no-commit"}, + {long: "--partial-clone"}, }, string(cli.CommandPull): { {long: "--branch", short: "-b", value: true}, @@ -102,7 +103,8 @@ var commandCompletionFlags = map[string][]completionFlag{ string(cli.CommandSetup): { {long: "--force", short: "-f"}, }, - string(cli.CommandStatus): {}, + string(cli.CommandStatus): {}, + string(cli.CommandUpgradeConfig): {{long: "--no-commit"}}, } var rootCompletionCommands = []string{ @@ -116,6 +118,7 @@ var rootCompletionCommands = []string{ string(cli.CommandStatus), string(cli.CommandVersion), string(cli.CommandCompletion), + string(cli.CommandUpgradeConfig), "help", } diff --git a/internal/command/completion_test.go b/internal/command/completion_test.go index 3d1643a..91e2444 100644 --- a/internal/command/completion_test.go +++ b/internal/command/completion_test.go @@ -226,7 +226,7 @@ func assertNoCandidate(t *testing.T, candidates []string, unwanted string) { func writeCompletionConfig(t *testing.T, repo string) { t.Helper() data := []byte(`{ - "config_version": 1, + "config_version": 2, "mirrors": { "apps/web/vendor/local": { "url": "https://example.test/local.git", diff --git a/internal/command/preflight.go b/internal/command/preflight.go index e6e89e1..57bd6d6 100644 --- a/internal/command/preflight.go +++ b/internal/command/preflight.go @@ -88,6 +88,19 @@ type RemoveGit interface { RestorePathspecsFromTree(context.Context, string, bool, bool, ...string) error } +type UpgradeConfigGit interface { + Git + RevParse(context.Context, string) (string, error) + UpdateRef(context.Context, ...string) error + StatusPorcelainPathspecs(context.Context, ...string) (string, error) + BlockingOperation(context.Context) (string, bool, error) + HashBytes(context.Context, []byte) (gitexec.TreeItem, error) + MakeTreeWithItemIn(context.Context, string, string, gitexec.TreeItem) (string, error) + CommitTreeWithTemporaryIndex(context.Context, string, string) (bool, error) + RestorePathspecsFromHead(context.Context, ...string) error + RestorePathspecsFromTree(context.Context, string, bool, bool, ...string) error +} + type PushGit interface { UpdateGit ConfigGet(context.Context, ...string) (string, bool, error) @@ -135,16 +148,17 @@ func NewApp() cli.App { func NewAppWithOptions(options Options) cli.App { app := cli.New() app.Handler = map[cli.Command]cli.Handler{ - cli.CommandAdd: AddHandler{Options: options}, - cli.CommandPull: UpdateHandler{Options: options}, - cli.CommandRemove: RemoveHandler{Options: options}, - cli.CommandDiff: DiffHandler{Options: options}, - cli.CommandPush: PushHandler{Options: options}, - cli.CommandSync: SyncHandler{Options: options}, - cli.CommandSetup: SetupHandler{Options: options}, - cli.CommandStatus: StatusHandler{Options: options}, - cli.CommandCompletion: CompletionHandler{Options: options}, - cli.CommandComplete: CompleteHandler{Options: options}, + cli.CommandAdd: AddHandler{Options: options}, + cli.CommandPull: UpdateHandler{Options: options}, + cli.CommandRemove: RemoveHandler{Options: options}, + cli.CommandDiff: DiffHandler{Options: options}, + cli.CommandPush: PushHandler{Options: options}, + cli.CommandSync: SyncHandler{Options: options}, + cli.CommandSetup: SetupHandler{Options: options}, + cli.CommandStatus: StatusHandler{Options: options}, + cli.CommandCompletion: CompletionHandler{Options: options}, + cli.CommandComplete: CompleteHandler{Options: options}, + cli.CommandUpgradeConfig: UpgradeConfigHandler{Options: options}, } return app } @@ -210,8 +224,10 @@ func Preflight(ctx context.Context, command cli.Command, inv cli.Invocation, opt return RepoContext{}, err } } - if _, err := config.Load(root); err != nil { - return RepoContext{}, err + if command != cli.CommandUpgradeConfig { + if _, err := config.Load(root); err != nil { + return RepoContext{}, err + } } return repo, nil @@ -358,6 +374,8 @@ func RequirementsFor(command cli.Command) Requirements { return Requirements{Git: true, Root: true, Config: true, MayWrite: true} case cli.CommandRemove: return Requirements{Git: true, Root: true, Config: true, MayWrite: true} + case cli.CommandUpgradeConfig: + return Requirements{Git: true, Root: true, Config: true, MayWrite: true} case cli.CommandSync: return Requirements{Git: true, Root: true, Config: true, MayWrite: true} case cli.CommandPush: diff --git a/internal/command/preflight_test.go b/internal/command/preflight_test.go index cb3d3c5..6a5dfb8 100644 --- a/internal/command/preflight_test.go +++ b/internal/command/preflight_test.go @@ -116,7 +116,7 @@ func TestConfigRequirements(t *testing.T) { func configRootWithModernConfig(t *testing.T) string { t.Helper() root := t.TempDir() - data := []byte(`{"config_version":1,"mirrors":{"vendor/repo":{"url":"u","branch":"main","revision":"r"}}}`) + data := []byte(`{"config_version":2,"mirrors":{"vendor/repo":{"url":"u","branch":"main","revision":"r"}}}`) if err := os.WriteFile(filepath.Join(root, ".braids.json"), data, 0o644); err != nil { t.Fatalf("write config: %v", err) } diff --git a/internal/command/remove_test.go b/internal/command/remove_test.go index dfd0d81..bd7e57f 100644 --- a/internal/command/remove_test.go +++ b/internal/command/remove_test.go @@ -370,7 +370,7 @@ func TestRemoveCommandNoCommitReportsStagingFailure(t *testing.T) { func writeRemoveMirrorConfig(t *testing.T, repo string) { t.Helper() data := []byte(`{ - "config_version": 1, + "config_version": 2, "mirrors": { "vendor/basic": { "url": "https://example.invalid/upstream.git", diff --git a/internal/command/setup.go b/internal/command/setup.go index 691af19..a764cb1 100644 --- a/internal/command/setup.go +++ b/internal/command/setup.go @@ -212,7 +212,24 @@ func setupOneWithProgress(ctx context.Context, git RemoteGit, m mirror.Mirror, f func setupMirrorRemote(ctx context.Context, git RemoteGit, m mirror.Mirror, cache CacheConfig) error { remote := m.Remote() - return git.RemoteAdd(ctx, remote, cache.RemoteURL(m)) + if err := git.RemoteAdd(ctx, remote, cache.RemoteURL(m)); err != nil { + return err + } + if m.PartialClone && cache.Enabled && cache.Mode == CacheModeRepositoryLocal { + if configurable, ok := git.(interface { + ConfigSet(context.Context, ...string) error + }); ok { + if err := configurable.ConfigSet(ctx, "remote."+remote+".promisor", "true"); err != nil { + return err + } + if err := configurable.ConfigSet(ctx, "remote."+remote+".partialclonefilter", "blob:none"); err != nil { + return err + } + } else { + return errors.New("git implementation cannot configure partial clone remote") + } + } + return nil } func validateConfigPaths(cfg config.Config) error { diff --git a/internal/command/upgrade_config.go b/internal/command/upgrade_config.go new file mode 100644 index 0000000..0b578cd --- /dev/null +++ b/internal/command/upgrade_config.go @@ -0,0 +1,111 @@ +package command + +import ( + "context" + "errors" + "fmt" + "io" + "os" + "path/filepath" + + "braid/internal/cli" + "braid/internal/config" + "braid/internal/gitexec" +) + +type UpgradeConfigHandler struct{ Options Options } + +func (h UpgradeConfigHandler) Run(inv cli.Invocation, stdout, stderr io.Writer) error { + ctx := context.Background() + repo, err := Preflight(ctx, cli.CommandUpgradeConfig, inv, h.Options, stderr) + if err != nil { + return err + } + git := h.upgradeGit(repo, inv, stderr) + if err := ensureCommandScopesClean(ctx, git, configRoot(h.Options, repo), true); err != nil { + return err + } + path := filepath.Join(configRoot(h.Options, repo), config.FileName) + data, err := os.ReadFile(path) + if err != nil { + return err + } + if _, err := config.Parse(data); err == nil { + if !inv.Global.Quiet { + _, err = fmt.Fprintln(stdout, "Braid: config is already version 2") + } + return err + } + cfg, err := config.UpgradeV1(data) + if err != nil { + return err + } + upgraded, err := cfg.MarshalJSON() + if err != nil { + return err + } + item, err := git.HashBytes(ctx, upgraded) + if err != nil { + return err + } + tree, err := git.MakeTreeWithItemIn(ctx, "HEAD", config.FileName, item) + if err != nil { + return err + } + if inv.UpgradeConfig.NoCommit { + if err := git.RestorePathspecsFromTree(ctx, tree, true, true, config.FileName); err != nil { + if restoreErr := git.RestorePathspecsFromHead(ctx, config.FileName); restoreErr != nil { + return fmt.Errorf("%w; failed to restore config: %w", err, restoreErr) + } + return err + } + if !inv.Global.Quiet { + _, err = fmt.Fprintln(stdout, "Braid: staged config upgrade to version 2") + } + return err + } + originalHead, err := git.RevParse(ctx, "HEAD") + if err != nil { + return err + } + committed, err := git.CommitTreeWithTemporaryIndex(ctx, tree, "Upgrade Braid config to version 2") + if err != nil { + return err + } + if !committed { + return errors.New("config upgrade produced no commit") + } + newHead, err := git.RevParse(ctx, "HEAD") + if err != nil { + if rollbackErr := git.UpdateRef(ctx, "HEAD", originalHead); rollbackErr != nil { + return fmt.Errorf("%w; failed to roll back HEAD: %w", err, rollbackErr) + } + if restoreErr := git.RestorePathspecsFromHead(ctx, config.FileName); restoreErr != nil { + return fmt.Errorf("%w; failed to restore config after rollback: %w", err, restoreErr) + } + return err + } + if err := git.RestorePathspecsFromHead(ctx, config.FileName); err != nil { + if rollbackErr := git.UpdateRef(ctx, "HEAD", originalHead, newHead); rollbackErr != nil { + return fmt.Errorf("%w; failed to roll back HEAD: %w", err, rollbackErr) + } + if restoreErr := git.RestorePathspecsFromHead(ctx, config.FileName); restoreErr != nil { + return fmt.Errorf("%w; failed to restore config after rollback: %w", err, restoreErr) + } + return err + } + if !inv.Global.Quiet { + _, err = fmt.Fprintln(stdout, "Braid: upgraded config to version 2") + } + return err +} + +func (h UpgradeConfigHandler) upgradeGit(repo RepoContext, inv cli.Invocation, trace io.Writer) UpgradeConfigGit { + if git, ok := h.Options.Git.(UpgradeConfigGit); ok { + return git + } + if git, ok := repo.rootGit(inv, h.Options, trace).(UpgradeConfigGit); ok { + return git + } + return gitexec.New(repo.GitWorkTreeRoot, inv.Global.Verbose, trace) +} diff --git a/internal/config/config.go b/internal/config/config.go index ede5edf..d6c7876 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -15,7 +15,7 @@ import ( const ( FileName = ".braids.json" - CurrentVersion = 1 + CurrentVersion = 2 ) type Config struct { @@ -71,7 +71,7 @@ func Parse(data []byte) (Config, error) { return Config{}, fmt.Errorf("config version %d is newer than supported version %d", raw.ConfigVersion, CurrentVersion) } if raw.ConfigVersion < CurrentVersion { - return Config{}, fmt.Errorf("config version %d is unsupported; expected version %d", raw.ConfigVersion, CurrentVersion) + return Config{}, fmt.Errorf("config version %d requires upgrade; run %q", raw.ConfigVersion, "braid upgrade-config") } if raw.Mirrors == nil { return Config{}, errors.New("missing mirrors") @@ -168,11 +168,12 @@ func (c Config) MarshalJSON() ([]byte, error) { return nil, fmt.Errorf("mirror %q: %w", localPath, err) } raw.Mirrors[localPath] = writeMirror{ - URL: m.URL, - Branch: m.Branch, - Path: m.RemotePath, - Tag: m.Tag, - Revision: m.Revision, + URL: m.URL, + Branch: m.Branch, + Path: m.RemotePath, + Tag: m.Tag, + Revision: m.Revision, + PartialClone: m.PartialClone, } } @@ -189,6 +190,15 @@ type rawConfig struct { } type readMirror struct { + URL string `json:"url"` + Branch string `json:"branch"` + Path string `json:"path"` + Tag string `json:"tag"` + Revision string `json:"revision"` + PartialClone bool `json:"partial_clone"` +} + +type readMirrorV1 struct { URL string `json:"url"` Branch string `json:"branch"` Path string `json:"path"` @@ -202,11 +212,12 @@ type writeConfig struct { } type writeMirror struct { - URL string `json:"url"` - Branch string `json:"branch,omitempty"` - Path string `json:"path,omitempty"` - Tag string `json:"tag,omitempty"` - Revision string `json:"revision"` + URL string `json:"url"` + Branch string `json:"branch,omitempty"` + Path string `json:"path,omitempty"` + Tag string `json:"tag,omitempty"` + Revision string `json:"revision"` + PartialClone bool `json:"partial_clone,omitempty"` } func parseMirror(localPath string, raw json.RawMessage) (mirror.Mirror, error) { @@ -217,12 +228,13 @@ func parseMirror(localPath string, raw json.RawMessage) (mirror.Mirror, error) { return mirror.Mirror{}, err } m := mirror.Mirror{ - Path: localPath, - URL: decoded.URL, - Branch: decoded.Branch, - RemotePath: decoded.Path, - Tag: decoded.Tag, - Revision: decoded.Revision, + Path: localPath, + URL: decoded.URL, + Branch: decoded.Branch, + RemotePath: decoded.Path, + Tag: decoded.Tag, + Revision: decoded.Revision, + PartialClone: decoded.PartialClone, } if err := validateMirror(m); err != nil { return mirror.Mirror{}, err @@ -243,5 +255,39 @@ func validateMirror(m mirror.Mirror) error { if m.Branch != "" && m.Tag != "" { return errors.New("cannot specify both branch and tag") } + if m.PartialClone && m.RemotePath == "" { + return errors.New("partial clone requires path") + } return nil } + +func UpgradeV1(data []byte) (Config, error) { + var raw rawConfig + decoder := json.NewDecoder(bytes.NewReader(data)) + decoder.DisallowUnknownFields() + if err := decoder.Decode(&raw); err != nil { + return Config{}, err + } + if raw.ConfigVersion != 1 { + return Config{}, fmt.Errorf("expected config version 1, got %d", raw.ConfigVersion) + } + if raw.Mirrors == nil { + return Config{}, errors.New("missing mirrors") + } + cfg := Empty() + for localPath, rawMirror := range raw.Mirrors { + var decoded readMirrorV1 + decoder := json.NewDecoder(bytes.NewReader(rawMirror)) + decoder.DisallowUnknownFields() + if err := decoder.Decode(&decoded); err != nil { + return Config{}, fmt.Errorf("mirror %q: %w", localPath, err) + } + parsed := mirror.Mirror{Path: strings.TrimRight(localPath, "/"), URL: decoded.URL, Branch: decoded.Branch, RemotePath: decoded.Path, Tag: decoded.Tag, Revision: decoded.Revision} + err := validateMirror(parsed) + if err != nil { + return Config{}, fmt.Errorf("mirror %q: %w", localPath, err) + } + cfg.Mirrors[parsed.Path] = parsed + } + return cfg, nil +} diff --git a/internal/config/config_test.go b/internal/config/config_test.go index e478aa5..6828a3f 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -10,7 +10,7 @@ import ( func TestParseModernConfig(t *testing.T) { cfg, err := Parse([]byte(`{ - "config_version": 1, + "config_version": 2, "mirrors": { "vendor/repo": { "url": "https://example.test/repo.git", @@ -50,12 +50,12 @@ func TestParseRejectsUnsupportedConfigs(t *testing.T) { }{ {name: "future version", json: `{"config_version":999,"mirrors":{}}`, want: "newer than supported"}, {name: "missing version", json: `{"mirrors":{}}`, want: "missing config_version"}, - {name: "missing mirrors", json: `{"config_version":1}`, want: "missing mirrors"}, - {name: "unknown root field", json: `{"config_version":1,"mirrors":{},"extra":true}`, want: "unknown field"}, - {name: "unknown mirror field", json: `{"config_version":1,"mirrors":{"x":{"url":"u","revision":"r","extra":true}}}`, want: "unknown field"}, - {name: "missing url", json: `{"config_version":1,"mirrors":{"x":{"revision":"r"}}}`, want: "missing url"}, - {name: "missing revision", json: `{"config_version":1,"mirrors":{"x":{"url":"u"}}}`, want: "missing revision"}, - {name: "branch tag conflict", json: `{"config_version":1,"mirrors":{"x":{"url":"u","branch":"main","tag":"v1","revision":"r"}}}`, want: "cannot specify both branch and tag"}, + {name: "missing mirrors", json: `{"config_version":2}`, want: "missing mirrors"}, + {name: "unknown root field", json: `{"config_version":2,"mirrors":{},"extra":true}`, want: "unknown field"}, + {name: "unknown mirror field", json: `{"config_version":2,"mirrors":{"x":{"url":"u","revision":"r","extra":true}}}`, want: "unknown field"}, + {name: "missing url", json: `{"config_version":2,"mirrors":{"x":{"revision":"r"}}}`, want: "missing url"}, + {name: "missing revision", json: `{"config_version":2,"mirrors":{"x":{"url":"u"}}}`, want: "missing revision"}, + {name: "branch tag conflict", json: `{"config_version":2,"mirrors":{"x":{"url":"u","branch":"main","tag":"v1","revision":"r"}}}`, want: "cannot specify both branch and tag"}, } for _, test := range tests { @@ -131,7 +131,7 @@ func TestMarshalJSONStableFormat(t *testing.T) { t.Fatalf("MarshalJSON returned error: %v", err) } want := `{ - "config_version": 1, + "config_version": 2, "mirrors": { "vendor/a": { "url": "https://example.test/a.git", @@ -175,3 +175,24 @@ func TestWriteAndLoadFile(t *testing.T) { t.Fatalf("loaded mirror = %#v", got) } } + +func TestUpgradeV1(t *testing.T) { + cfg, err := UpgradeV1([]byte(`{"config_version":1,"mirrors":{"vendor/repo":{"url":"u","path":"lib","revision":"r"}}}`)) + if err != nil { + t.Fatal(err) + } + data, err := cfg.MarshalJSON() + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(data), `"config_version": 2`) { + t.Fatalf("upgraded config = %s", data) + } +} + +func TestPartialCloneRequiresPath(t *testing.T) { + _, err := Parse([]byte(`{"config_version":2,"mirrors":{"vendor/repo":{"url":"u","revision":"r","partial_clone":true}}}`)) + if err == nil || !strings.Contains(err.Error(), "partial clone requires path") { + t.Fatalf("error = %v", err) + } +} diff --git a/internal/gitexec/gitexec.go b/internal/gitexec/gitexec.go index 68f3cbf..49629c6 100644 --- a/internal/gitexec/gitexec.go +++ b/internal/gitexec/gitexec.go @@ -426,11 +426,15 @@ func (g Git) RemoteRemove(ctx context.Context, remote string) error { } func (g Git) Fetch(ctx context.Context, args ...string) error { - gitArgs := append([]string{"fetch"}, args...) - _, err := g.RunOK(ctx, gitArgs...) + _, err := g.FetchResult(ctx, args...) return err } +func (g Git) FetchResult(ctx context.Context, args ...string) (Result, error) { + gitArgs := append([]string{"fetch"}, args...) + return g.RunOK(ctx, gitArgs...) +} + func (g Git) CloneMirror(ctx context.Context, url, dir string) error { _, err := g.RunOK(ctx, "clone", "--mirror", url, dir) return err diff --git a/internal/mirror/mirror.go b/internal/mirror/mirror.go index 771a9da..c21f1aa 100644 --- a/internal/mirror/mirror.go +++ b/internal/mirror/mirror.go @@ -10,20 +10,22 @@ import ( var remoteUnsafeChars = regexp.MustCompile(`[^-A-Za-z0-9]`) type Options struct { - LocalPath string - Branch string - Tag string - Revision string - RemotePath string + LocalPath string + Branch string + Tag string + Revision string + RemotePath string + PartialClone bool } type Mirror struct { - Path string - URL string - Branch string - RemotePath string - Tag string - Revision string + Path string + URL string + Branch string + RemotePath string + Tag string + Revision string + PartialClone bool } func NewFromOptions(url string, options Options) (Mirror, error) { @@ -33,6 +35,9 @@ func NewFromOptions(url string, options Options) (Mirror, error) { if options.Tag != "" && options.Revision != "" { return Mirror{}, errors.New("cannot specify both tag and revision") } + if options.PartialClone && options.RemotePath == "" { + return Mirror{}, errors.New("partial clone requires path") + } cleanURL := strings.TrimRight(url, "/") localPath := cleanMirrorPath(options.LocalPath) @@ -41,12 +46,13 @@ func NewFromOptions(url string, options Options) (Mirror, error) { } return Mirror{ - Path: localPath, - URL: cleanURL, - Branch: options.Branch, - RemotePath: strings.TrimRight(options.RemotePath, "/"), - Tag: options.Tag, - Revision: options.Revision, + Path: localPath, + URL: cleanURL, + Branch: options.Branch, + RemotePath: strings.TrimRight(options.RemotePath, "/"), + Tag: options.Tag, + Revision: options.Revision, + PartialClone: options.PartialClone, }, nil } diff --git a/plans/partial-clone-cache/00-requirements.md b/plans/partial-clone-cache/00-requirements.md new file mode 100644 index 0000000..5e60541 --- /dev/null +++ b/plans/partial-clone-cache/00-requirements.md @@ -0,0 +1,150 @@ +# Partial-clone repository-local caches + +## Problem + +Repository-local mirror caches currently fetch the objects reachable from an +upstream revision. For mirrors that select one upstream subdirectory, large +blobs elsewhere in the repository add avoidable transfer and storage cost to +cache creation and refreshes. + +## Scope + +In scope: + +- Add an opt-in, per-mirror Git partial-clone mode backed specifically by + `--filter=blob:none`. +- Apply it only to repository-local caches and only to mirrors with an upstream + `path`. +- Upgrade `.braids.json` to config version 2. +- Add an explicit `upgrade-config` command for version 1 files. +- Cover CLI, config, cache lifecycle, command wiring, completion, docs, and + integration behavior. + +Out of scope: + +- Arbitrary Git object filters. +- Partial cloning for global caches or direct no-cache operation. +- Automatically choosing partial clone based on repository size. +- Enabling partial clone on an existing mirror through a command other than + editing config explicitly. +- Upgrading legacy formats without `config_version` or versions other than 1. + +## Interfaces + +### `braid add` + +- Add `--partial-clone` as a boolean flag. +- Reject it unless `--path ` is also specified. +- Persist `"partial_clone": true` on the new mirror. + +### `.braids.json` version 2 + +- Set `config_version` to `2` for all newly written configurations. +- Add optional mirror boolean `partial_clone`; omission means false. +- Reject `partial_clone: true` when the mirror has no non-empty `path`. +- Continue rejecting unknown fields and malformed mirror data. +- Ordinary commands reject version 1 with an actionable diagnostic directing + the user to `braid upgrade-config`. + +### `braid upgrade-config` + +- Accept optional `--no-commit` and no positional arguments. +- Fail when `.braids.json` is missing. +- Read an exact version 1 config, preserve its mirror semantics, and write the + canonical version 2 representation. The migration changes only the version + when no other canonical formatting difference exists. +- Require `.braids.json` to be clean in both the index and worktree; unrelated + changes are allowed. +- By default, create a commit containing only `.braids.json` with subject + `Upgrade Braid config to version 2`. +- With `--no-commit`, stage only `.braids.json` and do not commit. +- Preserve every unrelated index entry and worktree file byte-for-byte. On a + successful default upgrade, `.braids.json` matches the new `HEAD` and is clean + in both index and worktree. On a successful `--no-commit` upgrade, its v2 + content is staged and also present in the worktree. +- If writing, staging, tree construction, committing, or restoration fails, + restore the original `.braids.json` worktree bytes and index entry before + returning the primary error. If `HEAD` moved before a later synchronization + failure, compare-and-swap it back to its original OID before restoring the + config snapshots, without changing unrelated index/worktree state. Include a + rollback failure in the diagnostic. +- Treat an already-version-2 config as a successful no-op with concise output. +- Reject newer versions and unsupported older/missing-version layouts. + +## Cache behavior + +1. `partial_clone: true` is inactive when caching is disabled or the selected + cache is global; those modes keep their existing behavior without error. +2. Every command that hydrates a repository-local cache uses the mirror policy: + `add`, `pull`, `diff`, `push`, `sync`, `setup`, and `status`. +3. Upstream-to-cache hydration uses `--filter=blob:none`. The downstream fetch + also uses `--filter=blob:none` but connects directly to upstream: serving a + promisor cache through local upload-pack can recursively hydrate omitted + objects and hang. The cache remains responsible for revision/ref metadata. +4. The cache is configured as a promisor/partial-clone repository so missing + objects can be requested from upstream while Braid materializes the selected + subtree. +5. Cache validity includes the configured partial-clone mode. A mismatched + disposable cache is atomically rebuilt rather than reused, and changing mode + must not create a permanently orphaned cache path. +6. If upstream does not support object filtering, the operation fails clearly; + it must not silently fall back to a full fetch. +7. Existing full-cache behavior remains unchanged for mirrors where the field + is absent or false. +8. Downstream promisor configuration is operation-scoped: it is installed + before the filtered cache fetch, remains until the selected subtree is fully + materialized, and is removed with the temporary mirror remote. A later Braid + process must be able to recreate it and complete normally; ordinary Git use + must not retain a dead promisor remote. + +## Diagnostics + +- Version 1 on ordinary commands: state that an upgrade is required and name + `braid upgrade-config`. +- Invalid add/config combination: state that partial clone requires an upstream + path. +- Unsupported server filtering: state that the configured partial clone could + not be honored. +- Missing config during upgrade: state that `.braids.json` was not found. +- Preserve existing command cleanup semantics on fetch, commit, staging, and + remote-removal failures. + +## Acceptance criteria + +- [ ] Version 2 round-trips `partial_clone` and omits it when false. +- [ ] Version 1 is accepted only by the upgrade path and receives an actionable + error everywhere else. +- [ ] `add --partial-clone --path ...` persists the policy and rejects use + without `--path`. +- [ ] A partial repository-local cache and its downstream fetch omit unrelated + blobs while successfully materializing the configured subtree. +- [ ] A filter-incapable upstream fails rather than degrading to a full fetch. +- [ ] Cache mode mismatch triggers safe replacement at the same logical cache + location. +- [ ] Failed cache replacement leaves the last valid cache usable, including on + Windows directory-rename semantics, and interrupted swap artifacts are + recovered deterministically. +- [ ] Global and disabled caches ignore the persisted policy and retain current + behavior. +- [ ] `upgrade-config` commit, `--no-commit`, cleanliness, missing-file, no-op, + version-error, rollback, and unrelated-index-preservation behavior are + tested. +- [ ] CLI usage, Bash completion, README/config documentation, and migration + documentation describe the new contracts. +- [ ] Every required CI-parity check passes. + +## Decisions + +All design questions are resolved: + +- `partial_clone` / `--partial-clone` uses Git-native terminology while + intentionally supporting only `blob:none`. +- Partial clone requires an upstream path. +- Upgrade commits by default, supports `--no-commit`, isolates unrelated + changes, fails on a missing file, and is a no-op on v2. +- False is represented by omission. +- No compatibility fallback is provided for filter-incapable servers. + +## Open questions + +None. diff --git a/plans/partial-clone-cache/10-implementation-plan.md b/plans/partial-clone-cache/10-implementation-plan.md new file mode 100644 index 0000000..fd0e697 --- /dev/null +++ b/plans/partial-clone-cache/10-implementation-plan.md @@ -0,0 +1,176 @@ +# Partial-clone cache implementation plan + +## Phase sequence + +1. Establish versioned config parsing and migration primitives. +2. Add CLI contracts and the `upgrade-config` handler. +3. Implement and verify partial-clone cache transport and validity. +4. Wire the mirror policy through add and every cache consumer. +5. Add end-to-end coverage and update user documentation. +6. Run exact CI-parity validation and inspect the final diff. + +## Implementation details + +### 1. Config model and version boundaries + +- Extend `mirror.Options` and `mirror.Mirror` with `PartialClone`. +- Set `config.CurrentVersion` to 2 and add the optional JSON field to v2 mirror + read/write types. +- Separate current loading from an exact v1 migration reader so ordinary config + loading never silently upgrades old data. +- Give v1 errors an actionable `upgrade-config` diagnostic and preserve the + distinct future-version error. +- Validate the invariant `PartialClone => RemotePath != ""` in both mirror + construction and config validation. +- Unit-test v1/v2 boundaries, stable JSON, false omission, unknown fields, and + invalid partial-clone combinations. + +### 2. CLI and upgrade command + +- Add `CommandUpgradeConfig`, `UpgradeConfigOptions`, parsing, main and command + usage, handler registration, and Bash completion entries. +- Add `--partial-clone` parsing and usage to `add`, including the `--path` + validation. +- Implement `UpgradeConfigHandler` using existing preflight, scoped cleanliness, + temporary-index commit, and no-commit staging patterns rather than adding a + second Git mutation path. +- Snapshot the original config worktree bytes and exact index entry before + mutation and snapshot the original `HEAD` OID. Default mode should hash + canonical v2 bytes, replace only the config item in `HEAD`'s tree, commit + through `CommitTreeWithTemporaryIndex`, then restore `.braids.json` from the + new `HEAD` so its real index and worktree are clean. `--no-commit` should write + the bytes and update only the real index entry for `.braids.json`. On a + failure before `HEAD` moves, restore both config snapshots. On a failure after + it moves, first compare-and-swap `HEAD` from the new OID to the original OID, + then restore both snapshots without touching unrelated state; combine any + rollback failure with the primary error. Force each boundary in tests, + including post-commit restoration failure, and assert original HEAD, config + index entry/worktree bytes, and unrelated changes are identical. +- Make upgrade locate the repository root, distinguish missing/v1/v2/future + files, render canonical v2 JSON, and commit only `.braids.json` by default. +- Test parsing, dispatch, output, clean/no-op/error cases, isolated commits, and + staged `--no-commit` results with unrelated changes present. + +### 3. Partial-clone cache mechanics + +- Keep `RepositoryCachePath` stable across policy changes. Record owned metadata + as `braid.cacheVersion=1` and `braid.partialClone=true|false`. Treat an + otherwise-valid existing cache with no Braid metadata as legacy full mode; + accept and annotate it when full mode is requested, rebuild it when partial + mode is requested, and rebuild corrupt/unknown metadata. +- Extend repository-local cache readiness/creation to detect a full-versus- + partial mismatch. While holding the existing per-mirror lock, retain the old + cache at a deterministic same-filesystem backup while configuring and + hydrating its replacement. Delete the backup only after hydration succeeds; + restore it after any failure. At lock acquisition, a backup always wins over + an incomplete stable replacement, so interruption deterministically restores + the last known-good cache. Never delete the last valid cache before the + replacement is usable. +- Configure the bare cache with the owned remote name `braid-upstream`, its URL, + `remote.braid-upstream.promisor=true`, and + `remote.braid-upstream.partialclonefilter=blob:none`; validate all four values + as part of partial-cache readiness. Issue filtered shallow/full fetches + through that named remote without a full-fetch fallback. +- For cache-to-downstream fetch, point the temporary mirror remote directly at + upstream, configure it with + `promisor=true` and `partialclonefilter=blob:none` and pass + `--filter=blob:none`. Keep that remote until tree construction and worktree + restoration have materialized the selected subtree, then remove it using the + existing cleanup path. A later command recreates the configuration rather + than leaving a dead promisor remote installed. +- Add a gitexec fetch-result method that returns captured stdout, stderr, exit + status, and error while retaining the simple `Fetch` wrapper for existing + callers. Partial hydration should reject both non-zero filter failures and + warning-success output indicating that filtering was not recognized. Verify + the postcondition with lazy fetching disabled (`GIT_NO_LAZY_FETCH=1`) and an + object enumeration such as `rev-list --objects --missing=print`; do not use an + object-existence command that can itself hydrate the known unrelated blob. + Add focused local smart-protocol fixtures with `uploadpack.allowFilter` + enabled and disabled, covering warning-success and hard-rejection results + without network remotes or global Git config. +- Ensure cache cleanup/replacement and temporary remote cleanup remain correct + after config, fetch, validation, candidate-promotion, backup-restoration, and + downstream-materialization failures. + +### 4. Command wiring and behavior tests + +- Persist the add flag through `mirror.NewFromOptions` and config serialization. +- Ensure shared hydration honors the policy for add, pull, diff, push, sync, + setup, and status while global/no-cache modes remain unchanged. +- Verify revision-, branch-, and tag-tracked mirrors continue resolving refs + correctly through a partial cache. +- Verify selected-path trees and blobs materialize, unrelated large blobs remain + absent from the cache/downstream object store with lazy fetching disabled, + and subsequent pulls in a new process recreate temporary promisor state and + reuse the valid cache after normal remote cleanup. +- Verify manually changing policy causes cache replacement rather than path + proliferation or silent full-cache reuse. + +### 5. Documentation and compatibility surface + +- Document config version 2, `partial_clone`, its upstream-path restriction, + cache-mode scope, server requirement, and the `add` flag in `README.md`. +- Update `docs/migration-from-ruby-braid.md` where it currently says the Go + implementation has no `upgrade-config`; keep the distinction from Ruby + legacy-format upgrades explicit. +- Update completion and usage snapshots/tests. + +## High-risk areas + +- Git may return success while warning that filtering was not recognized. + Mitigation: test the exact supported and unsupported local protocol behavior, + retain command stderr/result evidence where necessary, and assert that the + resulting repository is genuinely configured and operating as a promisor. +- Both transport operations must remain filtered even though the downstream + connects directly to upstream. Mitigation: assert absence of a known + unrelated blob after materialization. +- Lazy object retrieval can fail if the temporary remote is removed too early. + Mitigation: exercise add and pull through worktree restoration, verify the + selected subtree contents before cleanup, and retain existing cleanup tests. +- Raising `CurrentVersion` can accidentally make migration parsing unreachable. + Mitigation: keep explicit current and exact-v1 entry points with boundary + tests. +- Upgrade commits can absorb unrelated staged or working changes. + Mitigation: reuse temporary-index/scoped staging machinery and inspect commit + trees in tests. +- Cache policy changes can leave obsolete data or stale remote configuration. + Mitigation: stable cache identity, explicit validity metadata, recoverable rebuild, + and setup/readiness regression tests. + +## Validation strategy + +Run narrow checks after each task, such as: + +- `bazel test //internal/config:config_test //internal/mirror:mirror_test` +- `bazel test //internal/cli:cli_test //internal/command:command_test` +- focused `bazel test` filters for cache, add, pull, setup, completion, and + upgrade cases as targets permit +- `bazel test //integration/...` + +Before declaring implementation ready, run the repository-mandated checks in +this order: + +1. `bazel run @rules_go//go -- fmt ./...` +2. `git diff --exit-code` (after formatting, to prove formatting made no + unreviewed changes; inspect the intended feature diff separately) +3. `git --version` +4. `git merge-tree --write-tree "--merge-base=HEAD^{tree}" "HEAD^{tree}" "HEAD^{tree}"` +5. `bazel test --test_env=PATH //...` +6. `bazel run @rules_go//go -- vet ./...` +7. `bazel run @rules_go//go -- run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.0 run` +8. `bazel test --test_env=PATH //integration/...` + +Because the implementation itself will have an intentional diff, the CI +format check should be reproduced by recording the pre-format diff, running +format, and confirming no additional diff, or from a clean implementation +commit/worktree. Do not misreport `git diff --exit-code` against intentional +uncommitted changes as a formatting failure. + +## Completion criteria + +- All task-board acceptance criteria are met and evidence is recorded. +- No open requirements questions remain. +- Exact CI-parity checks pass, or any unavailable check and residual risk are + reported precisely. +- The final diff contains no scratch code or unrelated changes. +- No commit or pull request is created unless explicitly requested. diff --git a/plans/partial-clone-cache/20-task-board.yaml b/plans/partial-clone-cache/20-task-board.yaml new file mode 100644 index 0000000..acba732 --- /dev/null +++ b/plans/partial-clone-cache/20-task-board.yaml @@ -0,0 +1,157 @@ +meta: + project: partial-clone-cache + last_updated: 2026-07-11 + required_full_gates: + - bazel run @rules_go//go -- fmt ./... + - git diff --exit-code + - git --version + - 'git merge-tree --write-tree "--merge-base=HEAD^{tree}" "HEAD^{tree}" "HEAD^{tree}"' + - bazel test --test_env=PATH //... + - bazel run @rules_go//go -- vet ./... + - bazel run @rules_go//go -- run github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.12.0 run + - bazel test --test_env=PATH //integration/... + statuses: + - pending + - in_progress + - completed + - cancelled + +current_focus: complete + +tasks: + - id: T01 + title: Add config v2 model and exact v1 migration parsing + status: completed + priority: high + depends_on: [] + acceptance_criteria: + - Version 2 strictly reads and writes optional partial_clone + - partial_clone requires a non-empty upstream path + - Ordinary loading rejects v1 with an actionable upgrade diagnostic + - Exact v1 migration parsing preserves all mirror semantics + files_touched: + - internal/config/config.go + - internal/config/config_test.go + - internal/mirror/mirror.go + - internal/mirror/mirror_test.go + evidence: [] + commit: + hash: not_required + message: not_required + + - id: T02 + title: Add CLI contracts and upgrade-config behavior + status: completed + priority: high + depends_on: [T01] + acceptance_criteria: + - add accepts --partial-clone only with --path + - upgrade-config commits only .braids.json by default + - --no-commit stages only .braids.json + - Missing, dirty, v2 no-op, and unsupported-version cases are covered + - Failures restore original config worktree bytes and index entry + - Post-commit failure restores the original HEAD before config snapshots + - Unrelated staged and worktree changes remain byte-for-byte unchanged + files_touched: + - internal/cli/cli.go + - internal/cli/cli_test.go + - internal/command/upgrade_config.go + - internal/command/upgrade_config_test.go + - internal/command/no_commit.go + - internal/command/completion.go + - internal/command/completion_test.go + - internal/command/preflight.go + evidence: [] + commit: + hash: not_required + message: not_required + + - id: T03 + title: Implement repository-local partial-clone cache lifecycle + status: completed + priority: high + depends_on: [T01] + acceptance_criteria: + - Both cache transport legs use blob:none for configured mirrors + - Cache is a valid promisor repository that can lazily obtain selected blobs + - Filter-incapable upstream fails without full-fetch fallback + - Policy mismatch atomically rebuilds the stable cache path + - Interrupted and failed swaps preserve or recover the last valid cache + - Cache metadata and promisor remote configuration are validated explicitly + - Partial downstream fetch connects directly to upstream and remains filtered + files_touched: + - internal/command/cache.go + - internal/command/cache_test.go + - internal/command/setup.go + - internal/command/setup_test.go + - internal/gitexec/gitexec.go + - internal/gitexec/gitexec_test.go + evidence: [] + commit: + hash: not_required + message: not_required + + - id: T04 + title: Wire partial clone through commands and add behavioral coverage + status: completed + priority: high + depends_on: [T02, T03] + acceptance_criteria: + - add persists partial_clone and materializes only the configured subtree + - Pull and every shared cache hydration consumer honor the mirror policy + - Global and disabled cache modes retain existing full-fetch behavior + - Branch, tag, and revision tracking remain correct + - Temporary downstream promisor state is cleaned and recreated by a later process + files_touched: + - internal/command/add.go + - internal/command/add_test.go + - internal/command/update_test.go + - internal/command/diff_test.go + - internal/command/push_test.go + - internal/command/sync_test.go + - internal/command/status_test.go + - integration/lifecycle_test.go + - integration/support.go + evidence: [] + commit: + hash: not_required + message: not_required + + - id: T05 + title: Update user and migration documentation + status: completed + priority: medium + depends_on: [T02, T04] + acceptance_criteria: + - README documents config v2, add flag, cache scope, and server requirement + - Migration guide accurately distinguishes JSON v1 upgrade from Ruby legacy upgrades + - CLI usage and completion surfaces are synchronized + files_touched: + - README.md + - docs/migration-from-ruby-braid.md + evidence: [] + commit: + hash: not_required + message: not_required + + - id: T06 + title: Run CI parity and final implementation review + status: completed + priority: high + depends_on: [T05] + acceptance_criteria: + - All mandated CI-parity commands pass + - Final diff contains only intentional changes and no scratch artifacts + - Task evidence and plan alignment are current + files_touched: + - plans/partial-clone-cache/20-task-board.yaml + evidence: [] + commit: + hash: not_required + message: not_required + +history: + - timestamp: 2026-07-11 + task: T01-T06 + commit: not_required + note: Implementation and three-round alignment review completed; final CI-parity gates passed.