Skip to content

fix(brain-sync): make artifact sync work on Windows (discover-new + drain)#1672

Open
daveowenatl wants to merge 1 commit into
garrytan:mainfrom
daveowenatl:fix/windows-brain-sync-paths
Open

fix(brain-sync): make artifact sync work on Windows (discover-new + drain)#1672
daveowenatl wants to merge 1 commit into
garrytan:mainfrom
daveowenatl:fix/windows-brain-sync-paths

Conversation

@daveowenatl
Copy link
Copy Markdown

@daveowenatl daveowenatl commented May 23, 2026

Automatic artifact sync (gstack-brain-sync) is fully non-functional on Windows (Git Bash): --discover-new enqueues nothing and the --once drain stages nothing, so artifacts_sync_mode=full looks active but no artifacts ever reach the repo. No error surfaces — status reports idle.

Environment

Windows 11 (26200), Git Bash (Git for Windows 2.54.0), Bun 1.3.11, Python 3.14.3 (python3 on PATH), gstack v1.43.3.0.

Root causes (three, all in bin/gstack-brain-sync)

  1. discover-new path matchos.path.relpath returns backslash paths on Windows, which never match the forward-slash allowlist globs (projects/*/learnings.jsonl). Fixed by normalizing os.sep/.
    python3 -c "import os,fnmatch;print(fnmatch.fnmatchcase(os.path.relpath(r'C:\a\projects\p\learnings.jsonl', r'C:\a'),'projects/*/learnings.jsonl'))"  # -> False
    
  2. discover-new enqueue — enqueued via subprocess.run([gstack-brain-enqueue, rel]), but Windows Python can't exec a bash-shebang script, so --discover-new left the queue empty even when files matched. Now appends to the queue in-process. Same "spawnSync can't exec bash on Windows" family as fix: sourceLocalPath handles wrapped {sources:[...]} shape from gbrain v0.20+ #1571 / fix(sync-gbrain): Windows compat — capability check + brain-sync skip #1641, different code path.
  3. --once draincompute_paths_to_stage ends in print(p); Windows Python emits CRLF, the bash while IFS= read -r p keeps the trailing CR, and git add -f -- "path<CR>" fails with pathspec '…?' did not match under 2>/dev/null || true. Nothing is staged, the commit is empty, status reports idle. Now strips the trailing CR before git add.

The in-process enqueue mirrors gstack-brain-enqueue's contract rather than re-deriving it loosely: one atomic O_APPEND write per record (each line < PIPE_BUF) so a parallel writer-shim append can't interleave mid-record, and the discover cursor advances only after the write succeeds (a failed write retries instead of silently recording the file as synced). Skip-list entries are separator-normalized on both the discover and drain (compute_paths_to_stage) sides, so a backslash .brain-skip.txt entry can't be honored at discovery yet bypassed at commit — the drain is the boundary that actually stages files.

Repro

gstack-config set artifacts_sync_mode full   # ~/.gstack wired to a remote
mkdir -p ~/.gstack/projects/demo
echo '{}' > ~/.gstack/projects/demo/learnings.jsonl
gstack-brain-sync --discover-new
wc -l < ~/.gstack/.brain-queue.jsonl          # 0  (expected 1)

Testing

  • test/brain-sync-windows-paths.test.ts — static invariants guarding all three causes plus the two robustness properties (atomic per-record append, normalized skip matching). Behavioral spawn tests can't run on the Windows lane (Node/Bun can't exec the bin/ shebang scripts — the same reason brain-sync.test.ts is excluded from it), so this follows the static-invariant pattern of build-script-shell-compat.test.ts. Verified red→green. Wired into windows-free-tests.yml.
  • Manual end-to-end on Windows 11 / Git Bash: discover → drain → commit → push to a local bare remote lands sync: N file(s) with nested and single-level artifacts; non-allowlisted files excluded; a backslash .brain-skip.txt entry correctly keeps its file off the remote; queued records are valid compact JSONL; re-discover is idempotent.
  • macOS/Linux behavior unchanged: os.sep is already / so both separator-normalizations are no-ops, there's no CRLF to strip, and the inline O_APPEND append is equivalent to the previous per-file enqueue.

Related (not in this PR)

bin/gstack-brain-restore:117 has the same relpath+fnmatch bug in its clobber check — happy to follow up separately.

🤖 Generated with Claude Code

@daveowenatl daveowenatl marked this pull request as draft May 23, 2026 21:23
@daveowenatl daveowenatl force-pushed the fix/windows-brain-sync-paths branch from ab13b72 to dee462b Compare May 23, 2026 21:46
…rain)

Automatic artifact sync was fully non-functional on Windows (Git Bash):
--discover-new enqueued nothing and the --once drain staged nothing, so
artifacts_sync_mode looked active but no artifacts ever reached the repo.
Three independent Windows-only causes in bin/gstack-brain-sync:

1. discover-new matched os.path.relpath (backslash separators on Windows)
   against the forward-slash allowlist globs, so no nested file ever matched.
   Normalized the relpath to "/".
2. discover-new enqueued via subprocess.run([gstack-brain-enqueue, rel]), but
   Windows Python cannot exec a bash-shebang script, so nothing was enqueued
   even once matched. Now appends to the queue in-process.
3. compute_paths_to_stage ends in print(p); Windows Python emits CRLF, the
   bash `read -r` keeps the trailing CR, and `git add -- "path<CR>"` matches
   nothing under `2>/dev/null || true`. Now strips the CR before staging.

The in-process enqueue mirrors gstack-brain-enqueue's contract: one atomic
O_APPEND write per record (each line < PIPE_BUF) so a parallel writer-shim
append can't interleave mid-record, and the discover cursor advances only
after the write succeeds, so a failed write retries instead of silently
recording the file as synced. Skip-list entries are separator-normalized on
both the discover and drain (compute_paths_to_stage) sides, so a backslash
.brain-skip.txt entry can't be honored at discovery yet bypassed at commit.

Adds test/brain-sync-windows-paths.test.ts (static invariants -- behavioral
spawn tests cannot run on the Windows lane, since Node/Bun cannot exec the
bin/ shebang scripts there) and wires it into windows-free-tests.yml.
Verified red->green and end-to-end on Windows 11 / Git Bash; macOS/Linux
behavior unchanged (os.sep is already "/", no CRLF, compute path logic
unchanged besides the shared skip normalization).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@daveowenatl daveowenatl force-pushed the fix/windows-brain-sync-paths branch from dee462b to f4191bb Compare May 23, 2026 21:58
@daveowenatl daveowenatl marked this pull request as ready for review May 24, 2026 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant