Skip to content

feat: record module execution history incrementally#72

Draft
jahkeup wants to merge 1 commit into
masterfrom
feat/incremental-history-recording
Draft

feat: record module execution history incrementally#72
jahkeup wants to merge 1 commit into
masterfrom
feat/incremental-history-recording

Conversation

@jahkeup
Copy link
Copy Markdown
Member

@jahkeup jahkeup commented May 22, 2026

Issue #, if available:

n/a

Description of changes:

Module execution history is currently written to disk only after all modules finish. If the process crashes, is killed, or the instance is interrupted mid-run, no history is persisted for modules that already completed successfully. On the next boot those modules re-execute — potentially re-running expensive or non-idempotent operations like userdata scripts.

This change introduces HistoryRecorder, a concurrency-safe writer that flushes history to disk after each module completes. Each Record call appends the module result and atomically rewrites the history file (temp file + fsync + rename), so the on-disk state always reflects all modules that have finished so far. The old batched WriteHistoryFile() call at the end of run() is removed (the method is retained on InitConfig for backward compatibility).

Files changed:

  • lib/ec2macosinit/instancehistory.go — new HistoryRecorder type with NewHistoryRecorder and Record
  • run.go — instantiate recorder before the priority loop; call Record in each module goroutine; remove end-of-run WriteHistoryFile call
  • lib/ec2macosinit/instancehistory_test.go — tests for single record, multiple records, concurrent writes, failed modules, and incremental persistence

How to manually verify:

  1. Build and deploy to an EC2 Mac instance:

    GOOS=darwin GOARCH=arm64 go build -o ec2-macos-init .
    scp ec2-macos-init ec2-user@<instance>:/tmp/
    ssh ec2-user@<instance> 'sudo cp /tmp/ec2-macos-init /usr/local/bin/ec2-macos-init'
    
  2. Configure a slow module in /usr/local/aws/ec2-macos-init/init.toml (two priority groups):

    [[Module]]
      Name = "fast-module"
      PriorityGroup = 1
      RunPerBoot = true
      [Module.Command]
        Cmd = ["/bin/echo", "hello"]
        RunAsUser = "ec2-user"
    
    [[Module]]
      Name = "slow-module"
      PriorityGroup = 2
      RunPerBoot = true
      [Module.Command]
        Cmd = ["/bin/sleep", "30"]
        RunAsUser = "ec2-user"
  3. Clean history and run:

    sudo ec2-macos-init clean
    sudo ec2-macos-init run
    
  4. While slow-module is still running, check history in another terminal:

    cat /usr/local/aws/ec2-macos-init/instances/$(ec2metadata --instance-id)/history.json | python3 -m json.tool
    

    fast-module should already appear with "success": true before the run completes.

  5. Simulate a crash — kill the process while the second group runs:

    sudo kill -9 $(pgrep ec2-macos-init)
    

    Confirm history.json has results from priority group 1.

  6. Re-run after crash. RunPerInstance/RunOnce modules that succeeded should be skipped (look for "Skipping module ... due to Run type setting" in logs).

  7. Let a full run complete normally and verify history.json contains all module results — no regression from prior behavior.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Previously, module results were only written to the history file after
all modules completed. If the process crashed or was killed mid-run,
no history was persisted and modules would re-execute on the next boot.

Introduce HistoryRecorder which flushes to disk after each module
completes. This uses a mutex for goroutine safety within priority
groups and atomic file writes (via temp + rename) for crash safety.

Via: git.commit.create (mcp-server-go-git/v0.1.0-8-gb11cc3f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant