Skip to content

Add audit-logs download command#195

Open
yummybomb wants to merge 19 commits into
mainfrom
hypeship/audit-logs-download
Open

Add audit-logs download command#195
yummybomb wants to merge 19 commits into
mainfrom
hypeship/audit-logs-download

Conversation

@yummybomb

@yummybomb yummybomb commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds kernel audit-logs download for verified chunked audit-log exports.

  • Requires an explicit, exclusive [start, end) range of at most 30 days.
  • Downloads gzip-compressed JSONL with no alternate format modes.
  • Walks the API cursor sequentially and verifies every source chunk checksum before processing it.
  • Uses the SDK's multi-value method exclusions to send the default GET exclusion and --exclude-method together to the API.
  • Writes to an adjacent .partial file and publishes the final .jsonl.gz only after all chunks complete.
  • Removes incomplete partial output after transfer failures and preserves a completed partial if final publication fails.
  • Refuses to replace an existing output unless --force is passed; forced replacement happens only during the final rename.

Test plan

  • go test ./...
  • go vet ./...
  • Race-enabled audit-log tests
  • Windows amd64 test-binary cross-compile
  • Stacked method exclusions match search and download semantics
  • Finalization failures preserve completed partial and existing output files
  • Live download smoke test with stacked GET and POST exclusions

Note

Medium Risk
Touches audit-log export and local file handling (permissions, partial/final paths) but is isolated CLI logic with strong validation and test coverage; no server auth changes.

Overview
Adds kernel audit-logs download for bulk export of gzip-compressed JSONL over the chunked export API.

The command requires an explicit [start, end) window (max 30 days), reuses audit-logs search filter flags (including default GET exclusion stacked with --exclude-method), and walks X-Next-Cursor until X-Has-More is false. Each chunk is verified against X-Content-Sha256 before any bytes are written.

Output is streamed to a .partial file with restrictive permissions; the final .jsonl.gz is published only after a successful rename. Failed transfers drop the partial file; a completed partial is kept if finalization fails. Existing output is blocked unless --force.

AuditLogsService gains ExportChunk; tests cover multi-chunk assembly, checksum/header validation, overwrite behavior, and param building.

Reviewed by Cursor Bugbot for commit 17fc709. Bugbot is set up for automated code reviews on this repo. Configure here.

yummybomb and others added 10 commits July 9, 2026 14:26
Wraps GET /audit-logs/export/chunk (kernel-go-sdk v0.75.0). Chunks are
verified against X-Content-Sha256 before being appended to the output
file, and a sidecar state file persists the cursor and committed byte
offset so an interrupted download resumes without duplicating rows.
Ranges over the API's 30-day cap fail fast with ready-to-run per-window
commands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--include-get lifts the exclusion. Because chunks are appended verbatim,
a second exclusion can't stack client-side the way search filters, so
--exclude-method with the GET default active is rejected with guidance
rather than silently replacing it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Bind resume state to a credential identity (hashed API key or OAuth
  org ID) so switching credentials can't mix organizations in one archive
- Validate the output file against recorded progress on resume; a missing
  or shortened file now fails instead of being zero-extended and reported
  complete
- Fail closed on missing X-Has-More and X-Content-Sha256 headers, and
  validate pagination headers before any bytes are written
- Commit initial state before the first fetch so an early failure retries
  without --force; --force removes stale state immediately
- Write state via os.CreateTemp and use 0600/0700 modes for outputs
- Cap buffered chunk responses at 256 MiB
- Fix stale -o references in errors; document that resume needs explicit
  --start/--end

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@yummybomb yummybomb marked this pull request as ready for review July 10, 2026 15:12
yummybomb and others added 2 commits July 10, 2026 15:14
The server controls chunk sizing and every chunk is checksum-verified,
so the client-side byte ceiling only added a failure mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The params fingerprint already invalidates stale state files, and the
state is too short-lived to need format versioning.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Comment thread cmd/audit_logs_download.go
Comment thread cmd/audit_logs_download.go
Comment thread cmd/audit_logs_download.go Outdated
@yummybomb yummybomb changed the title Add audit-logs download command with resumable chunked export Add audit-logs download command Jul 10, 2026
Comment thread cmd/audit_logs_download.go

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ee8ccd4. Configure here.

Comment thread cmd/audit_logs_download.go Outdated
Comment thread cmd/audit_logs_download.go
Comment thread cmd/audit_logs_download.go
@pulumi

pulumi Bot commented Jul 13, 2026

Copy link
Copy Markdown

⚠️ Pulumi could not deploy preview(s) for this pull request because GitHub reports it is not mergeable (mergeable state: dirty). This usually means the branch has merge conflicts with its base branch. Resolve the conflicts and push a new commit to retry.

@yummybomb yummybomb requested a review from sjmiller609 July 13, 2026 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant