Skip to content

feat(sdk): publish findings through durable concurrent batches - #453

Open
kmbroai wants to merge 45 commits into
mainfrom
dev/kyleb/linear-publish-durable
Open

feat(sdk): publish findings through durable concurrent batches#453
kmbroai wants to merge 45 commits into
mainfrom
dev/kyleb/linear-publish-durable

Conversation

@kmbroai

@kmbroai kmbroai commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Publish prepared security findings in bounded concurrent batches and derive final results from validated durable records stored in local scan history.

Changes

  • Group connected Linear issue creation into batches of at most 20 findings.
  • Write finding-to-issue mappings into a private durable handoff instead of trusting model-generated summaries.
  • Validate exact finding, occurrence, destination, team, optional project, title, description, and priority before importing publication records.
  • Preserve team-only publication through private handoffs, recovered issue mappings, SQLite records, receipts, and results without inventing a project identifier.
  • Read final issue identities, URLs, and receipts back from the transactional workbench store.
  • Keep database-backed issue creation successful if its optional receipt fails, returning a sanitized warning without rerunning remote mutations.
  • Preserve partial successes, recover incomplete handoffs, and retain recoverable mappings if database persistence fails.
  • After cancellation, await Codex process-tree termination, persist verified completed issues, write an accurate partial receipt, and retain the private handoff with duplicate-avoidance recovery instructions.
  • Keep CLI commands, interactive presentation, and end-to-end CLI integration out of this SDK layer.

Testing

  • Real SQLite publication-store, connected-app event validation, and durable publisher tests: 70 passed with 286 assertions.
  • Verified a 41-finding 20 + 20 + 1 batching boundary, malformed or absent Codex output, out-of-order completion, exact destination validation, partial failures, and durable recovery.
  • Consolidated recovery assertions into the stronger case preserving both written and independently salvaged issue mappings when database persistence fails.
  • Verified pre-aborted requests, cancellation during preparation, interrupted partial imports, database failures, receipt failures, retained private handoffs, and real signal-resistant child processes.

Risk and rollout

  • Publication still uses the existing authenticated Codex profile and connected Linear app.
  • No issues are created before validating the completed local scan and every finding association.
  • Stacked on the workbench publication-persistence layer.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@kmbroai
kmbroai marked this pull request as ready for review August 15, 2026 19:30
wholley-oai
wholley-oai previously approved these changes Aug 15, 2026

@soyeon-oai soyeon-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the exact PR delta and exercised the stacked CLI against a real synthetic Linear publication. I found three durability/verification issues.

Comment thread sdk/typescript/src/publish.ts
const eventFailure = eventFailed.get(issue.findingId);
if (
saved === undefined &&
!observed.has(issue.findingId) &&

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Do not let a model-authored failure override a verified successful tool event. In a live synthetic E2E, the runtime observed and displayed a created Linear issue, but the handoff writer then failed and emitted a failure record; observed blocked this salvage path, so the CLI returned created: 0, persisted no association, deleted the handoff, and left the real issue remote. A retry would duplicate it. Prefer the exact-argument completed tool receipt, or retain recovery state and report an indeterminate outcome.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was reproduced in my e2e.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

failed in cli, but ticket was created.

Comment thread sdk/typescript/src/publish.ts

@soyeon-oai soyeon-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional issues reproduced during E2E review on the latest stack (#453 86384db; integrated CLI #455 1fd8b0d; identical publisher source).

);
continue;
}
if (observed.has(issue.findingId)) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve both successful issue IDs when duplicate handoffs are detected.

Found during E2E review:

completed issues: SYNTH-DUPLICATE-A, SYNTH-DUPLICATE-B
CLI: created=0, failed=1 | SQLite associations=0 | recovery handoffs=0

Detecting duplicates is correct, but discarding both IDs and deleting the handoff makes already-created issues unrecoverable. Persist the verified IDs or retain the handoff and report an indeterminate outcome.

handoffResults.created,
);
try {
result.created = await (

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Prevent concurrent publications from overwriting successful receipts.

Found during E2E review:

concurrent publication A: success
concurrent publication B: success
SQLite associations=2 | issues in the shared scan receipt=1

Fresh issues on republish are documented; silently losing one successful run from the scan-keyed receipt is the defect. Serialize same-scan publication or write attempt-scoped receipts.

environment,
);
options.signal?.throwIfAborted();
const handoff = await createPublicationHandoff(prepared, environment);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Remove source-bearing handoffs when the publisher cannot start.

Found during E2E review with a missing CODEX_CLI_PATH:

CLI exit=2 | spawned processes=0 | remote mutations=0
retained publication.json=1447 bytes | finding description=1011 bytes

The startup rejection bypasses cleanup, leaving finding details and source locations on disk despite no recoverable remote operation. Delete pre-mutation handoffs on startup failure.

Base automatically changed from dev/kyleb/linear-publish-persistence to main August 16, 2026 00:47
@daneschneider-oai
daneschneider-oai dismissed wholley-oai’s stale review August 16, 2026 00:47

The base branch was changed.

@github-actions github-actions Bot added the enhancement New feature or request label Aug 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants