improvement(provenance): attribute stored-envelope display reads to their execution (#7039)

* improvement(provenance): attribute stored-envelope display reads to their execution

A display materialization of an execution log imports the row's stored
provenance envelopes into throwaway registries, and each import of an
incomplete envelope re-emitted the registry's own summary — per envelope,
per view, carrying counts and a workspace but never the execution id. A
reader repeatedly materializing the same stored rows produced hundreds of
identical lines that could not say which executions to go look at, and
the volume scaled with views of a state that was fully recorded when the
run wrote it.

Verified against production before changing anything: essentially no new
incomplete envelopes are being stored since the writer fix shipped, and
no data drains exist — the stream is bounded re-reads of old rows through
the display paths, not a live producer.

The display registries are now staged — the existing concept for a
registry that filters one value for a caller that reports against the
real boundary — and each display function reports once per
materialization with the execution id, workflow, workspace, and the
parts that could not be vouched for. Severity is preserved: an
incomplete stored envelope stays at warn, a malformed one stays at
error. Projection behavior is unchanged everywhere — incomplete and
malformed envelopes still fail their values closed exactly as before;
only the reporting moves to the boundary that knows the execution.

* improvement(provenance): fold the incomplete-envelope predicate and pin dual-site reporting

Review pass over the previous commit: one helper instead of three copies
of the incomplete-envelope check, the staged TSDoc generalized to cover
both of its uses, and the block-outputs entry point's two-site reporting
of one run envelope documented and pinned rather than left implicit.

* improvement(provenance): classify every unusable stored envelope at the display boundary

Review findings from the first round, both accepted: a present-but-
malformed block or run envelope was withheld with no attributed line,
and a complete envelope whose entries fail decryption latched the
staged registry with only the unattributed entry-level error.

Fault classification moves into the one import helper the display
paths share, which now returns the registry and the fault together:
absent is not a fault, unparseable is malformed, unable-to-vouch is
incomplete, and a complete envelope whose registry latched during
import — entry decryption is the only latch on that trusted path —
is undecryptable. Every consumer reports through the same table,
severity per kind, so the exact-value loop stops being the only site
that could name a malformed envelope. Withholding behavior is
unchanged at every site.

Author: icecrasher321

apps/sim/executor/utils/resolved-secret-trace-registry.ts

@@ -880,9 +880,10 @@ export class ResolvedSecretTraceRegistry {
   private readonly scope?: ResolvedSecretTraceScopeV1
   private readonly completeProvenanceEnvelopeBytes: number
   /**
-   * A staged registry filters one value and is then discarded. Its caller re-reports whatever
-   * fault it hits against the real input path, so its own summary lines would restate that with
-   * strictly less context. Entry-level detail still logs — the caller cannot reconstruct it.
+   * A staged registry filters values for one operation and is then discarded. Its caller owns the
+   * reporting and says it with strictly more context — the real input path for a value filter, the
+   * execution for a display read — so the registry's own summary lines would only restate it.
+   * Entry-level detail still logs — the caller cannot reconstruct it.
    */
   private readonly staged: boolean
 

apps/sim/lib/logs/execution/trace-store.test.ts

@@ -3,10 +3,16 @@
  */
 import { beforeEach, describe, expect, it, vi } from 'vitest'
 
-const { decryptSecretMock, materializeLargeValueRefMock, storeLargeValueMock } = vi.hoisted(() => ({
-  decryptSecretMock: vi.fn(),
-  materializeLargeValueRefMock: vi.fn(),
-  storeLargeValueMock: vi.fn(),
+const { decryptSecretMock, materializeLargeValueRefMock, storeLargeValueMock, mockLogger } =
+  vi.hoisted(() => ({
+    decryptSecretMock: vi.fn(),
+    materializeLargeValueRefMock: vi.fn(),
+    storeLargeValueMock: vi.fn(),
+    mockLogger: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() },
+  }))
+
+vi.mock('@sim/logger', () => ({
+  createLogger: () => mockLogger,
 }))
 
 vi.mock('@/lib/core/security/encryption', () => ({
@@ -536,3 +542,233 @@ describe('projectExecutionDataForDisplay provenance handling', () => {
     expect(displayData.traceSpans).toEqual([])
   })
 })
+
+describe('stored provenance display reporting', () => {
+  const REGISTRY_SUMMARY_MESSAGES = [
+    'Resolved secret registry marked incomplete',
+    'Resolved secret input path marked incomplete',
+  ]
+
+  function registrySummaryLines(): unknown[] {
+    return [...mockLogger.warn.mock.calls, ...mockLogger.error.mock.calls].filter(([message]) =>
+      REGISTRY_SUMMARY_MESSAGES.includes(message as string)
+    )
+  }
+
+  /**
+   * The stored state was recorded when the run wrote it; a view re-deriving it must say which
+   * execution it served, once — not restate the latch through registry summaries that name none.
+   */
+  it('reports an incomplete stored envelope once, naming the execution and the parts', async () => {
+    const displayData = await projectExecutionDataForDisplay(
+      {
+        finalOutput: { result: 'value' },
+        executionState: {
+          resolvedSecretTraceProvenance: { version: 1, complete: false, entries: [] },
+          finalOutputResolvedSecretTraceProvenance: { version: 1, complete: false, entries: [] },
+        },
+      },
+      CONTEXT
+    )
+
+    expect(displayData).not.toHaveProperty('finalOutput')
+    expect(registrySummaryLines()).toHaveLength(0)
+    expect(mockLogger.warn).toHaveBeenCalledWith(
+      'Stored execution provenance cannot vouch for display content',
+      expect.objectContaining({
+        site: 'traceStore.displayProjection',
+        executionId: 'execution-1',
+        workflowId: 'workflow-1',
+        workspaceId: 'workspace-1',
+        parts: ['traceSpans', 'finalOutput'],
+        partCount: 2,
+      })
+    )
+    expect(mockLogger.error).not.toHaveBeenCalled()
+  })
+
+  it('reports a malformed stored envelope at error, keeping the value withheld', async () => {
+    const displayData = await projectExecutionDataForDisplay(
+      {
+        finalOutput: { result: 'value' },
+        executionState: {
+          resolvedSecretTraceProvenance: {
+            version: 1,
+            complete: true,
+            entries: [],
+            scope: { userId: 'user-1', workspaceId: 'workspace-1' },
+          },
+          finalOutputResolvedSecretTraceProvenance: 'garbage',
+        },
+      },
+      CONTEXT
+    )
+
+    expect(displayData).not.toHaveProperty('finalOutput')
+    expect(registrySummaryLines()).toHaveLength(0)
+    expect(mockLogger.error).toHaveBeenCalledWith(
+      'Stored execution provenance is malformed',
+      expect.objectContaining({
+        site: 'traceStore.displayProjection',
+        executionId: 'execution-1',
+        parts: ['finalOutput'],
+      })
+    )
+  })
+
+  /** A complete envelope whose entries cannot be decrypted withholds content like any fault. */
+  it('attributes an undecryptable stored envelope to its execution at error', async () => {
+    decryptSecretMock.mockRejectedValue(new Error('key rotated'))
+
+    const displayData = await projectExecutionDataForDisplay(
+      {
+        finalOutput: { result: 'value' },
+        executionState: {
+          resolvedSecretTraceProvenance: {
+            version: 1,
+            complete: true,
+            entries: [],
+            scope: { userId: 'user-1', workspaceId: 'workspace-1' },
+          },
+          finalOutputResolvedSecretTraceProvenance: {
+            version: 1,
+            complete: true,
+            entries: [{ name: 'SECRET', encryptedValue: 'ciphertext' }],
+            scope: { userId: 'user-1', workspaceId: 'workspace-1' },
+          },
+        },
+      },
+      CONTEXT
+    )
+
+    expect(displayData).not.toHaveProperty('finalOutput')
+    expect(registrySummaryLines()).toHaveLength(0)
+    expect(mockLogger.error).toHaveBeenCalledWith(
+      'Stored execution provenance could not be decrypted',
+      expect.objectContaining({
+        site: 'traceStore.displayProjection',
+        executionId: 'execution-1',
+        parts: ['finalOutput'],
+      })
+    )
+  })
+
+  it('reports a malformed block-output envelope at error, withholding the output', async () => {
+    const result = await materializeExecutionDataForDisplayWithBlockOutputs(
+      {
+        executionState: {
+          resolvedSecretTraceProvenance: {
+            version: 1,
+            complete: true,
+            entries: [],
+            scope: { userId: 'user-1', workspaceId: 'workspace-1' },
+          },
+          blockStates: {
+            'block-1': { output: { value: 1 }, resolvedSecretTraceProvenance: 'garbage' },
+          },
+        },
+      },
+      CONTEXT,
+      ['block-1']
+    )
+
+    expect(result.blockOutputs.has('block-1')).toBe(false)
+    expect(registrySummaryLines()).toHaveLength(0)
+    expect(mockLogger.error).toHaveBeenCalledWith(
+      'Stored execution provenance is malformed',
+      expect.objectContaining({
+        site: 'traceStore.blockOutputs',
+        executionId: 'execution-1',
+        parts: ['blockOutput:block-1'],
+      })
+    )
+  })
+
+  it('stays silent when every stored envelope is complete', async () => {
+    const displayData = await projectExecutionDataForDisplay(
+      {
+        finalOutput: { result: 'direct-literal' },
+        executionState: {
+          resolvedSecretTraceProvenance: {
+            version: 1,
+            complete: true,
+            entries: [],
+            scope: { userId: 'user-1', workspaceId: 'workspace-1' },
+          },
+          finalOutputResolvedSecretTraceProvenance: {
+            version: 1,
+            complete: true,
+            entries: [],
+            scope: { userId: 'user-1', workspaceId: 'workspace-1' },
+          },
+        },
+      },
+      CONTEXT
+    )
+
+    expect(displayData.finalOutput).toEqual({ result: 'direct-literal' })
+    expect(mockLogger.warn).not.toHaveBeenCalled()
+    expect(mockLogger.error).not.toHaveBeenCalled()
+  })
+
+  /** The block entry point runs both display functions; each names its own site for the envelope. */
+  it('attributes an incomplete run envelope under both sites on a block-outputs read', async () => {
+    await materializeExecutionDataForDisplayWithBlockOutputs(
+      {
+        finalOutput: { result: 'value' },
+        executionState: {
+          resolvedSecretTraceProvenance: { version: 1, complete: false, entries: [] },
+          blockStates: {
+            'block-1': { output: { value: 1 } },
+          },
+        },
+      },
+      CONTEXT,
+      ['block-1']
+    )
+
+    expect(registrySummaryLines()).toHaveLength(0)
+    expect(mockLogger.warn).toHaveBeenCalledWith(
+      'Stored execution provenance cannot vouch for display content',
+      expect.objectContaining({ site: 'traceStore.displayProjection', parts: ['traceSpans'] })
+    )
+    expect(mockLogger.warn).toHaveBeenCalledWith(
+      'Stored execution provenance cannot vouch for display content',
+      expect.objectContaining({ site: 'traceStore.blockOutputs', parts: ['run'] })
+    )
+  })
+
+  it('reports incomplete block-output envelopes once for the whole block read', async () => {
+    const result = await materializeExecutionDataForDisplayWithBlockOutputs(
+      {
+        executionState: {
+          resolvedSecretTraceProvenance: {
+            version: 1,
+            complete: true,
+            entries: [],
+            scope: { userId: 'user-1', workspaceId: 'workspace-1' },
+          },
+          blockStates: {
+            'block-1': {
+              output: { value: 1 },
+              resolvedSecretTraceProvenance: { version: 1, complete: false, entries: [] },
+            },
+          },
+        },
+      },
+      CONTEXT,
+      ['block-1']
+    )
+
+    expect(result.blockOutputs.has('block-1')).toBe(false)
+    expect(registrySummaryLines()).toHaveLength(0)
+    expect(mockLogger.warn).toHaveBeenCalledWith(
+      'Stored execution provenance cannot vouch for display content',
+      expect.objectContaining({
+        site: 'traceStore.blockOutputs',
+        executionId: 'execution-1',
+        parts: ['blockOutput:block-1'],
+      })
+    )
+  })
+})