refactor(agent-service): redesign sync-execution result and error model#6009
refactor(agent-service): redesign sync-execution result and error model#6009bobbai00 wants to merge 5 commits into
Conversation
Automated Reviewer SuggestionsBased on the
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #6009 +/- ##
============================================
+ Coverage 56.64% 56.90% +0.26%
Complexity 3028 3028
============================================
Files 1124 1125 +1
Lines 43298 43126 -172
Branches 4667 4647 -20
============================================
+ Hits 24525 24540 +15
+ Misses 17335 17145 -190
- Partials 1438 1441 +3
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 415 | 0.253 | 23,026/31,343/31,343 us | 🟢 -14.0% / 🔴 +104.4% |
| 🔴 | bs=100 sw=10 sl=64 | 927 | 0.566 | 109,426/127,858/127,858 us | 🔴 +5.2% / 🔴 +15.6% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,088 | 0.664 | 917,919/981,651/981,651 us | ⚪ within ±5% / 🟢 +11.1% |
Baseline details
Latest main 16a3ddf from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 415 tuples/sec | 445 tuples/sec | 757.16 tuples/sec | -6.7% | -45.2% |
| bs=10 sw=10 sl=64 | MB/s | 0.253 MB/s | 0.271 MB/s | 0.462 MB/s | -6.6% | -45.3% |
| bs=10 sw=10 sl=64 | p50 | 23,026 us | 20,250 us | 12,971 us | +13.7% | +77.5% |
| bs=10 sw=10 sl=64 | p95 | 31,343 us | 36,438 us | 15,333 us | -14.0% | +104.4% |
| bs=10 sw=10 sl=64 | p99 | 31,343 us | 36,438 us | 18,732 us | -14.0% | +67.3% |
| bs=100 sw=10 sl=64 | throughput | 927 tuples/sec | 971 tuples/sec | 957.66 tuples/sec | -4.5% | -3.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.566 MB/s | 0.593 MB/s | 0.585 MB/s | -4.6% | -3.2% |
| bs=100 sw=10 sl=64 | p50 | 109,426 us | 104,025 us | 103,982 us | +5.2% | +5.2% |
| bs=100 sw=10 sl=64 | p95 | 127,858 us | 122,334 us | 110,583 us | +4.5% | +15.6% |
| bs=100 sw=10 sl=64 | p99 | 127,858 us | 122,334 us | 118,369 us | +4.5% | +8.0% |
| bs=1000 sw=10 sl=64 | throughput | 1,088 tuples/sec | 1,114 tuples/sec | 979.6 tuples/sec | -2.3% | +11.1% |
| bs=1000 sw=10 sl=64 | MB/s | 0.664 MB/s | 0.68 MB/s | 0.598 MB/s | -2.4% | +11.1% |
| bs=1000 sw=10 sl=64 | p50 | 917,919 us | 895,581 us | 1,024,553 us | +2.5% | -10.4% |
| bs=1000 sw=10 sl=64 | p95 | 981,651 us | 956,229 us | 1,063,789 us | +2.7% | -7.7% |
| bs=1000 sw=10 sl=64 | p99 | 981,651 us | 956,229 us | 1,096,239 us | +2.7% | -10.5% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,482.01,200,128000,415,0.253,23026.11,31342.52,31342.52
1,100,10,64,20,2157.69,2000,1280000,927,0.566,109426.49,127857.60,127857.60
2,1000,10,64,20,18389.10,20000,12800000,1088,0.664,917918.64,981651.47,981651.47### What changes were proposed in this PR? Restructures the per-operator summary the sync-execution backend returns and the agent-service / frontend consume, for a leaner, consistent wire contract. This is a focused re-do of apache#5927 cut directly from `main` (no foundation stack): it changes only the execution result/error model and its consumers. - Replace the flat `OperatorInfo` with `OperatorExecutionSummary` (orthogonal sub-summaries: `state`, `errorMessages`, `resultSummary?`, `consoleLogsSummary?`); rename `SyncExecutionResult` → `WorkflowExecutionSummary`. - `resultSummary.sampleTuples` is now `SampleRow[]` (`{ rowIndex, tuple }`) instead of JSON rows with an embedded `__row_index__`; drop the table-shape types (the agent derives input-port shapes from the DAG). - Move `WorkflowFatalError` into `types/execution.ts` and reuse it for per-operator errors — the same type the workflow-compiling service returns for compilation errors, so compile and execution errors share one wire shape; `api/compile-api.ts` re-exports it so its existing importers are unchanged. - `errorMessages` / `errors` are non-optional (empty = none); drop `compilationErrors`; collapse the console-message types and derive warnings from `WARNING:`-titled messages. - Operator results are still pulled on demand via `GET /agents/:id/operator-results` (transport unchanged); that REST payload now carries the canonical `OperatorExecutionSummary`, and the frontend maps it to its flat display type (re-flattening `sampleTuples` so the display components are unchanged). Touches the Scala producer (`SyncExecutionResource`), the agent-service consumers (`result-formatting`, `workflow-execution-tools`, `workflow-result-state`, `server`), and the frontend mapping. Representation/type-level; behavior preserved (input-port shape lines are now derived rather than explicitly rendered). ### Any related issues, documentation, discussions? Closes apache#5750 Part of apache#5747. Supersedes apache#5927. ### How was this PR tested? - agent-service: `tsc --noEmit` clean, `bun test` 110/110 pass, `prettier --check` clean. - The Scala producer (`SyncExecutionResource`) is unchanged from apache#5927, which verified it via `sbt WorkflowExecutionService/compile` and a full-stack end-to-end run (a Claude Haiku 4.5 agent built and executed a CSV workflow; `/operator-results` returned the new shape — `resultSummary.sampleTuples: [{ rowIndex, tuple }]`, `errorMessages: []`). ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Opus 4.8 (1M context)
89eb9e9 to
84022c9
Compare
What changes were proposed in this PR?
Restructures the per-operator summary the sync-execution backend returns and the agent-service / frontend consume, for a leaner, consistent wire contract. This is a focused re-do of #5927 cut directly from
main(no foundation stack): it changes only the execution result/error model and its consumers.OperatorInfowithOperatorExecutionSummary(orthogonal sub-summaries:state,errorMessages,resultSummary?,consoleLogsSummary?); renameSyncExecutionResult→WorkflowExecutionSummary.resultSummary.sampleTuplesis nowSampleRow[]({ rowIndex, tuple }) instead of JSON rows with an embedded__row_index__; drop the table-shape types (the agent derives input-port shapes from the DAG).WorkflowFatalErrorintotypes/execution.tsand reuse it for per-operator errors — the same type the workflow-compiling service returns for compilation errors, so compile and execution errors share one wire shape;api/compile-api.tsre-exports it so its existing importers are unchanged.errorMessages/errorsare non-optional (empty = none); dropcompilationErrors; collapse the console-message types and derive warnings fromWARNING:-titled messages.GET /agents/:id/operator-results(transport unchanged); that REST payload now carries the canonicalOperatorExecutionSummary, and the frontend maps it to its flat display type (re-flatteningsampleTuplesso the display components are unchanged).Touches the Scala producer (
SyncExecutionResource), the agent-service consumers (result-formatting,workflow-execution-tools,workflow-result-state,server), and the frontend mapping. Representation/type-level; behavior preserved (input-port shape lines are now derived rather than explicitly rendered).Any related issues, documentation, discussions?
Closes #5750
Part of #5747.
Supersedes #5927.
How was this PR tested?
tsc --noEmitclean,bun test110/110 pass,prettier --checkclean.SyncExecutionResource) is unchanged from refactor(agent-service): redesign sync-execution result and error model #5927, which verified it viasbt WorkflowExecutionService/compileand a full-stack end-to-end run (a Claude Haiku 4.5 agent built and executed a CSV workflow;/operator-resultsreturned the new shape —resultSummary.sampleTuples: [{ rowIndex, tuple }],errorMessages: []).Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8 (1M context)