Skip to content

refactor(agent-service): redesign sync-execution result and error model#6009

Open
bobbai00 wants to merge 5 commits into
apache:mainfrom
bobbai00:refactor/sync-execution-result-model
Open

refactor(agent-service): redesign sync-execution result and error model#6009
bobbai00 wants to merge 5 commits into
apache:mainfrom
bobbai00:refactor/sync-execution-result-model

Conversation

@bobbai00

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Restructures the per-operator summary the sync-execution backend returns and the agent-service / frontend consume, for a leaner, consistent wire contract. This is a focused re-do of #5927 cut directly from main (no foundation stack): it changes only the execution result/error model and its consumers.

  • Replace the flat OperatorInfo with OperatorExecutionSummary (orthogonal sub-summaries: state, errorMessages, resultSummary?, consoleLogsSummary?); rename SyncExecutionResultWorkflowExecutionSummary.
  • resultSummary.sampleTuples is now SampleRow[] ({ rowIndex, tuple }) instead of JSON rows with an embedded __row_index__; drop the table-shape types (the agent derives input-port shapes from the DAG).
  • Move WorkflowFatalError into types/execution.ts and reuse it for per-operator errors — the same type the workflow-compiling service returns for compilation errors, so compile and execution errors share one wire shape; api/compile-api.ts re-exports it so its existing importers are unchanged.
  • errorMessages / errors are non-optional (empty = none); drop compilationErrors; collapse the console-message types and derive warnings from WARNING:-titled messages.
  • Operator results are still pulled on demand via GET /agents/:id/operator-results (transport unchanged); that REST payload now carries the canonical OperatorExecutionSummary, and the frontend maps it to its flat display type (re-flattening sampleTuples so the display components are unchanged).

Touches the Scala producer (SyncExecutionResource), the agent-service consumers (result-formatting, workflow-execution-tools, workflow-result-state, server), and the frontend mapping. Representation/type-level; behavior preserved (input-port shape lines are now derived rather than explicitly rendered).

Any related issues, documentation, discussions?

Closes #5750
Part of #5747.
Supersedes #5927.

How was this PR tested?

  • agent-service: tsc --noEmit clean, bun test 110/110 pass, prettier --check clean.
  • The Scala producer (SyncExecutionResource) is unchanged from refactor(agent-service): redesign sync-execution result and error model #5927, which verified it via sbt WorkflowExecutionService/compile and a full-stack end-to-end run (a Claude Haiku 4.5 agent built and executed a CSV workflow; /operator-results returned the new shape — resultSummary.sampleTuples: [{ rowIndex, tuple }], errorMessages: []).

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.8 (1M context)

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @Ma77Ball, @Yicong-Huang
    You can notify them by mentioning @Ma77Ball, @Yicong-Huang in a comment.

@codecov-commenter

codecov-commenter commented Jun 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 64.16185% with 62 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.90%. Comparing base (16a3ddf) to head (43166b9).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
...ervice/src/agent/tools/workflow-execution-tools.ts 8.88% 41 Missing ⚠️
...t/agent-interaction/agent-interaction.component.ts 14.28% 10 Missing and 2 partials ⚠️
...agent-interaction/agent-interaction.component.html 0.00% 4 Missing and 2 partials ⚠️
agent-service/src/agent/texera-agent.ts 0.00% 1 Missing ⚠️
agent-service/src/agent/workflow-result-state.ts 50.00% 1 Missing ⚠️
...result-table-frame/result-table-frame.component.ts 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #6009      +/-   ##
============================================
+ Coverage     56.64%   56.90%   +0.26%     
  Complexity     3028     3028              
============================================
  Files          1124     1125       +1     
  Lines         43298    43126     -172     
  Branches       4667     4647      -20     
============================================
+ Hits          24525    24540      +15     
+ Misses        17335    17145     -190     
- Partials       1438     1441       +3     
Flag Coverage Δ *Carryforward flag
access-control-service 70.00% <ø> (ø) Carriedforward from 84022c9
agent-service 45.06% <63.55%> (+0.46%) ⬆️
amber 58.96% <ø> (+0.34%) ⬆️ Carriedforward from 84022c9
computing-unit-managing-service 0.00% <ø> (ø) Carriedforward from 84022c9
config-service 52.30% <ø> (ø) Carriedforward from 84022c9
file-service 62.81% <ø> (ø) Carriedforward from 84022c9
frontend 49.58% <65.45%> (+0.19%) ⬆️
notebook-migration-service 78.57% <ø> (ø) Carriedforward from 84022c9
pyamber 90.20% <ø> (ø) Carriedforward from 84022c9
python 90.76% <ø> (ø) Carriedforward from 84022c9
workflow-compiling-service 55.14% <ø> (ø) Carriedforward from 84022c9

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 4 worse · ⚪ 9 noise (<±5%) · 0 without baseline

Compared against main 16a3ddf benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 415 0.253 23,026/31,343/31,343 us 🟢 -14.0% / 🔴 +104.4%
🔴 bs=100 sw=10 sl=64 927 0.566 109,426/127,858/127,858 us 🔴 +5.2% / 🔴 +15.6%
bs=1000 sw=10 sl=64 1,088 0.664 917,919/981,651/981,651 us ⚪ within ±5% / 🟢 +11.1%
Baseline details

Latest main 16a3ddf from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 415 tuples/sec 445 tuples/sec 757.16 tuples/sec -6.7% -45.2%
bs=10 sw=10 sl=64 MB/s 0.253 MB/s 0.271 MB/s 0.462 MB/s -6.6% -45.3%
bs=10 sw=10 sl=64 p50 23,026 us 20,250 us 12,971 us +13.7% +77.5%
bs=10 sw=10 sl=64 p95 31,343 us 36,438 us 15,333 us -14.0% +104.4%
bs=10 sw=10 sl=64 p99 31,343 us 36,438 us 18,732 us -14.0% +67.3%
bs=100 sw=10 sl=64 throughput 927 tuples/sec 971 tuples/sec 957.66 tuples/sec -4.5% -3.2%
bs=100 sw=10 sl=64 MB/s 0.566 MB/s 0.593 MB/s 0.585 MB/s -4.6% -3.2%
bs=100 sw=10 sl=64 p50 109,426 us 104,025 us 103,982 us +5.2% +5.2%
bs=100 sw=10 sl=64 p95 127,858 us 122,334 us 110,583 us +4.5% +15.6%
bs=100 sw=10 sl=64 p99 127,858 us 122,334 us 118,369 us +4.5% +8.0%
bs=1000 sw=10 sl=64 throughput 1,088 tuples/sec 1,114 tuples/sec 979.6 tuples/sec -2.3% +11.1%
bs=1000 sw=10 sl=64 MB/s 0.664 MB/s 0.68 MB/s 0.598 MB/s -2.4% +11.1%
bs=1000 sw=10 sl=64 p50 917,919 us 895,581 us 1,024,553 us +2.5% -10.4%
bs=1000 sw=10 sl=64 p95 981,651 us 956,229 us 1,063,789 us +2.7% -7.7%
bs=1000 sw=10 sl=64 p99 981,651 us 956,229 us 1,096,239 us +2.7% -10.5%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,482.01,200,128000,415,0.253,23026.11,31342.52,31342.52
1,100,10,64,20,2157.69,2000,1280000,927,0.566,109426.49,127857.60,127857.60
2,1000,10,64,20,18389.10,20000,12800000,1088,0.664,917918.64,981651.47,981651.47

@bobbai00 bobbai00 marked this pull request as draft June 29, 2026 01:00
bobbai00 added 4 commits June 28, 2026 21:56
### What changes were proposed in this PR?

Restructures the per-operator summary the sync-execution backend returns and the agent-service / frontend consume, for a leaner, consistent wire contract. This is a focused re-do of apache#5927 cut directly from `main` (no foundation stack): it changes only the execution result/error model and its consumers.

- Replace the flat `OperatorInfo` with `OperatorExecutionSummary` (orthogonal sub-summaries: `state`, `errorMessages`, `resultSummary?`, `consoleLogsSummary?`); rename `SyncExecutionResult` → `WorkflowExecutionSummary`.
- `resultSummary.sampleTuples` is now `SampleRow[]` (`{ rowIndex, tuple }`) instead of JSON rows with an embedded `__row_index__`; drop the table-shape types (the agent derives input-port shapes from the DAG).
- Move `WorkflowFatalError` into `types/execution.ts` and reuse it for per-operator errors — the same type the workflow-compiling service returns for compilation errors, so compile and execution errors share one wire shape; `api/compile-api.ts` re-exports it so its existing importers are unchanged.
- `errorMessages` / `errors` are non-optional (empty = none); drop `compilationErrors`; collapse the console-message types and derive warnings from `WARNING:`-titled messages.
- Operator results are still pulled on demand via `GET /agents/:id/operator-results` (transport unchanged); that REST payload now carries the canonical `OperatorExecutionSummary`, and the frontend maps it to its flat display type (re-flattening `sampleTuples` so the display components are unchanged).

Touches the Scala producer (`SyncExecutionResource`), the agent-service consumers (`result-formatting`, `workflow-execution-tools`, `workflow-result-state`, `server`), and the frontend mapping. Representation/type-level; behavior preserved (input-port shape lines are now derived rather than explicitly rendered).

### Any related issues, documentation, discussions?

Closes apache#5750
Part of apache#5747.
Supersedes apache#5927.

### How was this PR tested?

- agent-service: `tsc --noEmit` clean, `bun test` 110/110 pass, `prettier --check` clean.
- The Scala producer (`SyncExecutionResource`) is unchanged from apache#5927, which verified it via `sbt WorkflowExecutionService/compile` and a full-stack end-to-end run (a Claude Haiku 4.5 agent built and executed a CSV workflow; `/operator-results` returned the new shape — `resultSummary.sampleTuples: [{ rowIndex, tuple }]`, `errorMessages: []`).

### Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.8 (1M context)
@bobbai00 bobbai00 force-pushed the refactor/sync-execution-result-model branch from 89eb9e9 to 84022c9 Compare June 29, 2026 04:56
@bobbai00 bobbai00 marked this pull request as ready for review June 29, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent-service engine frontend Changes related to the frontend GUI refactor Refactor the code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(agent-service): redesign the sync-execution result and error model

2 participants