Skip to content

[codex] Handle lazy stream key request failures#2257

Draft
pranaygp wants to merge 1 commit into
mainfrom
pranaygp/codex/lazy-stream-key-request-diagnostics
Draft

[codex] Handle lazy stream key request failures#2257
pranaygp wants to merge 1 commit into
mainfrom
pranaygp/codex/lazy-stream-key-request-diagnostics

Conversation

@pranaygp
Copy link
Copy Markdown
Contributor

@pranaygp pranaygp commented Jun 4, 2026

Summary

  • defer Run.getReadable() encryption-key lookup until serialized stream data is actually read
  • include x-vercel-id and x-vercel-error in response-backed world-vercel transport errors and HTTP debug logs
  • add regressions for empty readable streams and Vercel correlation diagnostics

Root cause

Run.getReadable() eagerly started resolving the run encryption key. That path calls world.runs.get() (GET /v2/runs/:id?remoteRefBehavior=resolve) before any stream chunk requires deserialization. For unused or empty readable streams, a rejected lookup was not observed by a consumer and could surface as an unhandled rejection.

The Vercel request headers are only available when an HTTP response is received. A local AbortSignal.timeout() rejection still cannot include response headers because no Response exists in that path.

Validation

  • pnpm turbo build --filter=@workflow/core... --filter=@workflow/world-vercel...
  • pnpm --dir packages/core exec vitest run src/serialization.test.ts src/runtime/runs.test.ts
  • pnpm --dir packages/world-vercel exec vitest run src
  • git diff --check HEAD^ HEAD

@pranaygp
Handle lazy stream key request failures
f5e7c8d 
Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented Jun 4, 2026 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment Jun 4, 2026 9:39pm
example-nextjs-workflow-webpack Ready Ready Preview, Comment Jun 4, 2026 9:39pm
example-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workbench-astro-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workbench-express-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workbench-fastify-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workbench-hono-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workbench-nitro-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workbench-nuxt-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workbench-sveltekit-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workbench-tanstack-start-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workbench-vite-workflow Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workflow-docs Ready Ready Preview, Comment, Open in v0 Jun 4, 2026 9:39pm
workflow-swc-playground Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workflow-tarballs Ready Ready Preview, Comment Jun 4, 2026 9:39pm
workflow-web Ready Ready Preview, Comment Jun 4, 2026 9:39pm

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Jun 4, 2026

🦋 Changeset detected

Latest commit: f5e7c8d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
Name Type
@workflow/core Patch
@workflow/world-vercel Patch
@workflow/builders Patch
@workflow/cli Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/vitest Patch
@workflow/web-shared Patch
@workflow/web Patch
workflow Patch
@workflow/world-testing Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 4, 2026 •
edited
Loading

🧪 E2E Test Results

❌ Some tests failed

Summary

Passed Failed Skipped Total
❌ ▲ Vercel Production 1261 5 219 1485
✅ 💻 Local Development 1671 0 219 1890
❌ 📦 Local Production 1670 1 219 1890
✅ 🐘 Local Postgres 1671 0 219 1890
✅ 🪟 Windows 135 0 0 135
✅ 📋 Other 769 0 176 945
Total 7177 6 1052 8235

❌ Failed Tests

▲ Vercel Production (5 failed)

astro (2 failed):

  • hookWithSleepWorkflow - hook payloads delivered correctly with concurrent sleep | wrun_01KTA9JW32H369NQQR51XMGWF1 | 🔍 observability
  • experimental_setAttributes Promise.all of disjoint-key writes: every key lands

fastify (1 failed):

  • AbortController abortFetchInFlightWorkflow: aborting cancels an in-flight fetch

nextjs-turbopack (1 failed):

  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01KTA9H6HWA92Q4239E6FZN46Y | 🔍 observability

nextjs-webpack (1 failed):

  • pages router promiseAllWorkflow via pages router
📦 Local Production (1 failed)

fastify-stable (1 failed):

  • AbortController abortThrowIfAbortedWorkflow: throwIfAborted causes FatalError, no retries

Details by Category

❌ ▲ Vercel Production
App Passed Failed Skipped
❌ astro 107 2 26
✅ example 109 0 26
✅ express 109 0 26
❌ fastify 108 1 26
✅ hono 109 0 26
❌ nextjs-turbopack 132 1 2
❌ nextjs-webpack 132 1 2
✅ nitro 109 0 26
✅ nuxt 109 0 26
✅ sveltekit 128 0 7
✅ vite 109 0 26
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 110 0 25
✅ express-stable 110 0 25
✅ fastify-stable 110 0 25
✅ hono-stable 110 0 25
✅ nextjs-turbopack-canary 116 0 19
✅ nextjs-turbopack-stable-lazy-discovery-disabled 135 0 0
✅ nextjs-turbopack-stable-lazy-discovery-enabled 135 0 0
✅ nextjs-webpack-canary 116 0 19
✅ nextjs-webpack-stable-lazy-discovery-disabled 135 0 0
✅ nextjs-webpack-stable-lazy-discovery-enabled 135 0 0
✅ nitro-stable 110 0 25
✅ nuxt-stable 110 0 25
✅ sveltekit-stable 129 0 6
✅ vite-stable 110 0 25
❌ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 110 0 25
✅ express-stable 110 0 25
❌ fastify-stable 109 1 25
✅ hono-stable 110 0 25
✅ nextjs-turbopack-canary 116 0 19
✅ nextjs-turbopack-stable-lazy-discovery-disabled 135 0 0
✅ nextjs-turbopack-stable-lazy-discovery-enabled 135 0 0
✅ nextjs-webpack-canary 116 0 19
✅ nextjs-webpack-stable-lazy-discovery-disabled 135 0 0
✅ nextjs-webpack-stable-lazy-discovery-enabled 135 0 0
✅ nitro-stable 110 0 25
✅ nuxt-stable 110 0 25
✅ sveltekit-stable 129 0 6
✅ vite-stable 110 0 25
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 110 0 25
✅ express-stable 110 0 25
✅ fastify-stable 110 0 25
✅ hono-stable 110 0 25
✅ nextjs-turbopack-canary 116 0 19
✅ nextjs-turbopack-stable-lazy-discovery-disabled 135 0 0
✅ nextjs-turbopack-stable-lazy-discovery-enabled 135 0 0
✅ nextjs-webpack-canary 116 0 19
✅ nextjs-webpack-stable-lazy-discovery-disabled 135 0 0
✅ nextjs-webpack-stable-lazy-discovery-enabled 135 0 0
✅ nitro-stable 110 0 25
✅ nuxt-stable 110 0 25
✅ sveltekit-stable 129 0 6
✅ vite-stable 110 0 25
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 135 0 0
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 110 0 25
✅ e2e-local-dev-tanstack-start- 110 0 25
✅ e2e-local-postgres-nest-stable 110 0 25
✅ e2e-local-postgres-tanstack-start- 110 0 25
✅ e2e-local-prod-nest-stable 110 0 25
✅ e2e-local-prod-tanstack-start- 110 0 25
✅ e2e-vercel-prod-tanstack-start 109 0 26

📋 View full workflow run

❌ Some E2E test jobs failed:

  • Vercel Prod: failure
  • Local Dev: success
  • Local Prod: failure
  • Local Postgres: success
  • Windows: success

Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 4, 2026 •
edited
Loading

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Nitro 0.043s (+1.6%) 1.006s (~) 0.962s 10 1.00x
💻 Local Express 0.044s (+1.6%) 1.006s (~) 0.962s 10 1.01x
🐘 Postgres Express 0.053s (-20.4% 🟢) 1.012s (~) 0.959s 10 1.21x
💻 Local Next.js (Turbopack) 0.059s (+8.0% 🔺) 1.005s (~) 0.946s 10 1.37x
🐘 Postgres Nitro 0.062s (-2.4%) 1.013s (~) 0.951s 10 1.42x
🐘 Postgres Next.js (Turbopack) 0.068s (-2.3%) 1.012s (~) 0.944s 10 1.57x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 0.308s (+18.1% 🔺) 2.329s (-4.4%) 2.021s 10 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 1 step

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 1.097s (~) 2.006s (~) 0.910s 10 1.00x
🐘 Postgres Nitro 1.104s (-0.8%) 2.010s (~) 0.906s 10 1.01x
💻 Local Nitro 1.105s (+0.8%) 2.006s (~) 0.901s 10 1.01x
🐘 Postgres Express 1.112s (+1.9%) 2.010s (~) 0.898s 10 1.01x
💻 Local Next.js (Turbopack) 1.131s (+2.0%) 2.006s (~) 0.875s 10 1.03x
🐘 Postgres Next.js (Turbopack) 1.149s (+0.7%) 2.009s (~) 0.860s 10 1.05x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 1.645s (+5.0% 🔺) 3.811s (+8.0% 🔺) 2.166s 10 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 10 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 10.520s (~) 11.019s (~) 0.500s 3 1.00x
💻 Local Express 10.524s (~) 11.021s (~) 0.497s 3 1.00x
💻 Local Nitro 10.526s (~) 11.023s (~) 0.497s 3 1.00x
🐘 Postgres Express 10.577s (-1.3%) 11.018s (-2.9%) 0.440s 3 1.01x
💻 Local Next.js (Turbopack) 10.753s (+1.4%) 11.021s (~) 0.268s 3 1.02x
🐘 Postgres Next.js (Turbopack) 10.894s (+0.7%) 11.355s (+3.1%) 0.461s 3 1.04x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 13.600s (+2.1%) 15.883s (+1.9%) 2.283s 2 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 25 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 13.740s (-1.2%) 14.026s (-1.4%) 0.285s 5 1.00x
💻 Local Nitro 13.820s (+0.7%) 14.226s (+1.4%) 0.406s 5 1.01x
🐘 Postgres Nitro 13.870s (~) 14.017s (~) 0.147s 5 1.01x
🐘 Postgres Express 13.879s (~) 14.019s (-1.5%) 0.140s 5 1.01x
💻 Local Next.js (Turbopack) 14.276s (+1.8%) 15.029s (+2.7%) 0.753s 4 1.04x
🐘 Postgres Next.js (Turbopack) 14.426s (-1.1%) 15.017s (~) 0.591s 4 1.05x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 21.438s (-16.0% 🟢) 23.827s (-13.7% 🟢) 2.389s 3 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 50 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 12.397s (-2.7%) 13.017s (~) 0.620s 7 1.00x
💻 Local Express 12.406s (-1.5%) 13.024s (~) 0.618s 7 1.00x
💻 Local Nitro 12.430s (~) 13.024s (~) 0.594s 7 1.00x
🐘 Postgres Express 12.541s (-1.5%) 13.019s (-2.2%) 0.479s 7 1.01x
💻 Local Next.js (Turbopack) 13.672s (+5.6% 🔺) 14.027s (+5.4% 🔺) 0.355s 7 1.10x
🐘 Postgres Next.js (Turbopack) 13.775s (~) 14.020s (~) 0.244s 7 1.11x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 31.405s (+8.1% 🔺) 34.159s (+8.8% 🔺) 2.754s 3 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

Promise.all with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.181s (-1.6%) 2.007s (-0.7%) 0.826s 15 1.00x
🐘 Postgres Nitro 1.193s (-0.5%) 2.007s (~) 0.815s 15 1.01x
💻 Local Nitro 1.213s (~) 2.006s (~) 0.793s 15 1.03x
🐘 Postgres Next.js (Turbopack) 1.240s (-0.9%) 2.008s (~) 0.768s 15 1.05x
💻 Local Express 1.265s (~) 2.006s (~) 0.742s 15 1.07x
💻 Local Next.js (Turbopack) 1.326s (+8.4% 🔺) 2.006s (~) 0.681s 15 1.12x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.463s (-11.1% 🟢) 3.999s (-7.9% 🟢) 1.535s 8 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

Promise.all with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.234s (-6.9% 🟢) 2.008s (~) 0.774s 15 1.00x
🐘 Postgres Nitro 1.278s (+1.6%) 2.008s (~) 0.730s 15 1.04x
🐘 Postgres Next.js (Turbopack) 1.370s (-2.8%) 2.006s (~) 0.636s 15 1.11x
💻 Local Next.js (Turbopack) 1.660s (-3.8%) 2.149s (+7.1% 🔺) 0.489s 14 1.35x
💻 Local Nitro 1.854s (+8.8% 🔺) 2.150s (+3.7%) 0.295s 14 1.50x
💻 Local Express 1.896s (+0.7%) 2.292s (+3.2%) 0.396s 14 1.54x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.534s (-13.2% 🟢) 5.503s (-11.4% 🟢) 1.969s 6 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

Promise.all with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.343s (-3.7%) 2.007s (~) 0.664s 15 1.00x
🐘 Postgres Nitro 1.395s (-0.8%) 2.007s (~) 0.612s 15 1.04x
🐘 Postgres Next.js (Turbopack) 1.697s (-3.6%) 2.222s (-3.2%) 0.525s 14 1.26x
💻 Local Next.js (Turbopack) 4.512s (+2.6%) 5.178s (+3.3%) 0.666s 6 3.36x
💻 Local Express 5.349s (+0.9%) 5.845s (-2.8%) 0.497s 6 3.98x
💻 Local Nitro 5.511s (+16.8% 🔺) 6.014s (+20.0% 🔺) 0.502s 5 4.11x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 6.113s (-9.3% 🟢) 8.104s (-6.9% 🟢) 1.992s 5 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

Promise.race with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.186s (-1.4%) 2.009s (~) 0.822s 15 1.00x
🐘 Postgres Nitro 1.194s (~) 2.010s (~) 0.815s 15 1.01x
🐘 Postgres Next.js (Turbopack) 1.242s (-1.9%) 2.008s (~) 0.765s 15 1.05x
💻 Local Next.js (Turbopack) 1.392s (+6.6% 🔺) 2.006s (~) 0.614s 15 1.17x
💻 Local Nitro 1.546s (~) 2.006s (~) 0.460s 15 1.30x
💻 Local Express 1.563s (-5.7% 🟢) 2.006s (-3.2%) 0.444s 15 1.32x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.510s (-6.1% 🟢) 3.998s (-7.7% 🟢) 1.488s 8 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

Promise.race with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.238s (~) 2.008s (~) 0.770s 15 1.00x
🐘 Postgres Nitro 1.269s (~) 2.008s (~) 0.739s 15 1.02x
🐘 Postgres Next.js (Turbopack) 1.404s (~) 2.008s (~) 0.604s 15 1.13x
💻 Local Next.js (Turbopack) 1.935s (-4.3%) 2.314s (-7.6% 🟢) 0.380s 13 1.56x
💻 Local Nitro 1.996s (+13.6% 🔺) 2.393s (+15.4% 🔺) 0.397s 13 1.61x
💻 Local Express 2.055s (-3.3%) 2.592s (~) 0.537s 12 1.66x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.453s (-15.7% 🟢) 5.394s (-10.1% 🟢) 1.941s 6 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

Promise.race with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.354s (-9.7% 🟢) 2.151s (-10.3% 🟢) 0.797s 14 1.00x
🐘 Postgres Nitro 1.399s (-4.3%) 2.008s (~) 0.609s 15 1.03x
🐘 Postgres Next.js (Turbopack) 1.689s (+2.0%) 2.151s (+3.7%) 0.462s 14 1.25x
💻 Local Next.js (Turbopack) 5.426s (+5.1% 🔺) 6.215s (+3.3%) 0.789s 5 4.01x
💻 Local Express 5.973s (+1.9%) 6.615s (+6.5% 🔺) 0.642s 5 4.41x
💻 Local Nitro 6.192s (+30.6% 🔺) 6.819s (+27.6% 🔺) 0.627s 5 4.57x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 6.620s (+34.3% 🔺) 8.844s (+31.3% 🔺) 2.224s 4 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 10 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.596s (-2.0%) 1.024s (+1.6%) 0.427s 59 1.00x
💻 Local Express 0.597s (-9.0% 🟢) 1.005s (-3.4%) 0.407s 60 1.00x
🐘 Postgres Express 0.606s (-9.2% 🟢) 1.041s (-0.7%) 0.435s 58 1.02x
💻 Local Nitro 0.609s (-24.3% 🟢) 1.005s (-16.6% 🟢) 0.396s 60 1.02x
🐘 Postgres Next.js (Turbopack) 0.818s (+0.7%) 1.023s (+1.7%) 0.206s 59 1.37x
💻 Local Next.js (Turbopack) 0.856s (+22.9% 🔺) 1.021s (+1.7%) 0.165s 59 1.43x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 4.912s (-34.2% 🟢) 6.782s (-28.8% 🟢) 1.870s 10 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 25 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.367s (-3.5%) 2.029s (+1.1%) 0.663s 45 1.00x
🐘 Postgres Express 1.374s (-10.6% 🟢) 2.007s (-6.7% 🟢) 0.633s 45 1.01x
💻 Local Express 1.509s (-5.2% 🟢) 2.006s (-1.1%) 0.497s 45 1.10x
💻 Local Nitro 1.523s (+4.0%) 2.006s (~) 0.484s 45 1.11x
🐘 Postgres Next.js (Turbopack) 1.927s (-0.8%) 2.149s (-1.2%) 0.222s 43 1.41x
💻 Local Next.js (Turbopack) 2.071s (+19.8% 🔺) 2.851s (+40.6% 🔺) 0.780s 32 1.52x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 12.729s (-3.7%) 15.273s (-1.3%) 2.544s 6 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 50 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 2.652s (-9.0% 🟢) 3.033s (-7.5% 🟢) 0.381s 40 1.00x
🐘 Postgres Express 2.672s (-8.2% 🟢) 3.058s (-13.6% 🟢) 0.387s 40 1.01x
💻 Local Express 3.197s (-6.5% 🟢) 4.010s (~) 0.813s 30 1.21x
💻 Local Nitro 3.457s (+7.8% 🔺) 4.182s (+4.3%) 0.725s 29 1.30x
🐘 Postgres Next.js (Turbopack) 3.761s (-2.6%) 4.010s (-1.6%) 0.249s 30 1.42x
💻 Local Next.js (Turbopack) 4.302s (+15.4% 🔺) 5.011s (+21.9% 🔺) 0.709s 24 1.62x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 25.387s (-14.7% 🟢) 28.120s (-12.1% 🟢) 2.733s 5 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 10 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.211s (-10.6% 🟢) 1.006s (~) 0.796s 60 1.00x
🐘 Postgres Express 0.255s (+1.2%) 1.040s (+3.0%) 0.785s 58 1.21x
🐘 Postgres Next.js (Turbopack) 0.265s (-6.2% 🟢) 1.006s (~) 0.741s 60 1.26x
💻 Local Express 0.444s (-4.4%) 1.004s (~) 0.560s 60 2.11x
💻 Local Nitro 0.535s (+26.0% 🔺) 1.095s (+9.1% 🔺) 0.561s 55 2.54x
💻 Local Next.js (Turbopack) 0.601s (-1.6%) 1.022s (-1.6%) 0.421s 59 2.85x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.334s (~) 4.243s (-2.1%) 1.909s 15 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 25 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 0.331s (-2.2%) 1.006s (~) 0.675s 90 1.00x
🐘 Postgres Nitro 0.358s (-3.4%) 1.018s (+1.1%) 0.659s 89 1.08x
🐘 Postgres Next.js (Turbopack) 0.464s (-3.8%) 1.006s (~) 0.542s 90 1.40x
💻 Local Express 2.089s (-4.7%) 2.610s (-7.5% 🟢) 0.522s 35 6.30x
💻 Local Nitro 2.113s (+2.4%) 2.581s (-2.2%) 0.468s 35 6.37x
💻 Local Next.js (Turbopack) 2.398s (-2.1%) 3.044s (-2.1%) 0.645s 30 7.24x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 5.115s (-88.3% 🟢) 7.051s (-84.7% 🟢) 1.936s 14 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

workflow with 50 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 0.623s (~) 1.006s (-4.7%) 0.383s 120 1.00x
🐘 Postgres Nitro 0.654s (-7.5% 🟢) 1.006s (~) 0.352s 120 1.05x
🐘 Postgres Next.js (Turbopack) 0.943s (-3.7%) 1.323s (-17.7% 🟢) 0.380s 92 1.51x
💻 Local Nitro 9.639s (+10.7% 🔺) 10.360s (+11.0% 🔺) 0.721s 12 15.48x
💻 Local Express 9.707s (-5.9% 🟢) 10.191s (-6.2% 🟢) 0.484s 12 15.59x
💻 Local Next.js (Turbopack) 10.526s (+2.0%) 11.303s (~) 0.777s 11 16.90x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 116.711s (-62.5% 🟢) 118.780s (-62.1% 🟢) 2.069s 3 1.00x
▲ Vercel Express ⚠️ missing - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - -

🔍 Observability: Nitro

Stream Benchmarks (includes TTFB metrics)
workflow with stream

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.161s (-2.7%) 2.001s (~) 0.001s (-97.1% 🟢) 2.010s (-1.6%) 0.848s 10 1.00x
💻 Local Nitro 1.164s (+1.2%) 2.005s (~) 0.012s (+22.0% 🔺) 2.019s (~) 0.855s 10 1.00x
🐘 Postgres Nitro 1.167s (-1.2%) 2.004s (~) 0.001s (-21.4% 🟢) 2.011s (~) 0.844s 10 1.01x
💻 Local Express 1.175s (+0.5%) 2.005s (~) 0.013s (-1.6%) 2.020s (~) 0.845s 10 1.01x
💻 Local Next.js (Turbopack) 1.202s (+2.8%) 2.003s (~) 0.011s (+5.9% 🔺) 2.017s (~) 0.815s 10 1.03x
🐘 Postgres Next.js (Turbopack) 1.223s (-0.9%) 2.000s (~) 0.001s (-40.0% 🟢) 2.010s (~) 0.787s 10 1.05x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.208s (+2.0%) 3.683s (+7.8% 🔺) 12.823s (+2385.6% 🔺) 17.060s (+281.8% 🔺) 14.852s 10 1.00x
▲ Vercel Express ⚠️ missing - - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - - -

🔍 Observability: Nitro

stream pipeline with 5 transform steps (1MB)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.569s (-5.7% 🟢) 2.005s (~) 0.004s (+2.6%) 2.025s (~) 0.456s 30 1.00x
💻 Local Express 1.580s (-3.4%) 2.010s (~) 0.011s (-3.6%) 2.022s (~) 0.442s 30 1.01x
🐘 Postgres Express 1.584s (-8.8% 🟢) 2.006s (-6.4% 🟢) 0.004s (+16.0% 🔺) 2.022s (-6.4% 🟢) 0.438s 30 1.01x
💻 Local Nitro 1.593s (-8.6% 🟢) 2.010s (~) 0.010s (+9.8% 🔺) 2.022s (-8.0% 🟢) 0.430s 30 1.02x
💻 Local Next.js (Turbopack) 1.732s (+5.5% 🔺) 2.009s (~) 0.010s (-2.6%) 2.022s (~) 0.291s 30 1.10x
🐘 Postgres Next.js (Turbopack) 1.732s (-1.8%) 2.011s (~) 0.004s (-5.0%) 2.025s (~) 0.293s 30 1.10x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 6.104s (+6.2% 🔺) 7.557s (-0.5%) 0.582s (+72.3% 🔺) 8.700s (+2.5%) 2.596s 7 1.00x
▲ Vercel Express ⚠️ missing - - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - - -

🔍 Observability: Nitro

10 parallel streams (1MB each)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.697s (-3.6%) 1.015s (-1.5%) 0.000s (-66.7% 🟢) 1.046s (~) 0.350s 58 1.00x
🐘 Postgres Express 0.717s (-26.6% 🟢) 1.041s (-27.1% 🟢) 0.000s (-75.4% 🟢) 1.062s (-26.7% 🟢) 0.345s 57 1.03x
🐘 Postgres Next.js (Turbopack) 0.821s (-1.4%) 1.091s (+1.8%) 0.000s (+1.8%) 1.099s (+1.8%) 0.278s 55 1.18x
💻 Local Express 1.377s (-7.1% 🟢) 2.014s (~) 0.000s (-40.0% 🟢) 2.016s (~) 0.639s 30 1.98x
💻 Local Nitro 1.400s (+3.5%) 2.013s (~) 0.000s (-40.0% 🟢) 2.015s (~) 0.615s 30 2.01x
💻 Local Next.js (Turbopack) 1.500s (+3.6%) 2.012s (~) 0.000s (-12.5% 🟢) 2.016s (~) 0.516s 30 2.15x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.782s (+5.0% 🔺) 5.043s (-0.6%) 0.001s (+Infinity% 🔺) 5.602s (~) 1.820s 11 1.00x
▲ Vercel Express ⚠️ missing - - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - - -

🔍 Observability: Nitro

fan-out fan-in 10 streams (1MB each)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.484s (+5.2% 🔺) 2.176s (+9.0% 🔺) 0.000s (+Infinity% 🔺) 2.201s (+9.4% 🔺) 0.717s 28 1.00x
🐘 Postgres Express 1.486s (-8.0% 🟢) 2.180s (-3.3%) 0.000s (+Infinity% 🔺) 2.214s (-2.9%) 0.728s 28 1.00x
🐘 Postgres Next.js (Turbopack) 1.626s (-5.9% 🟢) 2.180s (-3.6%) 0.000s (NaN%) 2.188s (-3.6%) 0.563s 28 1.10x
💻 Local Express 3.023s (-9.3% 🟢) 3.768s (-1.8%) 0.000s (-33.3% 🟢) 3.778s (-1.7%) 0.755s 16 2.04x
💻 Local Next.js (Turbopack) 3.176s (-1.1%) 3.665s (-5.9% 🟢) 0.001s (+69.4% 🔺) 3.680s (-5.6% 🟢) 0.504s 17 2.14x
💻 Local Nitro 3.201s (+2.8%) 3.836s (-1.5%) 0.001s (+350.0% 🔺) 3.843s (-1.5%) 0.642s 16 2.16x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 66.287s (+841.3% 🔺) 68.223s (+695.9% 🔺) 0.000s (NaN%) 68.761s (+656.6% 🔺) 2.473s 5 1.00x
▲ Vercel Express ⚠️ missing - - - - -
▲ Vercel Next.js (Turbopack) ⚠️ missing - - - - -

🔍 Observability: Nitro

Summary

Fastest Framework by World

Winner determined by most benchmark wins

World 🥇 Fastest Framework Wins
💻 Local Express 12/21
🐘 Postgres Nitro 11/21
▲ Vercel Nitro 21/21
Fastest World by Framework

Winner determined by most benchmark wins

Framework 🥇 Fastest World Wins
Express 🐘 Postgres 14/21
Next.js (Turbopack) 🐘 Postgres 14/21
Nitro 🐘 Postgres 18/21
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Slurp: Time from first byte to complete stream consumption (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • Samples: Number of benchmark iterations run
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world (local development)
  • 🐘 Postgres: PostgreSQL database world (local development)
  • ▲ Vercel: Vercel production/preview deployment
  • 🌐 Turso: Community world (local development)
  • 🌐 MongoDB: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Jazz: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Redis + BullMQ: Community world (local development)
  • 🌐 Cloudflare: Community world (local development)
  • 🌐 MySQL: Community world (local development)
  • 🌐 Azure: Community world (local development)
  • 🌐 NATS JetStream: Community world (local development)
  • 🌐 Upstash: Community world (local development)

📋 View full workflow run

❌ Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@pranaygp