Skip to content

chore: update runtime compatibility matrix #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
github-actions wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore: update runtime compatibility matrix #67

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 92 additions & 28 deletions packages/std/__tests__/_matrix/results.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"generatedAt": "2026-07-29",
"generatedAt": "2026-08-01",
"package": "@opentf/std",
"version": "0.16.0",
"runtimes": [
Expand All @@ -8,9 +8,9 @@
"label": "Node.js 20",
"version": "20.20.2",
"status": "partial",
"passed": 1149,
"passed": 1501,
"failed": 1,
"total": 1150,
"total": 1502,
"percent": 99.9,
"capabilities": {
"Intl.Segmenter": true,
Expand Down Expand Up @@ -45,9 +45,9 @@
"label": "Node.js 22",
"version": "22.23.1",
"status": "partial",
"passed": 1149,
"passed": 1501,
"failed": 1,
"total": 1150,
"total": 1502,
"percent": 99.9,
"capabilities": {
"Intl.Segmenter": true,
Expand Down Expand Up @@ -82,9 +82,9 @@
"label": "Node.js 24",
"version": "24.18.0",
"status": "pass",
"passed": 1150,
"passed": 1502,
"failed": 0,
"total": 1150,
"total": 1502,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -108,11 +108,11 @@
{
"id": "node26",
"label": "Node.js 26",
"version": "26.5.0",
"version": "26.5.1",
"status": "pass",
"passed": 1207,
"passed": 1559,
"failed": 0,
"total": 1207,
"total": 1559,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -138,9 +138,9 @@
"label": "Bun",
"version": "1.3.14",
"status": "pass",
"passed": 1150,
"passed": 1502,
"failed": 0,
"total": 1150,
"total": 1502,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -166,9 +166,9 @@
"label": "Deno",
"version": "2.9.4",
"status": "pass",
"passed": 1207,
"passed": 1559,
"failed": 0,
"total": 1207,
"total": 1559,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -192,11 +192,11 @@
{
"id": "esrun",
"label": "ES-Runtime",
"version": "0.12.0",
"version": "0.13.0",
"status": "pass",
"passed": 1207,
"passed": 1559,
"failed": 0,
"total": 1207,
"total": 1559,
"percent": 100,
"capabilities": {
"Intl.Segmenter": true,
Expand All @@ -222,10 +222,10 @@
"label": "LLRT",
"version": "0.8.0-beta",
"status": "partial",
"passed": 1118,
"failed": 32,
"total": 1150,
"percent": 97.2,
"passed": 1462,
"failed": 40,
"total": 1502,
"percent": 97.3,
"capabilities": {
"Intl.Segmenter": false,
"Intl.DateTimeFormat names": false,
Expand Down Expand Up @@ -298,23 +298,79 @@
"category": "runtime-deviation",
"reason": "Runtime does not perform `IteratorClose` when a `for await...of` loop exits early, so a `finally` block in an async generator never runs. Required by ECMA-262; the equivalent sync `for...of` path is handled correctly. The fix belongs in the runtime. For `streamToIter` this means a `break` neither cancels the stream nor releases the reader lock, so the stream stays locked — drain it rather than exiting early on this runtime.",
"requires": null,
"affects": ["fromIterAsync", "streamToIter"]
"affects": ["fromIterAsync", "streamToIter", "zipIterAsync"]
},
{
"title": "Iter > zipIterAsync > closes the sources when the consumer stops early",
"message": "expected false to be true // Object.is equality",
"category": "runtime-deviation",
"reason": "Runtime does not perform `IteratorClose` when a `for await...of` loop exits early, so a `finally` block in an async generator never runs. Required by ECMA-262; the equivalent sync `for...of` path is handled correctly. The fix belongs in the runtime. For `streamToIter` this means a `break` neither cancels the stream nor releases the reader lock, so the stream stays locked — drain it rather than exiting early on this runtime.",
"requires": null,
"affects": ["fromIterAsync", "streamToIter", "zipIterAsync"]
},
{
"title": "Number > formatCompact",
"message": "not a function",
"category": "missing-capability",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation or currencies.",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation, currencies, or a plain number with grouping separators.",
"requires": "Intl.NumberFormat",
"affects": ["formatCompact", "formatCurrency"]
"affects": ["formatCompact", "formatCurrency", "formatNumber"]
},
{
"title": "Number > formatCurrency",
"message": "not a function",
"category": "missing-capability",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation or currencies.",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation, currencies, or a plain number with grouping separators.",
"requires": "Intl.NumberFormat",
"affects": ["formatCompact", "formatCurrency", "formatNumber"]
},
{
"title": "Number > formatNumber > groups the thousands",
"message": "not a function",
"category": "missing-capability",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation, currencies, or a plain number with grouping separators.",
"requires": "Intl.NumberFormat",
"affects": ["formatCompact", "formatCurrency", "formatNumber"]
},
{
"title": "Number > formatNumber > follows the locale for the separators",
"message": "not a function",
"category": "missing-capability",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation, currencies, or a plain number with grouping separators.",
"requires": "Intl.NumberFormat",
"affects": ["formatCompact", "formatCurrency"]
"affects": ["formatCompact", "formatCurrency", "formatNumber"]
},
{
"title": "Number > formatNumber > turns off grouping",
"message": "not a function",
"category": "missing-capability",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation, currencies, or a plain number with grouping separators.",
"requires": "Intl.NumberFormat",
"affects": ["formatCompact", "formatCurrency", "formatNumber"]
},
{
"title": "Number > formatNumber > applies the fraction bounds",
"message": "not a function",
"category": "missing-capability",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation, currencies, or a plain number with grouping separators.",
"requires": "Intl.NumberFormat",
"affects": ["formatCompact", "formatCurrency", "formatNumber"]
},
{
"title": "Number > formatNumber > formats a ratio as a percentage",
"message": "not a function",
"category": "missing-capability",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation, currencies, or a plain number with grouping separators.",
"requires": "Intl.NumberFormat",
"affects": ["formatCompact", "formatCurrency", "formatNumber"]
},
{
"title": "Number > formatNumber > handles the awkward numbers",
"message": "not a function",
"category": "missing-capability",
"reason": "Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation, currencies, or a plain number with grouping separators.",
"requires": "Intl.NumberFormat",
"affects": ["formatCompact", "formatCurrency", "formatNumber"]
},
{
"title": "streamToText > splits every byte into its own chunk",
Expand All @@ -330,15 +386,23 @@
"category": "runtime-deviation",
"reason": "Runtime does not perform `IteratorClose` when a `for await...of` loop exits early, so a `finally` block in an async generator never runs. Required by ECMA-262; the equivalent sync `for...of` path is handled correctly. The fix belongs in the runtime. For `streamToIter` this means a `break` neither cancels the stream nor releases the reader lock, so the stream stays locked — drain it rather than exiting early on this runtime.",
"requires": null,
"affects": ["fromIterAsync", "streamToIter"]
"affects": ["fromIterAsync", "streamToIter", "zipIterAsync"]
},
{
"title": "streamToIter > leaves the stream open under preventCancel",
"message": "expected true to be false // Object.is equality",
"category": "runtime-deviation",
"reason": "Runtime does not perform `IteratorClose` when a `for await...of` loop exits early, so a `finally` block in an async generator never runs. Required by ECMA-262; the equivalent sync `for...of` path is handled correctly. The fix belongs in the runtime. For `streamToIter` this means a `break` neither cancels the stream nor releases the reader lock, so the stream stays locked — drain it rather than exiting early on this runtime.",
"requires": null,
"affects": ["fromIterAsync", "streamToIter"]
"affects": ["fromIterAsync", "streamToIter", "zipIterAsync"]
},
{
"title": "String > formatList > follows the locale",
"message": "expected 'a, b, and c' to be 'a, b und c' // Object.is equality",
"category": "missing-capability",
"reason": "Localised list joining needs `Intl.ListFormat`. Where it is absent `formatList` falls back to the English forms rather than throwing, so only the language differs and the shape of the output is unchanged — every other `formatList` test passes on such a runtime. Only this one asserts a non-English result.",
"requires": "Intl.ListFormat",
"affects": ["formatList"]
},
{
"title": "String > stringReverse > basic reversal",
Expand Down
34 changes: 20 additions & 14 deletions website/app/docs/env-support/page.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,18 @@ Where a runtime falls short, the cause is listed so you can tell the difference

{/* MATRIX:START */}

_Measured on 2026-07-29 against `@opentf/std@0.16.0`, running the full suite of 1150 tests on each runtime._
_Measured on 2026-08-01 against `@opentf/std@0.16.0`, running the full suite of 1502 tests on each runtime._

| Runtime | Version | Tests passing | Status |
| --- | --- | --- | --- |
| Node.js 20 | `20.20.2` | 1149 / 1150 (99.9%) | ⚠️ Partial |
| Node.js 22 | `22.23.1` | 1149 / 1150 (99.9%) | ⚠️ Partial |
| Node.js 24 | `24.18.0` | 1150 / 1150 (100%) | <CheckIcon size={18} /> Fully supported |
| Node.js 26 | `26.5.0` | 1207 / 1207 (100%) | <CheckIcon size={18} /> Fully supported |
| Bun | `1.3.14` | 1150 / 1150 (100%) | <CheckIcon size={18} /> Fully supported |
| Deno | `2.9.4` | 1207 / 1207 (100%) | <CheckIcon size={18} /> Fully supported |
| ES-Runtime | `0.12.0` | 1207 / 1207 (100%) | <CheckIcon size={18} /> Fully supported |
| LLRT | `0.8.0-beta` | 1118 / 1150 (97.2%) | ⚠️ Partial |
| Node.js 20 | `20.20.2` | 1501 / 1502 (99.9%) | ⚠️ Partial |
| Node.js 22 | `22.23.1` | 1501 / 1502 (99.9%) | ⚠️ Partial |
| Node.js 24 | `24.18.0` | 1502 / 1502 (100%) | ✅ Fully supported |
| Node.js 26 | `26.5.1` | 1559 / 1559 (100%) | ✅ Fully supported |
| Bun | `1.3.14` | 1502 / 1502 (100%) | ✅ Fully supported |
| Deno | `2.9.4` | 1559 / 1559 (100%) | ✅ Fully supported |
| ES-Runtime | `0.13.0` | 1559 / 1559 (100%) | ✅ Fully supported |
| LLRT | `0.8.0-beta` | 1462 / 1502 (97.3%) | ⚠️ Partial |

### Node.js 20 20.20.2

Expand Down Expand Up @@ -64,24 +64,30 @@ Localised duration text needs `Intl.DurationFormat` and `Intl.RelativeTimeFormat

Affected utilities: `Duration`. Everything else works as expected.

**Runtime spec deviation** (3 tests)
**Runtime spec deviation** (4 tests)

Runtime does not perform `IteratorClose` when a `for await...of` loop exits early, so a `finally` block in an async generator never runs. Required by ECMA-262; the equivalent sync `for...of` path is handled correctly. The fix belongs in the runtime. For `streamToIter` this means a `break` neither cancels the stream nor releases the reader lock, so the stream stays locked — drain it rather than exiting early on this runtime.

Affected utilities: `fromIterAsync`, `streamToIter`. Everything else works as expected.
Affected utilities: `fromIterAsync`, `streamToIter`, `zipIterAsync`. Everything else works as expected.

**Missing platform API — `Intl.NumberFormat` unavailable** (2 tests)
**Missing platform API — `Intl.NumberFormat` unavailable** (8 tests)

Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation or currencies.
Number formatting needs `Intl.NumberFormat`. Runtimes built without ICU cannot format compact notation, currencies, or a plain number with grouping separators.

Affected utilities: `formatCompact`, `formatCurrency`. Everything else works as expected.
Affected utilities: `formatCompact`, `formatCurrency`, `formatNumber`. Everything else works as expected.

**Runtime spec deviation — `TextDecoder stream: true` unavailable** (1 test)

Runtime `TextDecoder` ignores the `stream: true` option, so the trailing bytes of a multi-byte character split across two chunks are decoded as replacement characters instead of being held back for the next call. Required by the Encoding spec; the fix belongs in the runtime. Text arriving in whole characters per chunk is unaffected.

Affected utilities: `streamToText`, `streamToLines`. Everything else works as expected.

**Missing platform API — `Intl.ListFormat` unavailable** (1 test)

Localised list joining needs `Intl.ListFormat`. Where it is absent `formatList` falls back to the English forms rather than throwing, so only the language differs and the shape of the output is unchanged — every other `formatList` test passes on such a runtime. Only this one asserts a non-English result.

Affected utilities: `formatList`. Everything else works as expected.

**Missing platform API — `Intl.Segmenter` unavailable** (20 tests)

Grapheme segmentation needs `Intl.Segmenter`. Runtimes built without full ICU cannot count, reverse or wrap multi-codepoint characters correctly. Text that is entirely printable ASCII is measured without segmenting and works everywhere.
Expand Down