Skip to content

fix(iterator): stop silently truncating spread/Array.from at 100,000 elements (#7562) - #7567

Merged
proggeramlug merged 2 commits into
mainfrom
fix/7562-iterator-truncation
Aug 7, 2026
Merged

fix(iterator): stop silently truncating spread/Array.from at 100,000 elements (#7562)#7567
proggeramlug merged 2 commits into
mainfrom
fix/7562-iterator-truncation

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Fixes #7562.

The bug

[...m.values()] on a 250,000-entry Map returned 100,000 elements. No error, no warning — a plausible-looking wrong answer.

                 before          node
map size:        250000          250000
spread length:   100000    <-    250000
Array.from:      100000    <-    250000

Three drain loops in array/iterator.rs carried a hardcoded for _ in 0..100_000 "safety limit" and simply fell out of the loop when it was hit, returning whatever had accumulated. This affected every spread, Array.from, and iterator-protocol drain — Maps, Sets, generators, user iterables alike.

Same severity class as the JSON key-loss corruption fixed earlier (#7546): silent wrong output is worse than a crash, because nothing signals it.

The fix, and why not simply remove the limit

  • The bound becomes MAX_ITERATOR_DRAIN = JavaScript's own maximum array length (u32::MAX - 1). Every realistic workload is unaffected.
  • Exhausting it now throws a RangeError instead of truncating.

Node applies no limit at all: [...it] runs until the iterator finishes or the process runs out of memory. Matching that exactly would trade silent truncation for an unbounded loop on a runaway iterator — so I kept a bound but made it report itself. A visible, recoverable error is the right third option; returning short data is strictly worse than either hanging or throwing.

Verification

  • test_gap_7562_iterator_no_truncation.ts — all three drain paths past the old bound (Map values/keys/entries and [...m], a 130k Set, a 120k generator), plus a small-iterator case to prove the common path is untouched. Byte-identical to node 26.5.1.
  • Sabotage-verified: restoring the 100,000 bound makes that test exit 1. A green run is therefore evidence, not decoration.
  • cargo test -p perry-runtime --no-fail-fast: 1808 passed, 0 failed.
  • raw_handle_debt.py 998 (baseline), check_file_size.sh, check_test_registration.py, cargo fmt --all --check all clean.

Pre-existing — confirmed at 969b447cc. Found by the map_1m investigation (#7561) while building its semantics matrix.

CI has a deep repo-wide runner backlog and may not report; the above is local validation and I am not claiming CI green.

Summary by CodeRabbit

  • Bug Fixes

    • Iterator conversions no longer silently truncate results after 100,000 elements.
    • Large Map, Set, and generator iterators now preserve all values.
    • Exceeding the supported array size now reports a RangeError.
  • Tests

    • Added coverage for large synchronous and asynchronous iterator operations.
  • Chores

    • Updated the application version to 0.5.1319.

@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c4e0442d-fe5a-4555-b8d0-44a6d6ef1466

📥 Commits

Reviewing files that changed from the base of the PR and between dc0778c and bf4a3b9.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (5)
  • CLAUDE.md
  • Cargo.toml
  • changelog.d/7565-iterator-no-silent-truncation.md
  • crates/perry-runtime/src/array/iterator.rs
  • test-files/test_gap_7562_iterator_no_truncation.ts

📝 Walkthrough

Walkthrough

Iterator drains now use a shared u32::MAX - 1 limit. Drains throw RangeError instead of silently truncating results. Regression tests cover large Map, Set, and generator inputs. Project version references are updated to 0.5.1319.

Changes

Iterator drain truncation fix

Layer / File(s) Summary
Runtime limit and drain behavior
crates/perry-runtime/src/array/iterator.rs
The runtime adds a shared maximum and RangeError helper. Synchronous and asynchronous iterator drains throw when the maximum is reached.
Regression coverage and release metadata
test-files/test_gap_7562_iterator_no_truncation.ts, changelog.d/7565-iterator-no-silent-truncation.md, CLAUDE.md, Cargo.toml
Tests validate complete large-iterator results and unchanged small-iterator behavior. The changelog and version references record the update.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

  • PerryTS/perry#7495: Both changes modify iterator-to-array handling, but address different runtime issues.

Suggested labels: bug, parity

Suggested reviewers: thehypnoo, andrewtdiz

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/7562-iterator-truncation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@proggeramlug
proggeramlug merged commit 286758e into main Aug 7, 2026
10 of 12 checks passed
@proggeramlug
proggeramlug deleted the fix/7562-iterator-truncation branch August 7, 2026 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Spread / Array.from over any iterator silently truncates at 100,000 elements

1 participant