Decompress: guess a multiple of the input for unknown-size frames

[KrisKennawayDD]

Supersedes #169 (per review: keep a single decode path instead of adding a retry branch).

Problem

decompressSizeHint returned a flat lower bound of decompressSizeBufferLimit (1 MB) whenever the frame did not carry its decompressed size — legacy zstd v0.5 frames and streaming frames compressed without a pledged size. So every Decompress of such a frame with no adequately-sized caller buffer (e.g. Decompress(nil, src)) allocated at least 1 MB regardless of the real payload size.

Background

The flat-bound sizing dates to two 2022 commits:

• 30c4b29 "Add a sanity limit to decompress buffer size allocation" added decompressSizeHint/decompressSizeBufferLimit to fix a DoS (Decoder doesn't check output size before allocating #60): the code was trusting ZSTD_getFrameContentSize, so a crafted frame could force a huge allocation. The cap (≤ 50× input, ≥ 1 MB) + streaming fallback fixed that.
• 489b911 "Remove Decompress multiple retries" then dropped the earlier grow-and-retry sizing, on the premise that "zstd 1.3.0+ frames always carry a size hint," so you either decode right away from the hint or fall back to streaming.

That premise does not hold for legacy v0.5 frames or unpledged streaming frames — which is exactly this case. For them the hint is absent, so the code fell back to the flat 1 MB bound on every decode.

Fix

Now that decompressSizeHint reports whether the size was found (added in #168), return 3× the input length for the unknown case instead of the flat bound. Decompress already falls back to streaming if that is too small, so this needs no new branch on the decode path — the logic stays in decompressSizeHint.

• It only ever sizes from the input length (never an attacker-claimed size), and 3× is always within the existing 50× cap, so the DoS guard from 30c4b29 is unchanged.
• BulkProcessor.Decompress has no streaming fallback (the streaming reader ignores the dictionary), so it keeps allocating the upper bound for unknown-size frames rather than this smaller guess, which it could not recover from.

🤖 Generated with Claude Code

[Viq111]

thanks!