fix: decode %3F back to "?" in toFileSystemPath (#427) - #428
Conversation
urlEncodePatterns encodes both # and ? when converting a filesystem path to a URL, but urlDecodePatterns only reversed #, $, &, ,, and @. A local path containing a literal ? (legal on POSIX filesystems) was encoded to %3F on the way in and never decoded back on the way out, so resolution of any such path failed with ENOENT. Adds the missing /%3F/g, "?" pair, in the same hex-ordered position the other pairs already follow. Adds a symmetric test for # alongside the new ? test.
|
Thanks for the fix — the intended literal- There is a double-decoding/path-aliasing regression: Because the new The new tests also fail in the live Windows Node and browser jobs. They use POSIX One smaller completeness issue: Suggested fix:
The focused and full suites pass locally on macOS, and I did not find a direct SSRF or path-traversal issue, but the wrong-file read and failing Windows checks should be fixed before merge. |
…ecoding Addresses review feedback from @jonluca on this PR. Root cause: fromFileSystemPath percent-escapes a literal "%" character as "%25", so a real filename containing the literal text "%3F" round-trips through encoding as "%253F". toFileSystemPath previously ran decodeURI() BEFORE the manual reserved-char decode pass -- decodeURI decodes "%25" back to a literal "%", which reveals a literal "%3F" substring that did not exist in the encoded form. The manual pass then wrongly decoded that revealed text a second time (%3F -> "?"), so parse("defs%3F1.json") would silently read a "defs?1.json" sibling file instead -- a real path-aliasing bug, and double-decoding a single escape violates RFC 3986 section 2.4. Fix: run the manual reserved-char decode pass BEFORE decodeURI, not after. The literal "%253F" does not contain the substring "%3F" (it's "%25" followed by "3F"), so it's untouched by the manual pass and correctly decoded exactly once by decodeURI (%25 -> %). Also made the reserved-char patterns case-insensitive (RFC 3986 section 2.1: percent-encoding hex digits are case-insensitive, so "%3f" must decode the same as "%3F" -- the previous pattern was uppercase-only). Tests: added a regression test reproducing the exact double-decoding case (confirmed it fails without the fix, passes with it), a lowercase-%3f case, and moved the existing round-trip tests inside the same isWindows()=false mock the neighboring "Handle Linux file paths" block already uses (they were asserting absolute-POSIX-path behavior outside any Windows mock, which is why the Windows CI job was failing). Full suite: 77/77 files, 525 passed, 0 failed, unchanged from before this commit.
|
Thanks for the thorough review — fixed all four points:
Full suite: 77/77 files, 525 passed, 0 failed. Didn't add a full end-to-end |
Problem
Fixes #427.
toFileSystemPathnever decodes%3Fback to?, so any$RefParsercall on a local path containing a literal?(a legal POSIX filename character) fails withResolverError: ENOENT. Confirmed onmainHEADe8190c1.Root cause
lib/util/url.tshas a matched pair of manual encode/decode tables for charactersencodeURI/decodeURIleave alone.urlEncodePatterns(line 13-16) encodes both?and#.urlDecodePatterns(line 19) only reverses#,$,&,,,@—?is missing, even though the comment directly above the decode loop names it explicitly ("This includes characters such as#and?...").Fix
Adds
/%3F/g, "?"tourlDecodePatterns, in the same hex-ordered position the other pairs follow. 1 line changed inlib/util/url.ts.How to test
Reproduced end-to-end too:
$RefParser.parse()on a local file named e.g.defs?1.jsonthrowsENOENTon unpatched code and resolves correctly with the fix.Backward compatibility
No breaking changes. Strictly adds a decode case that was previously a no-op (falling through unchanged); no existing behavior for any other character changes.
Assisted-by: Claude (code generation, reviewed and tested locally)