fix(just): replace Unicode in Windows PowerShell recipe strings#260
Merged
Conversation
Windows PowerShell 5.1 (powershell.exe) reads UTF-8-no-BOM files as the
system ANSI code page (CP1252). When just writes a #!powershell recipe
body to a temp file, multi-byte Unicode codepoints whose UTF-8
continuation byte lands in 0x91-0x94 decode as smart quotation marks
(U+2018-U+201D), which the PowerShell tokenizer treats as string
delimiters. This silently terminates string literals mid-line and
surfaces downstream as "Unexpected token" / "Missing closing {" errors
far from the real cause.
The reported failure mode:
> just bake-day
At C:\Users\rnio\AppData\Local\Temp\just-8Y4WhE\bake-day.ps1:259 char:1
+ } else {
+ ~
Unexpected token "}" in expression or statement.
error: Recipe `bake-day` failed with exit code 1
Line 259 of dev.just contains `} else {`. The actual breakage is
upstream — the em-dash `—` (UTF-8 E2 80 94) on line 227 decodes to
`â€"` under CP1252; the trailing 0x94 byte is U+201D (right double
quote) which terminates the surrounding "..." string early, leaving
PowerShell to mis-parse everything until it gives up at the next `}`.
Three recipes were affected:
- just/dev.just::bake-day (the reported failure)
- just/dev.just::update-tools (same trap, same file)
- just/build.just::sync (em-dash on line 548)
All Unicode in PowerShell string literals replaced with ASCII
equivalents. Unicode inside `#` comments is left intact (the
tokenizer skips comment bodies). Unix recipe variants of the same
recipes are unchanged.
An explanatory comment block has been added to just/shared.just
above the `set windows-shell` directive documenting the trap and
the ASCII-only rule for future contributors.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
Windows PowerShell 5.1 (
powershell.exe) reads UTF-8-no-BOM files as the system ANSI code page (CP1252).justwrites#!powershellrecipe bodies as UTF-8 without a BOM, so any multi-byte Unicode codepoint whose UTF-8 continuation byte lands in0x91-0x94decodes as a smart quotation mark (U+2018-U+201D), which the PowerShell tokenizer treats as a string delimiter. This silently terminates string literals mid-line and surfaces downstream asUnexpected token/Missing closing }errors far from the real cause.Reported failure
Line 259 of
just/dev.justcontains} else {. The actual breakage is upstream: the em-dash—(UTF-8E2 80 94) on line 227 decodes toâ€"under CP1252; the trailing0x94byte is U+201D (right double quote) which terminates the surrounding"..."string early, leaving PowerShell to mis-parse everything until it gives up at the next}.Affected recipes
just/dev.just::bake-day(the reported failure)just/dev.just::update-tools(same trap, same file)just/build.just::sync(em-dash on line 548)Fix
Replaced all Unicode in
[windows]#!powershellstring literals with ASCII tags ([OK],[FAIL],[WARN],[TIP],[DONE],[Rust],[Pkg], etc.). Unicode in#comments was left intact — the tokenizer skips comment bodies. Unix recipe variants are unchanged.Added a comment block above
set windows-shellinjust/shared.justdocumenting:Verification
just --evaluateparses clean on Mac0x91-0x94) in any PowerShell string across alljust/*.justfileslint-ci-windows) passed locallyTest plan
just bake-dayruns to completion on Windows (was the reported failure)just update-toolsparses on Windowsjust syncparses on Windows