Skip to content

fix: filesystem commands name the actual error kind, and the Oils spec parser reads the format - #74

Open
davydog187 wants to merge 11 commits into
mainfrom
fix/issue-70-strerror-and-spec-parser
Open

fix: filesystem commands name the actual error kind, and the Oils spec parser reads the format#74
davydog187 wants to merge 11 commits into
mainfrom
fix/issue-70-strerror-and-spec-parser

Conversation

@davydog187

@davydog187 davydog187 commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Two independent pieces of the #70 roadmap. They touch disjoint files, so a
reviewer who wants them split can take either alone:

  • 13dbe9c, e64d948, 273ac60, 7708142, 51b6d47, 0d0b54d, e1028b6
    item 1's first bullet: the FS.strerror/1 sweep, plus the commands the
    sweep's grep could not reach because they printed nothing
  • bb55214, 7aa07f8 — item 7's prerequisite: the Oils spec parser

Nothing else from the roadmap is here. No Docker oracle, no alphabet matrices,
no filesystem-shape matrix, and the spec suite is not activated in CI.

Review round two is folded in — six per-site conformance fixes, the coverage
hole in the matrix, and the parser's handling of bash-keyed annotations. The
finding-by-finding reply is
here.


Piece A — the strerror sweep

What was broken

33 command modules matched {:error, _}, threw the error kind away, and
spelled every filesystem failure No such file or directory. Only 13 called
FS.strerror/1. stat.ex:50 was the representative case in the issue:

defp accumulate_stat_result({:error, _}, file, _format, {acc_out, acc_err, _acc_has_err, fs}) do
  err = "stat: cannot stat '#{file}': No such file or directory\n"

Repro, with /f a regular file so /f/x is ENOTDIR:

$ stat /f/x
stat: cannot stat '/f/x': No such file or directory     # exists, wrong kind
$ wc /d                                                 # /d is a directory
wc: /d: No such file or directory                       # should be: Is a directory
$ cat /mnt/f                                            # unreadable mount
cat: /mnt/f: No such file or directory                  # should be: Permission denied

GNU coreutils prints the kind, not a guess. This is the issue's stated
prerequisite for the rest of the roadmap: it is estimated at 40–60% of all
divergences an enumerated matrix would report, and it is the same diff a
thousand times.

The fix

Every one of them now renders FS.strerror/1 on the error it was already
handed. The message template is untouched wherever GNU uses one template for
every kind, so the diff is one line per site. Two incidental cleanups fall out:
rm's :enoent clause became identical to the general clause below it and is
gone, and source's catch-all no longer says cannot read for every kind it
did not enumerate.

JustBash.exec_file/2 in lib/just_bash.ex had the same defect and is fixed
with them — it is outside commands/, so it is the one site the issue's grep
did not name.

Where GNU does not use one template for every kind, the template branches:
head, tail, tac, sort and uniq open(2) a directory successfully and
only fail at read(2), so EISDIR gets its own wording.

ghead d   -> ghead: error reading 'd': Is a directory
gtac d    -> gtac: d: read error: Is a directory
gtac nope -> gtac: failed to open 'nope' for reading: No such file or directory
gsort d   -> gsort: read failed: d: Is a directory

Two more per-site fixes: file -b names the reason instead of returning a bare
cannot open, and realpath without -e canonicalises a missing final
component and exits 0 the way GNU does — -e and -m were being parsed and
discarded, and all three modes are real now.

The other half of the family: the commands that printed nothing

Grepping for the literal "No such file or directory" reaches every module
that printed a wrong message and none that printed no message at all. Those
are the worse members of the family, and every one is on #70 item 3's operand
list:

sort /nope    rc=0 silent  ->  rc=2 sort: cannot read: /nope: ...
sort /d       rc=0 silent  ->  rc=2 sort: read failed: /d: Is a directory
cut -f1 /nope rc=0 silent  ->  rc=1 cut: /nope: ...
uniq /nope    rc=0 silent  ->  rc=1 uniq: /nope: ...
uniq /d       rc=0 silent  ->  rc=1 uniq: error reading '/d': Is a directory
grep x /nope  rc=1 silent  ->  rc=2 grep: /nope: ...
cat < /nope   rc=0 silent  ->  rc=1 bash: /nope: ...
wc -l < /nope rc=0 and "0" ->  rc=1 bash: /nope: ..., no stdout

An agent writing sort report.txt | head -5 against a mistyped path got a
clean exit-0 empty result with nothing to distinguish it from an empty file.
wc -l < /nope printing a fabricated 0 is the sharpest one, because the
number looks like an answer.

grep keeps GNU's rule that -q with a line selected exits 0 even after an
error, and cut keeps the files it could read. < is the read side of
025673c: the shell opens the target, so the shell reports the failure and the
command never runs. EISDIR is deliberately excluded there — open(2) on a
directory succeeds, so bash runs the command and the command's own read fails
(cat: stdin: Is a directory), which is not a shell-level diagnostic and has
no seam here.

md5sum was writing its diagnostic to stdout, between checksum lines, so
md5sum a b > sums.txt wrote a malformed checksum line into the file and
2>/dev/null did not silence it; it writes to stderr now, and md5sum -c on
an unreadable checksum file reports instead of exiting 0. file writing to
stdout is correct — real file(1) does the same.

The tests

test/commands/error_message_test.exs is the cross product of the
filesystem-touching commands and the kinds a path can fail with — 188 cases
generated from a table, not written one at a time. Each row is the invocation,
the kinds it covers, the stream the diagnostic belongs on, and its template,
with PATH and MSG filled per kind:

{"head PATH", @stat_kinds, :stderr, "head: cannot open 'PATH' for reading: MSG\n"},
{"head PATH", [:eisdir], :stderr, "head: error reading 'PATH': MSG\n"},

Reading file contents can hit all four of
:enoent/:enotdir/:eisdir/:eacces; a command that only inspects metadata
is crossed with three, because a directory is a perfectly good answer for
stat, find, du and friends. A row carries its own kind list rather than
sitting in a "content reader" or "metadata reader" bucket, which is what lets
one command have two templates.

Operand arity is part of the cross product too. head, tail and wc each
have a single-file arm and a reduce over several files, and sha256sum and
shasum have a third arm behind -c; with every row pinned to one operand,
seven of the sites this sweep exists to fix could be reverted with the suite
still green (proven by mutation: replacing the rendered strerror with a literal
at all seven left 4923 tests, 0 failures). Rows naming two operands, -c FILE,
and a checksum file that lists an unreadable target reach them — the same
mutation now turns 21 tests red.

:eacces is unreachable from the in-memory backend — it has no permission
model, and chmod 000 is ignored on read. test/support/failing_backend.ex
is a VFS.Mountable whose every operation fails with one configured kind,
mounted at /mnt. That is also the only route to :erofs/:eio/:enotsup
when the roadmap gets to them.

82 of the first 126 failed on main:

     left:  "find: /f/x: No such file or directory\n"
     right: "find: /f/x: Not a directory\n"
     left:  "comm: /mnt/f: No such file or directory\n"
     right: "comm: /mnt/f: Permission denied\n"
     left:  "tail: cannot open '/mnt/f' for reading: No such file or directory\n"
     right: "tail: cannot open '/mnt/f' for reading: Permission denied\n"
     left:  "wc: /f/x: No such file or directory\n"
     right: "wc: /f/x: Not a directory\n"

126 tests, 82 failures

Piece B — lib/just_bash/spec_test/parser.ex

What was broken

Nothing calls the parser, so nothing caught any of it.

The four the issue lists

  1. :skip is never assigned. runner.ex:48's guard was dead code.
    Compounding it, Oils writes ## SKIP above the script and
    collect_script/2 stopped at the first ## line — so a SKIP-marked case
    was parsed with an empty script and passed vacuously. All 247 of them:

    #### Command Sub trailing newline removed
    ## SKIP (unimplementable): python2 not available
    s=$(python2 -c 'print("ab\ncd\n")')
    argv.py "$s"
    ## stdout: ['ab\ncd']
    

    parsed as script: "", skip_reason: nil.

  2. String.trim_leading(line, "#### ") removes every leading occurrence
    of its argument, not one. #### #### nested yields nested, not
    #### nested. No corpus case is spelled that way today, which is why it
    went unnoticed — so this one defect is tested on synthetic input while the
    others are tested on real files.

  3. String.trim(script) trims whitespace characters. The format means
    to drop the blank lines that frame a case; trimming characters also rewrites
    the last command of every case whose script ends in a space.

    The corpus pins down the correct rule. builtin-trap-err.test.sh:

    #### trap can use original $LINENO
    
    trap 'echo line=$LINENO' ERR
    
    false
    false
    echo ok
    ## STDOUT:
    line=3
    line=4
    

    line=3 only holds if the leading blank line is dropped and the interior
    one is kept. So the parser now trims at line granularity: whole blank lines
    at either end go, everything between the first and last non-blank line is
    preserved byte for byte.

  4. ## STDERR is never read. 65 of the 2728 cases carry a stderr
    expectation, and stderr is where the silently-ignored-flag class lives.

Two more, from review

  1. Every ## OK bash … / ## BUG bash … / ## N-I bash … annotation was
    dropped
    as "another shell's expectation". It is the opposite: the
    unannotated default records osh, and an annotation naming a shell is what
    that shell actually does. append.test.sh "Try to append list to
    element" records ## stdout-json: "" / ## status: 2 and then
    ## OK bash status: 0 with ## OK bash STDOUT:['1', '2 3']; the
    parser produced the osh expectation, so a JustBash that reproduced bash
    exactly would have been scored a failure. 440 such lines cover 323 of the
    2728 cases across 89 files
    — including two ## OK just-bash markers
    written by this repo.

  2. A bare ## stdout: — the format's spelling of one expected empty line —
    was not recognised, because the clause required the separating space. Four
    cases parsed to expected_stdout: nil, and Runner.check_output(nil, _)
    passes a case whatever it printed. sh-func.test.sh "Locals don't leak"
    would have been scored passed if the local had leaked.

The fix

A case body is now sorted line by line into script or directive, in either
order, rather than assuming directives only follow the script. A multiline
block keyed to another shell (## N-I dash STDOUT:## END) is consumed
along with its body, so lines of expected output no longer fall through into
the script. TestCase gains expected_stderr; Runner reports a skipped case
as skipped instead of as an error, keeps skips out of the pass rate, and
compares stderr when the case names one.

Annotations are parsed into {qualifier, shells, key: value} and applied per
key
, weakest first: unannotated default → bashjust-bash. The format
overrides one key at a time, which is why append.test.sh keeps its default
stdout-json until ## OK bash STDOUT: replaces it while ## OK bash status: 0 replaces the status on its own line. bash-2 is bash 2.x and stays
foreign.

The tests

test/just_bash/spec_test_parser_test.exs, against the real files in
test/command_spec_cases/bash/cases/. The corpus-wide counts independently
reproduce three numbers the issue states from the other side — 2728 cases,
247 SKIP markers, 65 stderr expectations:

assert markers == 247
assert Enum.count(all_cases(), & &1.skip_reason) == markers
assert Enum.count(all_cases(), & &1.expected_stderr) == 65
assert length(all_cases()) == 2728

Those four are blind to a directive spelling the parser silently stops
recognising, so three more are pinned — 2562 cases with a stdout
expectation, 239 with a non-zero status, and only 59 that assert
nothing at all — and an independent reader re-derives the annotations straight
from the files without going through the parser: 440 bash-keyed lines, 323
cases, 89 files, and every one of the 124 cases carrying a bash-keyed
status: N must parse to that status.

Plus: no script contains a ## line; the $LINENO case above parses to the
script its expectation assumes; Runner does not execute a skipped case and
does not count it as a failure.

13 of the 32 fail with the old parser (git stash push -- parser.ex):

  1) test a bare directive value Runner fails a case whose bare stdout expectation diverges
  9) test a shell annotation naming bash every bash-keyed status annotation in the corpus is the case's status
 12) test a shell annotation naming bash just-bash wins over bash
 13) test a shell annotation naming bash a multiline STDOUT block overrides the unannotated expectation

32 tests, 13 failures

The suite is still inert. Activating it — the curated subset, the re-recording
and the ratchet — is the rest of item 7.


Gates

$ mix compile --warnings-as-errors --force
Compiling 168 files (.ex)
Generated just_bash app

$ mix format --check-formatted
formatted

$ mix credo --strict
  Refactoring opportunities
| [F] Avoid `apply/2` and `apply/3` when the number of arguments is known.
|     test/support/banned_fixture_apply.ex:4:16 #(BannedCallTracer.Fixture.Apply.run)

Analysis took 1.5 seconds (0.09s to load, 1.4s running 68 checks on 232 files)
5183 mods/funs, found 1 refactoring opportunity.

$ mix test
Finished in 28.0 seconds (27.7s async, 0.2s sync)
2 doctests, 62 properties, 4988 tests, 0 failures (5 excluded)

$ mix dialyzer
Total errors: 13, Skipped: 13, Unnecessary Skips: 0
done (passed successfully)

The one credo finding is the pre-existing intentional test fixture, and the 13
dialyzer errors are the pre-existing entries in .dialyzer_ignore.exs. 4878
tests before, 4988 after: +110.

Refs #70 (items 1 and the item-7 parser prerequisite; the rest of the roadmap stays open)

@davydog187 davydog187 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two lenses were applied to this PR: a conformance lens (does each changed site now match real GNU/bash on this machine — ghead, gtail, gsort, grealpath, gmd5sum, file, bash -c) and a test-strength lens (can the new tests actually detect a regression at each site the diff touched). 13 candidate findings went in; 7 survived verification, 6 were merged or refuted. Everything below was reproduced on bb55214 in a scratch worktree.

The two commits are good and the sweep is the right shape. Nothing here is a blocker — the two major items are a coverage hole and a moduledoc claim that is wrong about the corpus format, and the rest are per-site conformance nits that the new matrix currently pins rather than catches.


1. The parser discards the corpus's bash-keyed expectations, so 323 cases carry another shell's ground truth

majorlib/just_bash/spec_test/parser.ex:145 (and the nil branch at :149)

The moduledoc added by this PR says ## N-I <shells>, ## BUG <shells>, ## OK <shells> are "the expectation for a shell that is not bash. Bash is the oracle here, so these are dropped." In the Oils format that is exactly backwards for the annotations that name bash: ## OK bash status: 0 is bash's recorded status, overriding the unannotated default (which records osh/dash). block_opener/1 classifies ## OK bash STDOUT: as :other_shell and collect_block/2 throws away the body; the single-line forms (## OK bash status: 0) fall through the generic ## clause at :149 and are dropped too.

Measured on the corpus: 440 annotation lines name bash or just-bash, across 89 files, covering 323 of the 2728 cases (~12%). Two of them (loop.test.sh:288-291, var-op-bash.test.sh:489) are ## OK just-bash markers written by this repo and are now silently discarded.

Failure scenario, reproduced:

$ mix run -e 'JustBash.SpecTest.Parser.parse(File.read!("test/command_spec_cases/bash/cases/append.test.sh")) |> ...'
%TestCase{name: "Try to append list to element", expected_stdout: "", expected_status: 2}

The file (append.test.sh:134-148) records ## stdout-json: "" / ## status: 2 for osh, then ## OK bash status: 0 and ## OK bash STDOUT:['1', '2 3']. A JustBash that reproduces bash exactly is scored as a failure by Runner.run_test_case/2. Same for loop.test.sh:273 "too many args to continue", which parses to expected_stdout: "" / status: 2 while the file records ## BUG bash STDOUT: a\n-- with ## BUG bash status: 0.

This PR is the stated prerequisite for item 7's ratchet. Building the baseline on ~12% of cases whose expectation is knowably not bash's is the opposite of a ratchet.

Fix: treat an annotation whose shell list contains bash (or just-bash) as an override of the case's default expectation rather than as foreign — parse the shell list out of ## (OK|BUG|N-I|BUG-\d+) <shells> <key>: and apply it when bash ∈ shells. Correct the moduledoc paragraph either way; as written it asserts something false about the format.


2. The strerror matrix pins operand arity to 1 — five changed sites are untestable-by-construction

majortest/commands/error_message_test.exs:97

Every row of @content_readers/@metadata_readers passes exactly one path operand (comm/diff pass two copies of the same one). But head, tail and wc each have two independent error sites — a single-file arm and a reduce over multiple files — and the diff changed both. sha256sum.ex and shasum.ex each have three (hash_files, check_checksums, verify_single_checksum); only hash_files is reachable from the matrix.

Proven by mutation. I replaced #{FS.strerror(error)} with the literal MUTANT at five sites — head.ex:48, tail.ex:48, wc.ex:66, sha256sum.ex:114, shasum.ex:121 — and ran the full suite:

Finished in 24.7 seconds
2 doctests, 62 properties, 4894 tests, 0 failures (5 excluded)

Five of the sites this PR exists to fix can be fully undone without turning the suite red. This is the exact anti-pattern #70 is about: the matrix looks like a cross product, but one axis (operand arity) is pinned to a single value and everything behind the other value is invisible.

Fix: add a second operand to the head/tail/wc rows ("head PATH /f" etc., asserting the stderr line while stdout carries /f), plus one sha256sum -c PATH row and one shasum -c PATH row.


3. md5sum writes its diagnostic to stdout, corrupting the checksum stream

majorlib/just_bash/commands/md5sum.ex:64

The diff rewrote this exact line to render FS.strerror/1 but left it appending to acc_out; compute_hashes/3 then returns stderr: "" unconditionally. sha256sum.ex and shasum.ex — the same shape, touched by the same sweep — already do {out, err <> …}.

$ md5sum /f /nope
  rc=1
  out="764efa883dda1e11db47671c4a3bbd9e  /f\nmd5sum: /nope: No such file or directory\n"
  err=""

$ gmd5sum nope                      # coreutils 9.x
  rc=1  out=[]  err=[gmd5sum: nope: No such file or directory]

So md5sum /f /nope > sums.txt writes the diagnostic into sums.txt as a malformed checksum line, and md5sum /nope 2>/dev/null still prints it. test/commands/error_message_test.exs:46 records the divergence ({"md5sum PATH", :stdout, …}) with a comment calling it "a separate bug from the message text" — but it is the same line the sweep touched, and recording it pins it for whoever runs the enumerated matrix later. (file.ex:51 has the same shape and is genuinely correct: real file does print to stdout — verified.)

Fix: {acc_out, acc_err <> "md5sum: …"} and thread acc_err into the returned stderr, then flip the matrix row to :stderr.


4. file -b still hardcodes "cannot open" and drops the reason the diff just threaded in

minorlib/just_bash/commands/file.ex:70

format_error_line/2 became /3 and renders FS.strerror(error) in the non-brief branch, but the brief branch two lines above still returns the constant "cannot open\n". -b suppresses the filename prefix, not the errno text.

$ file -b /f/x    -> "cannot open\n"                              (indistinguishable from ENOENT)
$ file -b /nope   -> "cannot open\n"
$ file /f/x       -> "/f/x: cannot open (Not a directory)\n"      (correct)

$ file -b f/x     # real file(1)
cannot open `f/x' (Not a directory)

The matrix only exercises file PATH, never file -b PATH, so the half-fixed function is uncovered.

Fix: "cannot open (#{FS.strerror(error)})\n" in the brief branch, and add a file -b PATH row.


5. head/tail use the open-failure template for EISDIR; GNU uses a different one

minorlib/just_bash/commands/head.ex:48, :68, lib/just_bash/commands/tail.ex:48, :68

GNU head/tail open(2) a directory successfully and only fail at read(2), so EISDIR gets a different template from ENOENT/ENOTDIR/EACCES. Verified against coreutils 9.x on this machine:

ghead nope -> ghead: cannot open 'nope' for reading: No such file or directory
ghead f/x  -> ghead: cannot open 'f/x' for reading: Not a directory
ghead d    -> ghead: error reading 'd': Is a directory        <-- different template
gtail d    -> gtail: error reading 'd': Is a directory

just_bash emits head: cannot open '/d' for reading: Is a directory. The PR's premise is that the kind must be named exactly; here the kind also selects the template, and the sweep's "templates untouched" rule mis-renders it. test/commands/error_message_test.exs:24-25 now asserts the wrong string for the :eisdir row, so this becomes a test to change rather than a test to add.

Same shape, lower priority: tac /nope is GNU tac: failed to open '/nope' for reading: … (just_bash: tac: /nope: …) and tac /d is GNU tac: /d: read error: Is a directory.

Fix: branch on :eisdir in the head/tail templates, or leave the code and add a known_gap-style comment on the matrix row so it doesn't read as ground truth.


6. realpath errors on a missing final component where GNU exits 0 — and the matrix pins it

minorlib/just_bash/commands/realpath.ex:34, test/commands/error_message_test.exs:55

GNU realpath without -e requires only the parent components to exist; a missing last component is canonicalised and printed with exit 0. just_bash FS.stats the full path and errors.

$ realpath /nope        # just_bash
  rc=1  err="realpath: /nope: No such file or directory\n"

$ grealpath nope        # coreutils 9.x
  rc=0  out=/private/tmp/gnuprobe/nope

$ grealpath nope/deep/x # intermediate missing — GNU does error here
  rc=1  err=grealpath: nope/deep/x: No such file or directory

The sweep changed this line and the matrix now asserts realpath: /nope: No such file or directory with a non-zero exit for the :enoent row — recording the wrong ground truth for the one kind where realpath is not supposed to fail at all. The :enotdir and :eacces rows are correct as written.

Fix: either stat the parent instead of the full path (and gate the full-path stat behind -e), or drop the :enoent row for realpath with a comment saying why.


7. Bare ## stdout: is not recognised, so 4 corpus cases assert nothing

minorlib/just_bash/spec_test/parser.ex:116

The clause is collect([{"## stdout: " <> value, _} | …]) — it requires the space after the colon. The Oils format spells an expected single empty line as a bare ## stdout: (the value is empty, so no separating space is written; upstream's KEY_VALUE_RE uses :\s*(.*)). Those lines fall through to the generic ## clause, block_opener/1 returns nil, and the directive is dropped. expected_stdout stays nil and Runner.check_output(nil, _) returns true, so the case passes whatever it printed. The same hole applies to a bare ## stderr:.

Four cases in the corpus are spelled this way; all four parse to expected_stdout: nil:

sh-func.test.sh:10          "Locals don't leak"
var-op-strip.test.sh:34     "Remove const suffix from undefined"
redirect-command.test.sh:141 "Redirect in command sub"
assign.test.sh:134          "Empty env binding"

sh-func.test.sh "Locals don't leak" (f() { local f_var=f_var; }; f; echo $f_var) would still be scored passed: true if the local did leak and stdout were "f_var\n".

The new test/just_bash/spec_test_parser_test.exs pins five corpus-wide counts (2728 / 136 / 247 / 65 / 11), none of which constrains expected_stdout or expected_status — deleting the "## stdout: " <> value clause outright leaves all five holding and the file green. Measured on this branch: 173 cases end with expected_stdout == nil and 67 assert literally nothing (nil stdout, nil stderr, status 0, not skipped).

Fix: match "## stdout:" <> value / "## stderr:" <> value and strip one leading space. Then add a sixth corpus assertion on the assert-nothing count — it would have caught this, and it will fail loudly on any future directive spelling the parser stops recognising (including finding #1).


8. The sweep was grep-driven, so it missed the commands that print no message at all

major, follow-up scope — lib/just_bash/commands/sort.ex:49, cut.ex:90, uniq.ex:32, grep.ex:157; also lib/just_bash/interpreter/executor/redirection.ex:369

Not a defect introduced here — none of these lines are in the diff — but it is a methodology gap worth naming, because it decides whether #70 item 1's family is closed. The sweep was driven by grepping for the literal "No such file or directory", so it reached every module that printed a wrong message and none of the modules that print nothing. Those are the worse members of the family, and they are not in the new 126-cell matrix. All five are named in #70 item 3's own operand list.

Reproduced on this branch with /f a regular file and /d a directory:

sort /nope      -> rc=0 out="" err=""    | gsort: cannot read: nope: No such file or directory, rc=2
sort /d         -> rc=0 out="" err=""    | gsort: read failed: d: Is a directory, rc=2
cut -f1 /nope   -> rc=0 out="" err=""    | gcut: nope: No such file or directory, rc=1
uniq /nope      -> rc=0 out="" err=""    | guniq: nope: No such file or directory, rc=1
grep x /nope    -> rc=1 out="" err=""    | grep: nope: No such file or directory, rc=2
cat < /nope     -> rc=0 out="" err=""    | bash: nope: No such file or directory, rc=1
wc -l < /nope   -> rc=0 out="0\n"        | bash: nope: No such file or directory, rc=1, no stdout

wc -l < /nope printing a fabricated 0 is the sharpest one. 025673c ("a command whose redirect fails does not run at all") only covered the write side — the docstring at redirection.ex:62 explicitly excludes < from preflight/2 as a redirect that "touches no file", and extract_stdin_content/2 swallows the read error to "".

An agent writing sort report.txt | head -5 against a mistyped path gets a clean exit-0 empty result with nothing to distinguish it from an empty file.

Fix (follow-up PR is fine): thread the error out of get_content/3 in those four commands, and add them as four rows to @content_readers — that is the whole test-side surface. < wants its own treatment: report from the shell with the shell's prefix and skip the command, matching the write-side behaviour 025673c established.


Considered and dismissed (6)

"Unreachable collect_block/2 clause for a case header" (parser.ex:176) — refuted. The claim was that collect_block([{"## " <> _, _} | _], acc) at :174 shadows the "#### " <> _ clause at :176 because "#### foo" starts with "## ". It does not: the prefix is #,#,space, and "#### foo" has # in the third position. Verified — match?("## " <> _, "#### foo") is false. The secondary argument (that extract_test_cases/2 already splits bodies at case_header?/1, so a body can never contain a #### line) is true, which makes the clause defensive rather than dead-and-shadowed. No traced cost; leaving it is fine.

"Corpus-count assertions cannot detect a regression that turns expectations into nil" (spec_test_parser_test.exs:1402) — merged into finding #7. The measurement is correct (173 nil-stdout, 67 assert-nothing, all five pinned counts survive deleting the ## stdout: clause), but on its own it is a suggestion rather than a defect. It is the right fix for #7, so it lives there.

"Parser discards bash-specific expectations" reported twice — the two lenses found the same thing; merged into finding #1, keeping the just-bash-marker detail from one and the append.test.sh repro from the other.

"Bare ## stdout:" reported twice — merged into finding #7.

file.ex:51 printing the diagnostic to stdout — not a bug. Real file(1) writes nope: cannot open ... to stdout and even exits 0; verified locally. Only md5sum (finding #3) is on the wrong stream.

"wc /d /f prints No such file or directory" — the failure scenario attached to finding #2 was wrong; that describes the reverted state. On this branch wc /d /f correctly prints wc: /d: Is a directory. The finding survives on its coverage argument, which the mutation test proves independently.

@davydog187

Copy link
Copy Markdown
Collaborator Author

All eight findings are addressed. Nothing disputed — every repro reproduced first, went red as a test, and now matches the oracle.

# Finding Resolution Commit
1 Parser discards bash-keyed ## OK/BUG/N-I bash annotations (323 cases) fixed 7aa07f8
8 Bare ## stdout: not recognised, 4 cases assert nothing fixed 7aa07f8
3 md5sum writes its diagnostic to stdout fixed e64d948
5 head/tail use the open-failure template for EISDIR (+ tac) fixed 273ac60
7 file -b hardcodes cannot open fixed 7708142
6 realpath errors on a missing final component fixed 51b6d47
2 Matrix pins operand arity to 1 — five sites untestable fixed 0d0b54d
4 sort/cut/uniq/grep/< file swallow the error entirely fixed e1028b6

1 + 8 — the spec parser (7aa07f8)

Annotations are now parsed into {qualifier, shells, key: value} and applied per key, weakest first: unannotated default → bashjust-bash. append.test.sh "Try to append list to element" parses to ['1', '2 3'] / status 0 instead of "" / status 2; loop.test.sh:273 takes this repo's own ## OK just-bash marker over ## BUG bash. bash-2 is bash 2.x and stays foreign. The moduledoc paragraph that asserted the opposite is rewritten.

Bare ## stdout: / ## stderr: — the format's spelling of one expected empty line — is read; sh-func.test.sh "Locals don't leak" would previously have been scored passed if the local had leaked.

Test-strength, as you asked for:

  • an independent reader splits the 136 files on their #### headers and re-derives the annotations without touching the parser: 440 bash-keyed lines over 323 cases in 89 files, and every one of the 124 cases with a bash-keyed status: N must parse to that status.
  • the corpus counts now include the ones that move: 2562 cases with a stdout expectation, 239 with a non-zero status, and 59 — not 67 — that assert nothing at all.
  • 13 of the 32 tests in the file fail against the pre-fix parser (git stash push -- parser.ex).

2 — operand arity (0d0b54d)

Reproduced your mutation exactly. Replacing the rendered strerror with MUTANT at wc.ex:66, sha256sum.ex:73, sha256sum.ex:114, shasum.ex:87, shasum.ex:121 and the head/tail reduce arms left the suite at 4923 tests, 0 failures.

Rows naming two operands (head PATH /f, wc PATH /f, cut -f1 PATH /f, grep hi PATH /f), -c FILE, and a checksum file that lists an unreadable target now reach all seven. Same mutation: 21 failures.

The matrix rows carry the kinds they cover ({script, kinds, stream, template}), which is what lets one command have two templates; the content-reader/metadata-reader split it replaces was only ever a way to spell "these kinds".

3 — md5sum (e64d948)

{acc_out, acc_err <> …} and stderr threaded through. md5sum -c FILE on a checksum file it cannot read was silently exit 0 as well; it now reports and exits 1, and an unreadable checksum file does not feed the WARNING: N computed checksums did NOT match line. The matrix row is :stderr, and file_info_test.exs:484 (which asserted the stdout spelling) was updated.

5 — EISDIR templates (273ac60)

Verified against coreutils 9.x here:

ghead d -> ghead: error reading 'd': Is a directory
gtac d  -> gtac: d: read error: Is a directory
gtac nope -> gtac: failed to open 'nope' for reading: No such file or directory

tac was on the wrong template for every kind, not just EISDIR, so all four of its rows changed.

6 — realpath (51b6d47)

The three modes are real now: default requires the parent to be a directory (so /f/x is still ENOTDIR even though /f stats cleanly), -e requires the whole path, -m requires nothing. -e and -m were previously parsed and discarded. The :enoent row moved to realpath -e PATH, and there are explicit tests for realpath /nope → rc 0 and realpath /nope/deep/x → rc 1.

7 — file -b (7708142)

format_error_line/3's brief branch renders the reason; file -b PATH is a matrix row.

4 — the silent commands (e1028b6)

Taken as the deliberate scope extension you flagged.

sort /nope    rc=0 silent  ->  rc=2 sort: cannot read: /nope: ...
sort /d       rc=0 silent  ->  rc=2 sort: read failed: /d: Is a directory
cut -f1 /nope rc=0 silent  ->  rc=1 cut: /nope: ...
uniq /nope    rc=0 silent  ->  rc=1 uniq: /nope: ...
uniq /d       rc=0 silent  ->  rc=1 uniq: error reading '/d': Is a directory
grep x /nope  rc=1 silent  ->  rc=2 grep: /nope: ...
cat < /nope   rc=0 silent  ->  rc=1 bash: /nope: ...
wc -l < /nope rc=0 and "0" ->  rc=1 bash: /nope: ..., no stdout

grep keeps GNU's rule that -q with a line selected exits 0 even after an error, and cut keeps the files it could read. < is the read side of 025673c: extract_heredoc_stdin/2 returns {:ok, stdin, rest} | {:error, result} and a new with_stdin/3 in the executor short-circuits all four call sites, so the body never runs.

One deliberate omission, so it is not mistaken for an oversight: :eisdir is absent from the < rows. open(2) on a directory succeeds, so bash runs the command and the command's own read fails — cat: stdin: Is a directory, not a shell-level diagnostic. There is no seam for that here, so cat < /d keeps its current behaviour with a comment saying why, rather than pinning a message bash never prints.

sort.ex is touched only on the read-error path, to keep the conflict with #72 narrow.


Also noticed, not fixed (out of scope)

  • A pipeline drops every stage's stderr but the last. Pre-existing and unrelated to <: ls /nope | wc -l is rc=0 out="0\n" err="" on main too. So is cat < /nope | wc -l after this change. Worth its own issue.
  • file exits 1 and prints to stdout where real file(1) exits 0, and spells the message cannot open `f/x' (Not a directory) with the name inside the parenthetical form even under -b. Left as the review had it.

Gates

$ mix compile --warnings-as-errors --force
Compiling 168 files (.ex)
Generated just_bash app

$ mix format --check-formatted
formatted

$ mix credo --strict
| [F] Avoid `apply/2` and `apply/3` when the number of arguments is known.
|     test/support/banned_fixture_apply.ex:4:16 #(BannedCallTracer.Fixture.Apply.run)
5183 mods/funs, found 1 refactoring opportunity.

$ mix test
Finished in 28.0 seconds (27.7s async, 0.2s sync)
2 doctests, 62 properties, 4988 tests, 0 failures (5 excluded)

$ mix dialyzer
Total errors: 13, Skipped: 13, Unnecessary Skips: 0
done (passed successfully)

The one credo finding and the 13 dialyzer errors are the pre-existing entries. 4894 tests before this round, 4988 after: +94.

@davydog187

Copy link
Copy Markdown
Collaborator Author

Verification of review fixes

Independent re-run of all 8 findings against e1028b6 (detached checkout, fresh deps/_build).
Every repro below was executed on this head; oracle output is from coreutils 9.x (g*),
file(1) and bash on darwin.

Per-finding

# Finding Verdict Evidence
1 Spec parser discards bash-keyed annotations fixed append.test.sh "Try to append list to element" now parses to stdout="['1', '2 3']\n" status=0 (was "" / 2). loop.test.sh:273 "too many args to continue"stdout="a\n" status=1, i.e. ## OK just-bash beats ## BUG bash beats the default. loop.test.sh:261 "bad arg to break"stdout="hi\n" status=128: default stdout kept, status taken from ## OK bash status: 128 — per-key precedence works. Census re-derived from the raw files without the parser: 440 annotation lines naming bash/just-bash across 89 files, 2728 cases — the finding's numbers. Mutation: reverting directive/1 to drop every annotation ([_line, _shells, keyed] -> if block_key(keyed), do: :drop_block, else: :ignore) turns 7 of 32 parser tests red, including every bash-keyed status annotation in the corpus is the case's status and bash-2 is a different shell from bash. Moduledoc rewritten.
2 strerror matrix pins operand arity to 1 fixed Applied the named mutation myself at wc.ex:66, sha256sum.ex:73, sha256sum.ex:114, shasum.ex:87, shasum.ex:121 (interpolation → literal MUTANT) and at head.ex:48 / tail.ex:48 (read_error(file, error)"MUTANT"). Result: 4988 tests, 28 failures. All seven sites are individually reached — the red rows are wc PATH /f, head PATH /f, tail PATH /f, sha256sum -c PATH, shasum -c PATH, echo '0 PATH' > /sums; sha256sum -c /sums, echo '0 PATH' > /sums; shasum -c /sums, each ×4 kinds. Reverted; back to 0 failures.
3 md5sum diagnostic on stdout fixed md5sum /f /noperc=1 out="764efa883dda1e11db47671c4a3bbd9e /f\n" err="md5sum: /nope: No such file or directory\n" — the same split as gmd5sum f nope. md5sum /nope 2>/dev/nullrc=1 out="" err="". md5sum -c /noperc=1, message on stderr, stdout clean. md5sum /f /nope > /out leaves /out holding only the checksum line.
4 sort/cut/uniq/grep and < file swallow the error fixed (one residual gap, below) sort /noperc=2 "sort: cannot read: /nope: No such file or directory"gsort nope. sort /drc=2 "sort: read failed: /d: Is a directory"gsort d. cut -f1 /noperc=1; cut -f1 /f /noperc=1 out="hi\n"gcut. uniq /noperc=1; uniq /drc=1 "uniq: error reading '/d': Is a directory"guniq d. grep x /noperc=2; grep -q hi /f /noperc=0 with the diagnostic and grep -q zz /f /noperc=2, matching GNU exactly. cat < /nope and wc -l < /noperc=1 "bash: /nope: No such file or directory" with no fabricated 0 ≡ bash. Heredocs, here-strings, while read … < /two and for … < file all still behave.
5 head/tail EISDIR template (plus tac) fixed head /dhead: error reading '/d': Is a directoryghead d; tail /d likewise ≡ gtail d. head /nope still head: cannot open '/nope' for reading: …ghead. tac /nopetac: failed to open '/nope' for reading: …gtac nope; tac /dtac: /d: read error: Is a directorygtac d.
6 realpath on a missing final component fixed realpath /noperc=0 "/nope\n"grealpath nope (rc=0, canonicalised). realpath /nope/deep/xrc=1 "realpath: /nope/deep/x: No such file or directory"grealpath. realpath /f/xrc=1 "realpath: /f/x: Not a directory"grealpath f/x; realpath /f/x/y likewise. -m accepts everything (realpath -m /f/x/y → rc=0 /f/x/y), -e still fails (realpath -e /nope → rc=1, realpath -e /d/nope → rc=1). Both flags were previously eaten by the catch-all.
7 file -b hardcodes "cannot open" fixed file -b /f/x"cannot open (Not a directory)\n", file -b /nope"cannot open (No such file or directory)\n" — now distinguishable, reason threaded through. file /f/x and file -b /f unchanged. The remaining divergence from real file(1) (filename inside the parens, exit 0) is the one the review scoped out. file -b PATH matrix row present over all three metadata kinds.
8 Bare ## stdout: not recognised fixed All four named cases now parse to "\n" instead of nil: sh-func.test.sh "Locals don't leak", var-op-strip.test.sh "Remove const suffix from undefined", redirect-command.test.sh "Redirect in command sub", assign.test.sh "Empty env binding". Corpus: 166 nil-stdout (was 173), 59 assert-nothing (was 67), 2562 with a stdout expectation, 239 non-zero status. Mutation: restoring the space-requiring clause ("stdout: " <> value) turns 7 of 32 parser tests red, including the count test 2562 cases carry a stdout expectation and only 59 assert nothing — the strengthened counts do their job.

Gates (re-run here, not taken on report)

mix compile --warnings-as-errors   Compiling 168 files (.ex) / Generated just_bash app  — clean
mix format --check-formatted       exit 0
mix credo --strict                 5183 mods/funs, found 1 refactoring opportunity
                                   (test/support/banned_fixture_apply.ex:4 — the intentional
                                    tracer fixture; present unchanged on origin/main)
mix test                           2 doctests, 62 properties, 4988 tests, 0 failures (5 excluded)
mix dialyzer                       Total errors: 13, Skipped: 13, Unnecessary Skips: 0 — passed

New problems introduced by these commits

Both come from e1028b6 making < file fail loudly. Each is reproduced against bb55214
(this PR's pre-fix head) to confirm the commit caused it.

1. < /dev/null now fails — blocker

/dev/null is special-cased only on the write side (redirection.ex:187-193,
classify_redirection/3). The read side goes through the VFS, where /dev/null does not exist.
extract_stdin_content/2 used to swallow that and return "", which was accidentally right.

                     e1028b6                                        bb55214   bash
cat < /dev/null      rc=1 "bash: /dev/null: No such file or directory"  rc=0 ""   rc=0 ""
read x < /dev/null   rc=1 "bash: /dev/null: No such file or directory"  rc=1 ""   rc=1 ""

cmd < /dev/null is a standard idiom; it now aborts the command and returns 1 with a diagnostic.
The :eisdir carve-out in extract_stdin_content/2 needs a /dev/null sibling.

2. sort - / uniq - / grep pat - now hard-error — major

GNU reads - as stdin. just_bash resolves it as a path; before the commit that produced a silent
empty exit-0 result, now it produces a diagnostic and a non-zero exit, so a pipeline under set -e
or pipefail that used to (wrongly) pass now dies.

                     e1028b6                                    bb55214    GNU
echo x | sort -      rc=2 "sort: cannot read: -: No such file…"  rc=0 ""    rc=0 "x"
echo x | uniq -      rc=1 "uniq: -: No such file…"               rc=0 ""    rc=0 "x"
echo x | grep x -    rc=2 "grep: -: No such file…"               rc=1 ""    rc=0 "x"

Both directions are wrong, but the exit code moved 0→2/1/2 and the change is this PR's. The
read-error path should recognise the - operand before it reaches the VFS.

Residual gap — pre-existing, not caused by this PR

sort /f /noperc=0 out="hi\n", silent; GNU is rc=2 "gsort: cannot read: nope: …".
just_bash's sort ignores every operand after the first (sort /f /g prints only hi), so its
multi-operand read error is unreachable. Finding 2 established operand arity as an axis of the
matrix; sort PATH /f is the row that would have caught this and it is absent. Follow-up, not a
blocker on these fixes.

@davydog187

Copy link
Copy Markdown
Collaborator Author

Both regressions from the fix round are closed — d59d84d

The verifier was right on both, and both come from the same place: e1028b6 taught the read paths to report the read they could not do, and it now reports on operands that are not paths.

Regression 1 (BLOCKER) — < /dev/null

/dev/null was special-cased only on the write side (classify_redirection/3). The read side went through the VFS, where it does not exist, and extract_stdin_content/2 no longer swallowed that.

                                   at e1028b6                                          now      bash
cat < /dev/null; echo rc=$?        rc=1  bash: /dev/null: No such file or directory     rc=0 ""  rc=0 ""
read x < /dev/null; echo rc=$?     rc=1  bash: /dev/null: No such file or directory     rc=1 ""  rc=1 ""
wc -l < /dev/null                  rc=1  bash: /dev/null: No such file or directory     0        0

Both directions now share one @null_device in redirection.ex, so > /dev/null discarding and < /dev/null reading empty are the same fact stated once. I checked the whole special-file set as asked: /dev/null is the only path the write side services itself, so it is the only one the read side needed. cat < /nope/dev/null still reports, and cd /; cat < dev/null still resolves as a path on both sides — the two directions agree, which is the property that was missing.

Regression 2 (MAJOR) — sort - / uniq - / grep pat -

                    at bb55214   at e1028b6                                       now        GNU
echo x | sort -     rc=0 ""      rc=2  sort: cannot read: -: No such file...      rc=0 "x"   rc=0 "x"
echo x | uniq -     rc=0 ""      rc=1  uniq: -: No such file...                   rc=0 "x"   rc=0 "x"
echo x | grep x -   rc=1 ""      rc=2  grep: -: No such file...                   rc=0 "x"   rc=0 "x"

Implemented as reading stdin, not as suppressing the diagnostic — rc=0 with no output is the silent-success class this issue exists to remove. set -e; printf 'b\na\n' | sort -; echo done now prints a b done and exits 0.

The audit

Driven off Registry.list() rather than by hand, and it found the same gap in nineteen more commands. Sixteen were already broken before the sweep; they were just quieter about it, which is exactly why grepping for No such file or directory did not reach them:

before now
cut -c1 - cut: invalid option '-' reads stdin
head - / tail - cannot open '-' for reading reads stdin
wc -l - wc: -: No such file... 1 -
nl - fold - expand - base64 - md5sum - comm - diff - invalid option reads stdin
od -c - xxd - unknown option: - reads stdin
sed -n p - awk '{print}' - jq . - -: No such file... reads stdin
cat - tac - rev - paste - sha256sum - shasum - file - already correct unchanged

Ten of them never reached a read path at all: their arg parsers called a bare - an invalid option, which POSIX says it never is. Those get one clause each; the rest go through a new JustBash.Commands.StdinOperand.read/4 that mirrors FS.read_file/2, so each call site keeps the error arm it already had.

Per-operand labels follow GNU, which does not agree with itself:

echo x | grep x - /f   ->  (standard input):x
                           /f:x
echo x | head - /f     ->  ==> standard input <==
                           ==> /f <==
echo x | wc -l - /f    ->        1 -
                                 1 /f

One real bug fell out of the audit: base64 - /f silently dropped stdin (read_single_file/4 matched - and returned the accumulator untouched). It now concatenates — eAo= became eAp4Cg==.

Not touched

Tests

test/commands/stdin_operand_test.exs, 37 tests. The registry test asserts every command in Registry.list() is either a --reads-stdin row or explicitly listed as not one, so a command added later has to be classified rather than silently skipped. Each row is checked against the same command reading a file holding the same bytes rather than against a literal transcript — the invariant is that the operand names a source, not that nl pads to six columns.

Mutation-checked: deleting the - arm of StdinOperand.read/4 turns 18 red; deleting the /dev/null arm of read_stdin_target/2 turns 4 red.

Gates

$ mix compile --warnings-as-errors --force
Compiling 169 files (.ex)
Generated just_bash app

$ mix format --check-formatted
format exit=0

$ mix credo --strict
  Refactoring opportunities
┃ [F] ↘ Avoid `apply/2` and `apply/3` when the number of arguments is known.
┃       test/support/banned_fixture_apply.ex:4:16 #(BannedCallTracer.Fixture.Apply.run)
5204 mods/funs, found 1 refactoring opportunity.        # pre-existing, intentional fixture

$ mix test
2 doctests, 62 properties, 5025 tests, 0 failures (5 excluded)

$ mix dialyzer
Total errors: 13, Skipped: 13, Unnecessary Skips: 0
done (passed successfully)

@davydog187

Copy link
Copy Markdown
Collaborator Author

Final verification

Independent re-run at d59d84d (fix: \-` and /dev/null are operands the read path must not resolve as paths) on a clean tree (git status --porcelainempty). Every repro below was executed, not reasoned about; eachJustBash.exec/2` call ran inside a monitored process with a 10 s deadline and nothing hit it.

Per-item verdicts

# Item Claim Verdict
1 Regression 1 (BLOCKER) — < /dev/null fails with rc=1 + shell diagnostic fixed fixed
2 Regression 2 (MAJOR) — sort - / uniq - / grep pat - hard-error fixed fixed
3 Audit every command taking a file operand, driven off the registry fixed gap remains — 4 commands still resolve - as a path when it is combined with another operand
4 #72's sort /d EISDIR wording — deliberately not fixed here not done upheld — correctly untouched, no collision
5 head multi-file extra blank line — pre-existing, out of scope not done upheld — reproduced byte-identical at the pre-fix head

1. Regression 1 — < /dev/null — FIXED

                                     d59d84d (head)   e1028b6 (prev head)                                  bash oracle
cat  < /dev/null; echo rc=$?         rc=0, err ""     rc=1, "bash: /dev/null: No such file or directory"    rc=0, err ""
read x < /dev/null; echo rc=$?       rc=1, err ""     rc=1, same diagnostic                                 rc=1, err ""  (EOF)
wc -l  < /dev/null                   "0\n", rc=0      rc=1, same diagnostic                                 "0",  rc=0
sort   < /dev/null; echo rc=$?       rc=0, err ""     rc=1, same diagnostic                                 rc=0
grep x < /dev/null; echo rc=$?       rc=1, err ""     rc=1, same diagnostic                                 rc=1          (no match)
cat /f < /dev/null                   "hi\n", rc=0     rc=1, command never ran                               "hi", rc=0
set -e; cat < /dev/null; echo done   "done\n", rc=0   rc=1, aborted                                         "done", rc=0
while read l < /dev/null; ...; done  rc=0, err ""     rc=0 but leaked the diagnostic                        rc=0
T=/dev/null; cat < $T; echo rc=$?    rc=0, err ""     rc=1, diagnostic                                      rc=0

Still reports for operands that really are paths, so this is not a blanket suppression:

cat < /nope/dev/null   ->  rc=1  "bash: /nope/dev/null: No such file or directory"
cat < ./dev/null       ->  rc=1  "bash: ./dev/null: No such file or directory"
cat < /dev//null       ->  rc=1  "bash: /dev//null: No such file or directory"
cd /dev; cat < null    ->  rc=1  "bash: null: No such file or directory"

The write side is unchanged — > /dev/null, 2> /dev/null, &> /dev/null, >> /dev/null all rc=0, no file created. Both directions now name the one path through @null_device.

I checked the rest of the special-file set the instruction asked for: classify_redirection/3 special-cases exactly one path, so "/dev/null is the only one the read side needed" holds for the redirection layer. /dev/stdin, /dev/zero and /dev/stderr are serviced by neither direction, before or after this commit.

Mutation test. Deleting read_stdin_target(@null_device, _bash), do: "" from redirection.ex37 tests, 4 failures. Restored → 37 tests, 0 failures.

2. Regression 2 — - names stdin — FIXED

                          d59d84d          e1028b6                                   GNU oracle
echo x | sort -           rc=0 "x\n"       rc=2 "sort: cannot read: -: ..."          rc=0 "x"
echo x | uniq -           rc=0 "x\n"       rc=1 "uniq: -: ..."                       rc=0 "x"
echo x | grep x -         rc=0 "x\n"       rc=2 "grep: -: ..."                       rc=0 "x"
printf 'b\na\n' | sort -  rc=0 "a\nb\n"    rc=2                                      rc=0 "a\nb"

set -e; printf 'b\na\n' | sort -; echo done  ->  "a\nb\ndone\n" rc=0   (was rc=2, aborted)
set -o pipefail; echo x | sort - | cat       ->  "x\n"          rc=0   (was rc=2)
set -eo pipefail; echo x | grep x - | wc -l  ->  "1\n"          rc=0   (was rc=2, "0")

Fixed by actually reading stdin, not by muting the diagnostic — the output is right, not merely quiet. sort -x is still treated as a path (rc=2 + diagnostic), so the - clause is exact-match and swallows no flags.

I also checked every shape where - is not a file operand: grep - /f (pattern -), sed -, awk -, jq -, cut -d - -f1, sort -- -, grep -- x -, paste - - (the two-column idiom → a\tb), sort -r/-u, cut -c-2/-c2-, base64 -d -, xxd -p -, nl -ba -, expand -t2 -, fold -w2 -, awk -F, '{print $1}' -, and grep under -r/-l/-H/-c/-v. All unchanged or matching GNU.

Mutation test. Deleting def read(fs, _cwd, @dash, stdin) from StdinOperand37 tests, 18 failures. Restored → green. Both mutation counts match the commit message exactly.

3. Registry-driven audit — real, but incomplete

The audit is genuine. 16 commands that previously rejected a bare - (cut head tail wc nl fold expand base64 md5sum od xxd comm diff sed awk jq) now read stdin, and the registry-enumeration test does force a newly added command to be classified. I verified all 26 rows individually against the same command reading a file with the same bytes, and checked the labels against GNU (grep(standard input), head/tailstandard input, wc/md5sum/sha*sum-).

But the audit only exercises cmd - with - as the sole operand. With - mixed with a real file, four of the seven commands the note calls "already-correct" still resolve - as a path:

$ printf 'SSS\n' | shasum - /f
shasum: -: No such file or directory                      <- stderr
fb5d5f20adab4de2ac06bedb9f5d0768186157db  /f
rc=1
# oracle:  printf 'SSS\n' | shasum - mf   ->  rc=0, both hashed

$ printf 'SSS\n' | sha256sum - /f
sha256sum: -: No such file or directory                   <- stderr
5e98a01a...  /f
rc=1
# oracle:  printf 'SSS\n' | gsha256sum - mf  ->  rc=0, both hashed

$ printf 'SSS\n' | tac - /f   ->  "FFF\n"     # oracle gtac - mf: "SSS\nFFF"  (stdin silently dropped)
$ printf 'SSS\n' | rev - /f   ->  "FFF\n"     # same

shasum/sha256sum guard with files == [] or files == ["-"] and otherwise hand - straight to FS.read_file/2. tac/rev do Enum.reject(args, &String.starts_with?(&1, "-")), which deletes the operand outright — the same shape as the base64 - /f bug this commit did find and fix.

This is pre-existing, not a regression from this PR — byte-identical output at bb55214 and e1028b6. But shasum - /f is literally Regression 2's signature (-: No such file or directory + non-zero exit), and tac - /f is the silent-wrong-answer class #70 exists to remove. The registry test cannot catch either, because it never builds a multi-operand invocation.

Not blocking. Worth a follow-up that extends the audit table with a cmd - /f row per command.

4 & 5 — the two "not done" items


Regression sweep

A 141-case behavioural diff of d59d84d against e1028b6 (previous head) and against bb55214 (pre-fix head), plus three targeted sweeps (23 cmd - vs cmd /f pairs, 27 multi-operand cmd - /f cases, 45 special-file / option-parsing edge cases). Everything that changed changed in the intended direction; nothing else moved.

Swept and clean — no change vs e1028b6 outside the two fixes:

  • Every command taking a file operand, with a real pathsort uniq grep cut head tail wc nl fold expand base64 md5sum sha256sum shasum od xxd sed awk jq cat tac rev paste file comm diff: identical stdout/stderr/rc for a readable file, a missing file, and a directory operand. All 26 missing-file diagnostics byte-identical.
  • cmd - vs cmd /f with the same bytes — 23 pairs; the only differences are the operand label (-, standard input, (standard input), /dev/stdin), which is exactly GNU's own inconsistency.
  • Stdin redirection< file, < missing, < dir (still silently empty, matching the deliberate EISDIR carve-out), heredoc, <<<, < $VAR, quoted and relative targets. Unchanged.
  • Write-side redirection>, >>, 2>, &>, &>>, >&2, > /dev/null and friends. Unchanged.
  • Pipelines, set -e, set -o pipefail — 6 combinations; the only movement is the two fixes. set -e; sort /nope still aborts; set -o pipefail; echo x | sort /nope | cat is still rc=2.
  • Option parsing around - — 20 cases; no flag newly swallowed as an operand, no operand newly read as a flag.

Found in the sweep and deliberately not reported as findings — each verified byte-identical at bb55214, so pre-existing and outside this change:

  • head -n - raises FunctionClauseError out of Enum.take/2 (also head -n '').
  • sort /f /g, od -c /f /g, xxd /f /g read only one of the operands.
  • cat /dev/null, sort /dev/null, wc -l /dev/null report ENOENT — there is no /dev/null file in the VFS; only redirection services it. Worth noting that d59d84d therefore makes cat < /dev/null (works) and cat /dev/null (ENOENT) disagree; that is the scope the instruction asked for, but it is an asymmetry now visible.
  • wc pads a single-operand count where GNU does not; od -c column spacing differs from GNU.
  • head -q, head -v, tail -f, base64 -w0, shasum -a256 are unimplemented flags that fall through into the file list.

Timing: no case exceeded its 10 s deadline; full suite 22.3 s, in line with the previous head.

Gates

All five run at d59d84d on a clean tree.

mix format --check-formatted     clean
mix compile --warnings-as-errors --force
                                 Compiling 169 files (.ex) / Generated just_bash app — zero warnings
mix credo --strict               5204 mods/funs, found 1 refactoring opportunity.
                                 (test/support/banned_fixture_apply.ex:4 — the intentional test fixture)
mix test                         2 doctests, 62 properties, 5025 tests, 0 failures (5 excluded)
mix dialyzer                     Total errors: 13, Skipped: 13, Unnecessary Skips: 0 — done (passed successfully)

Bottom line

Both regressions are genuinely fixed, both fixes are covered by tests that go red when the fix is removed (18 and 4, as claimed), and the sweep found nothing newly broken. The one gap is that the registry audit stops at single-operand -, leaving shasum/sha256sum hard-erroring and tac/rev silently dropping stdin when - is combined with a file operand. That is pre-existing and does not block this PR, but it is the same defect class, and calling those commands "already-correct" overstates the coverage.

@davydog187

Copy link
Copy Markdown
Collaborator Author

Round 4 — - mixed with a file operand (fe87d76)

Confirmed, and it was mine to fix: d59d84d's registry enumeration only ever
built a single-operand invocation, which is blind to the exact bug it was
written to catch. A parser that deletes - from the operand list still looks
correct when - is the only operand, because deleting it leaves the command
falling back to stdin for a second reason. Put a file beside it and four of the
commands that commit's note calls already-correct still resolve - as a path.

Byte-identical at bb55214 and e1028b6 — pre-existing, not a regression —
but in scope, because shasum - /f is Regression 2's failure signature and
because the note's classification was wrong.

Before (d59d84d)

$ printf 'SSS\n' | shasum - /f
shasum: -: No such file or directory                                  # stderr
fb5d5f20adab4de2ac06bedb9f5d0768186157db  /f
rc=1

$ printf 'SSS\n' | sha256sum - /f
sha256sum: -: No such file or directory                               # stderr
5e98a01a1f7970b51849ea59100ba9f5b9b9740e7356fbfef1df1957197ea975  /f
rc=1

$ printf 'SSS\n' | tac - /f      ->  "FFF\n"      rc=0   # stdin dropped
$ printf 'SSS\n' | rev - /f      ->  "FFF\n"      rc=0   # stdin dropped
$ printf 'SSS\n' | tac /f -      ->  "FFF\n"      rc=0
$ printf 'SSS\n' | rev /f -      ->  "FFF\n"      rc=0

After (fe87d76)

$ printf 'SSS\n' | shasum - /f
ba3189fb712d534af1ea2fe27ea107dbdc296775  -
fb5d5f20adab4de2ac06bedb9f5d0768186157db  /f
rc=0

$ printf 'SSS\n' | sha256sum - /f
40aed8af70d9534288ecbd92c2f780c970f4245a0e0ca4a6866eed88b24d39b1  -
5e98a01a1f7970b51849ea59100ba9f5b9b9740e7356fbfef1df1957197ea975  /f
rc=0

$ printf 'SSS\n' | tac - /f      ->  "SSS\nFFF\n"   rc=0
$ printf 'SSS\n' | tac /f -      ->  "FFF\nSSS\n"   rc=0
$ printf 'SSS\n' | rev - /f      ->  "SSS\nFFF\n"   rc=0
$ printf 'SSS\n' | rev /f -      ->  "FFF\nSSS\n"   rc=0

Oracles on the same bytes, and the hashes match digit for digit:

$ printf 'SSS\n' | shasum - f
ba3189fb712d534af1ea2fe27ea107dbdc296775  -
fb5d5f20adab4de2ac06bedb9f5d0768186157db  f
$ printf 'SSS\n' | gsha256sum - f
40aed8af70d9534288ecbd92c2f780c970f4245a0e0ca4a6866eed88b24d39b1  -
5e98a01a1f7970b51849ea59100ba9f5b9b9740e7356fbfef1df1957197ea975  f
$ printf 'SSS\n' | gtac - f   ->  SSS\nFFF
$ printf 'SSS\n' | gtac f -   ->  FFF\nSSS

Causes

shasum/sha256sum: files == [] or files == ["-"] is the only place - is
recognised, and every other operand goes to FS.read_file/2. Replaced with
StdinOperand.read/4 per operand, plus [] -> ["-"] so the no-operand case is
the same code path as a lone - instead of a second one. -c is untouched.

tac/rev: Enum.reject(args, &String.starts_with?(&1, "-")) — the same
shape as the base64 - /f bug d59d84d already found.

The reject is also a flag bug, as you flagged

It is not an option/operand split at all. Two things it gets wrong beyond -:

-- is not honoured. Every operand after it is deleted, so naming a file
whose name begins with a dash is impossible, silently:

d59d84d:  cd /; tac -- -f   ->  ""  rc=0     # /-f holds 12\n34\n
fe87d76:  cd /; tac -- -f   ->  "34\n12\n"  rc=0
oracle:   gtac -- -f        ->  "34\n12\n"  rc=0

tac -- /f and rev -- /f happened to work before only by accident: --
starts with a dash, so it was dropped along with everything that should have
been protected by it. Fixed in StdinOperand.operands/1: - is an operand,
-- ends the options, everything else unchanged.

An unrecognised option is swallowed. Not fixed, and deliberately so:

fe87d76:  rev -x /f   ->  "FFF\n"  rc=0
oracle:   rev -x f    ->  rev: illegal option -- x       rc=1
          gtac -x f   ->  gtac: invalid option -- 'x'    rc=1

Diagnosing it means also implementing tac -b/-r/-s, which are real GNU flags
this tac currently ignores — gtac -r f succeeds, so a blanket "unknown flag
is an error" would trade one divergence for another. Separate change; noted
below rather than smuggled in here.

tac needed more than the operand split

GNU reverses each operand's lines on its own and writes the operands in the
order given. tac a b is tac a; tac b, not cat a b | tac:

$ gtac a b        # a = 1\n2\n   b = 3\n4\n
2
1
4
3
$ cat a b | gtac
4
3
2
1

This tac concatenated first, so it reversed the operands against each other
too. Now per-operand, which is what makes tac - /b print 2 1 4 3 and
tac /b - print 4 3 2 1, matching gtac. rev is line-local, so
operand-order concatenation already places each operand's lines correctly and
it keeps the simpler shape.

Closing the hole in the test

Extended the same way the existing rows are checked — against the same command
reading a real file, not against a transcript. Every row now also runs as
cmd - FILE and cmd FILE -, with /s (a real file holding exactly the stdin
bytes) standing in for - in the reference run:

for {dash, ref} <- [{{"-", "/f"}, {"/s", "/f"}}, {{"/f", "-"}, {"/f", "/s"}}] do
  {piped, _bash} = run(script, dash, input, other)
  {from_file, _bash} = run(script, ref, input, other)

  assert piped.exit_code == from_file.exit_code
  assert piped.stderr == String.replace(from_file.stderr, "/s", shown)
  assert piped.stdout == String.replace(from_file.stdout, "/s", shown)
end

The two operands hold different bytes, so dropping either is visible. The
reference is itself a two-operand run, so a command that reads only its first
operand compares equal to itself — what is under test is the classification of
-, not multi-operand support. That is what keeps the matrix honest: it goes
red for a mis-split operand list and stays green for an unrelated multi-operand
gap.

Proven red against d59d84d before the fix, on exactly these four and
nothing else:

$ mix test test/commands/stdin_operand_test.exs      # at d59d84d, new test only
  1) shasum FILE: `- FILE` and `FILE -` name stdin in either position
  2) tac FILE: ...
  3) sha256sum FILE: ...
  4) an operand after `--` is a path, dash and all
  5) rev FILE: ...
64 tests, 5 failures

The other 22 rows — cat tail head wc grep sort uniq cut nl fold expand paste comm base64 md5sum od xxd diff sed awk jq file — pass the two-operand probe,
so the rest of the earlier classification does hold up.

Mutation check on the final head:

mutation red
StdinOperand.read/4 loses its - arm 40 (was 18 before this matrix)
operands/1 loses its - clause (the old reject) 3
operands/1 loses its -- clause 1

Plus a transcript test for tac's per-operand ordering, which the equality
matrix cannot see.

Divergences found while checking, left for separate changes

Measured, all pre-existing and none about - — the two-operand matrix stays
green on them because it compares - against a file rather than against GNU:

  • sort - /f reads only the first operand; so does sort /a /b. GNU merges.
  • base64 - /f concatenates; gbase64 - f is extra operand 'f', rc=1.
  • rev does not newline-terminate an unterminated final line, so operands
    still bleed across a file boundary in rev a b when a has no trailing
    newline. GNU prints ba\ndc\n there; we print dcba\n.
  • tac -b/-r/-s and rev's unknown-option diagnostic, above.

Gates

mix compile --warnings-as-errors   clean
mix format --check-formatted       clean
mix credo --strict                 1 finding: test/support/banned_fixture_apply.ex
                                   (the intentional fixture)
mix test                           2 doctests, 62 properties, 5053 tests,
                                   0 failures (5 excluded)
mix dialyzer                       Total errors: 13, Skipped: 13,
                                   Unnecessary Skips: 0 -- passed successfully

Diff is 5 lib files and 1 test file, +169 / -79.

@davydog187

Copy link
Copy Markdown
Collaborator Author

Final verification

Independent re-run at fe87d76, in a clean worktree with its own deps/_build. Every repro below
was executed here; every oracle is the tool on this machine (shasum, gsha256sum, gtac, gsort,
gmd5sum, BSD rev/sed).

Per-item

# Claim Verdict Evidence
1 shasum - /f resolved - as a path fixed printf 'SSS\n' | shasum - /f → rc=0, ba3189fb712d534af1ea2fe27ea107dbdc296775 - then fb5d5f20adab4de2ac06bedb9f5d0768186157db /f. Oracle shasum - f → same two digests, same order, rc=0. shasum /f - mirrors it. shasum -a 256 - /f and shasum -a 512 - also correct.
2 sha256sum - /f same guard fixed rc=0, 40aed8af… - / 5e98a01a… /f; oracle gsha256sum - f identical. Both positions.
3 tac deleted the - operand; also concatenated instead of per-operand fixed printf 'SSS\n' | tac - /f"SSS\nFFF\n" (was "FFF\n"), oracle gtac - f same. Ordering: tac /a /b2 1 4 3, tac /b /a4 3 2 1, printf '1\n2\n' | tac - /b2 1 4 3, tac /b -4 3 2 1 — all four match gtac.
4 rev deleted the - operand fixed rev - /f"SSS\nFFF\n", rev /f -"FFF\nSSS\n". No GNU rev exists; BSD rev - f errors on - (rc=1). The note's framing is right — rev - = stdin was already established in d59d84d, so the positional inconsistency was the defect.
5 -- not honoured by tac/rev fixed cd /; tac -- -f"34\n12\n" rc=0 (oracle gtac -- -f identical); rev -- -f"21\n43\n" (oracle BSD rev -- -f identical). tac -- / rev -- alone still read stdin; tac -- - reads the file named -… i.e. nothing here, rc=0 — consistent with the split.
6 Registry enumeration extended to two operands fixed See "Coverage can fail" below.
7 tac/rev swallow an unknown option not_done, accurately described tac -x /f → rc=0 "2\n1\n"; oracle gtac -x f → rc=1 invalid option -- 'x'. rev -x /f → rc=0; oracle rev -x f → rc=1 illegal option -- x. tac -r /f → rc=0 ignoring -r; gtac -r f → rc=0 too. The stated reason for deferring holds.
8 Other multi-operand divergences not_done, accurately described sort - /f"a\nb\nb\n" (first operand only) and sort /a /b does the same, vs gsort merging. base64 - /f concatenates; gbase64 - f → rc=1 extra operand. rev /n /a with /n unterminated → "1\n12\n2\n", BSD rev n a"1\n2\n1\n2\n". All three reproduced exactly as written.

Note on #6's numbers: the note says "64 tests, 5 failures" against d59d84d. The file as merged is 65
tests and goes 6 red
— the tac per-operand-ordering transcript fails there too. Stale count from
before that test was added, not a defect.

Coverage can fail

Lib reverted to d59d84d, new test file kept — 65 tests, 6 failures, exactly:

1) an operand after `--` is a path, dash and all
2) tac reverses each operand on its own, in operand order
3) tac FILE: `- FILE` and `FILE -` name stdin in either position
4) rev FILE: …
5) shasum FILE: …
6) sha256sum FILE: …

The other 22 rows stay green there, confirming the earlier classification for the rest of the set.

Mutations on the final head (full suite in parentheses):

Mutation stdin_operand_test.exs full suite
remove the - arm of StdinOperand.read/4 40 failures 49
remove do_operands/2's - clause 3 3
remove do_operands/2's -- clause 1 1

Matches the claimed 40 / 3 / 1. Restored and re-verified clean after each.

Sweep

Independent audit of the whole "already-correct" set. All 26 rows run as cmd - /f and
cmd /f - against the real tool with the same bytes. - is classified as stdin correctly in both
positions for every row.
The residual differences are all multi-operand support, reproduced with
two plain files (cmd /a /b) and therefore not about -:

  • od, xxd read only the last operand (od -c /a /b/b only).
  • sort, uniq read only the first (sort /a /b/a only).
  • jq . /ja /jb → rc=1 jq: too many arguments.
  • base64 /a /b concatenates; wc pads to a different width; head emits an extra blank line
    between operand headers; comm/uniq lack GNU's unsorted-order warnings and uniq's OUTPUT operand.

Clean, i.e. byte-identical to the oracle in both positions: file, head/tail (modulo the header
newline), grep, cat, tac, cut, nl, fold, expand, paste, md5sum, sha256sum, shasum,
awk, sed, diff (semantics), jq (single operand).

Pre-existing, not regressions — verified by diffing the four command files bb55214..d59d84d
(only tac.ex's error template changed) and by re-running the sweep with lib/ at d59d84d.

Operands beginning with a dash / --: -- is honoured inconsistently across the codebase and
this commit only fixed tac/rev. sort -- /f, wc -l -- /f, head -- /f accept it; cat -- /f
rc=1 cat: --: No such file or directory; md5sum -- /f / base64 -- /f → rc=1 invalid option '--';
shasum -- /f / sha256sum -- /f → rc=1 shasum: --: No such file or directory plus the correct
hash. Oracles all rc=0. Identical at d59d84d — pre-existing.

Test timing: stdin_operand_test.exs 37 → 65 tests, 0.1s → 0.3s. Full suite 22.2s, unchanged.
No other test's timing assumptions touched (nothing outside this file and the four command modules
was modified).

Still open in the same commands (pre-existing, flagged not fixed)

  1. shasum -c - / sha256sum -c - still resolve - as a path — the exact Regression 2 signature,
    in a command this round fixed:

    printf 'fb5d5f20…  /f\n' | shasum -c -   → rc=1  "shasum: -: No such file or directory"
    oracle: shasum -c - < sums.txt           → rc=0  "f: OK"
    

    And with no operand at all it is a silent success — the class Differential testing roadmap: enumerate the remaining alphabets, compare filesystem state, and activate the 2,728 inert Oils spec cases #70 exists to remove:

    printf 'fb5d5f20…  /f\n' | shasum -c     → rc=0, no output
    oracle: shasum -c < sums.txt             → rc=0  "f: OK"
    

    check_checksums/3 reduces over files with plain FS.read_file/2, so [] is a no-op and ["-"]
    hits the filesystem. Byte-identical at d59d84d, so not a regression; the note says "-c untouched"
    without giving the oracle.

  2. tac's unterminated final line, the analogue of the disclosed rev bleed, not mentioned:

    /n = "1\n2"
    tac /n     → "2\n1\n"        gtac n    → "21\n"
    tac /n /a  → "2\n1\n2\n1\n"  gtac n a  → "21\n2\n1\n"
    

    The two-operand case changed here (d59d84d gave "2\n21\n1\n") — closer, operand order now right,
    but the record split is still wrong.

Gates (all re-run here)

$ mix compile --force --warnings-as-errors
Compiling 169 files (.ex)
Generated just_bash app                                    # clean

$ mix format --check-formatted
                                                           # clean, rc=0

$ mix credo --strict
5222 mods/funs, found 1 refactoring opportunity.
  [F] test/support/banned_fixture_apply.ex:4:16             # the intentional fixture

$ mix test
Finished in 22.2 seconds (22.0s async, 0.2s sync)
2 doctests, 62 properties, 5053 tests, 0 failures (5 excluded)

$ mix dialyzer
Total errors: 13, Skipped: 13, Unnecessary Skips: 0
done (passed successfully)

Verdict: all six claimed fixes verified against the oracle, both not_done items accurately
described, coverage proven able to fail, no regressions found. The two items above are pre-existing
and out of the stated scope, but item 1 is the same failure signature in the same command, so it is
worth a follow-up rather than a merge blocker.

… saying "No such file or directory"

33 command modules matched `{:error, _}` and threw the error kind away,
then spelled every failure "No such file or directory". `stat /f/x` where
`/f` is a regular file claimed the path did not exist, when it exists and
its parent component is not a directory; the same wrong message came out
of a directory read (`:eisdir`) and an unreadable mount (`:eacces`).

Every one of them now renders `FS.strerror/1` on the error it was already
given, so the message matches the kind. The message templates are
unchanged.

`JustBash.exec_file/2` had the same defect and is fixed with them.

The new test is the cross product of 34 filesystem-touching commands and
the kinds a path can fail with, enumerated off a table rather than
written case by case — 126 cases, 82 of which failed before this change.
`:eacces` is unreachable from the in-memory backend, which has no
permission model, so it comes from a new test-support mount that refuses
every operation with one configured kind.

Refs #70
…reads ## STDERR

`lib/just_bash/spec_test/parser.ex` has no callers, so its four defects had
nothing to catch them:

  * `:skip` was never assigned, making `Runner`'s skip guard dead code.
    Compounding it, Oils writes `## SKIP` *above* the script and the parser
    stopped collecting the script at the first `##` line — so all 247
    SKIP-marked cases ran, with an empty script, and passed vacuously.
  * `String.trim_leading(line, "#### ")` strips the prefix *repeatedly*, so a
    case name that itself starts with the header prefix loses part of itself.
  * `String.trim(script)` trims whitespace *characters*, rewriting the last
    command of every case whose script ends in a space.
  * `## STDERR` was never read, so the 65 cases carrying a stderr expectation
    asserted nothing about stderr — the class the silently-ignored-flag bugs
    live in.

A case body is now sorted line by line into script or directive, in either
order, so a directive above the script no longer eats it, and a multiline
block keyed to another shell (`## N-I dash STDOUT:`) is consumed to `## END`
instead of falling through into the script. The script is preserved between
its first and last non-blank line: only whole blank lines at the ends are
dropped, because `$LINENO` and bash's `line N:` diagnostics count from the
first surviving line and the corpus asserts on both.

`Runner` reports a skipped case as skipped rather than as an error, and
compares stderr when the case names one.

The suite is still not wired into CI — activating it is a separate roadmap
item. The tests here exercise the parser directly against the real corpus:
it reads 2728 cases from 136 files, marks exactly the 247 SKIP markers the
files contain, and finds exactly 65 stderr expectations.

Refs #70
An Oils annotation naming a shell records what *that shell* actually does.
The parser treated every one of them as foreign and dropped it, so the 323
cases (440 annotation lines, 89 files) that spell bash's behaviour as
'## OK bash STDOUT:' / '## OK bash status: 0' were left pinned to the
unannotated default, which records osh. append.test.sh 'Try to append list
to element' asked for stdout "" and status 2 while the file plainly records
status 0 and ['1', '2 3'] for bash.

Annotations are now parsed into {qualifier, shells, key: value} and applied
per key, weakest first: unannotated default, then bash, then this repo's own
'## OK just-bash' markers, which were being discarded too. 'bash-2' is bash
2.x and stays foreign.

Separately, '## stdout:' with nothing after the colon is how the format
spells one expected empty line; the clause required the separating space, so
four cases parsed to expected_stdout: nil and Runner.check_output(nil, _)
passed them whatever they printed. Same hole for a bare '## stderr:'.

The corpus-wide counts in the test now include the ones that move when a
directive spelling stops being recognised: 2562 cases with a stdout
expectation, 239 with a non-zero status, and 59 - not 67 - that assert
nothing at all.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
…tream

'md5sum /f /nope' put 'md5sum: /nope: No such file or directory' into stdout
between checksum lines, so 'md5sum a b > sums.txt' wrote a malformed checksum
line into sums.txt and '2>/dev/null' did not silence it. gmd5sum writes to
stderr with an empty stdout, and sha256sum/shasum already did here.

'md5sum -c FILE' on a checksum file it cannot read also said nothing and
exited 0; it now reports the kind and exits 1. An unreadable checksum file is
not a checksum that did not match, so it does not feed the WARNING line.

The matrix row that recorded the divergence as :stdout now asserts :stderr.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
GNU head, tail and tac open(2) a directory successfully and only fail at
read(2), so EISDIR selects a different template from the kinds that fail to
open at all. Verified against coreutils 9.x:

  ghead d -> ghead: error reading 'd': Is a directory
  gtac d  -> gtac: d: read error: Is a directory
  gtac nope -> gtac: failed to open 'nope' for reading: No such file or directory

The sweep's 'templates untouched' rule mis-rendered all three, and the matrix
asserted the wrong string for the :eisdir row rather than catching it. tac was
using the wrong template for every kind, not just EISDIR.

The matrix rows now carry the kinds they cover, which is what lets one command
have two templates; the content-reader/metadata-reader split it replaces was
only ever a way to spell 'these kinds'.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
format_error_line/3 rendered FS.strerror/1 in the non-brief branch but the
brief branch still returned the constant 'cannot open', so 'file -b /f/x' and
'file -b /nope' were indistinguishable. -b suppresses the filename prefix, not
the errno text. The matrix only ever exercised 'file PATH', which is why the
half-fixed function read as covered; it now exercises 'file -b PATH' too.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
GNU realpath without -e requires only the parent components to exist:
'grealpath nope' exits 0 and prints the canonicalised path, while
'grealpath nope/deep/x' exits 1. just_bash stat'd the whole path, so
'realpath /nope' failed - and the matrix recorded that as ground truth for
the one kind where realpath is not supposed to fail.

The three modes are now real: default requires the parent to be a directory
(so '/f/x' is still ENOTDIR even though '/f' stats cleanly), -e requires the
whole path, -m requires nothing. -e and -m were being parsed and discarded.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
…sites

Every row passed exactly one path operand, so the multi-file arms of head,
tail and wc and both -c arms of sha256sum and shasum were untestable by
construction. Replacing the rendered strerror with the literal MUTANT at
wc.ex:66, sha256sum.ex:73, sha256sum.ex:114, shasum.ex:87, shasum.ex:121 and
the head/tail reduce arms left the whole suite green at 4923 tests.

Rows naming two operands, '-c FILE', and a checksum file listing an
unreadable target now cover all seven: the same mutation turns 21 tests red.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
…t do

The sweep was driven by grepping for the literal 'No such file or directory',
so it reached every module that printed the wrong message and none that
printed nothing. These printed nothing:

  sort /nope    rc=0 silent  ->  rc=2 sort: cannot read: /nope: ...
  sort /d       rc=0 silent  ->  rc=2 sort: read failed: /d: Is a directory
  cut -f1 /nope rc=0 silent  ->  rc=1 cut: /nope: ...
  uniq /nope    rc=0 silent  ->  rc=1 uniq: /nope: ...
  uniq /d       rc=0 silent  ->  rc=1 uniq: error reading '/d': Is a directory
  grep x /nope  rc=1 silent  ->  rc=2 grep: /nope: ...
  cat < /nope   rc=0 silent  ->  rc=1 bash: /nope: ...
  wc -l < /nope rc=0 and '0' ->  rc=1 bash: /nope: ..., no stdout

An agent writing 'sort report.txt | head -5' against a mistyped path got a
clean exit-0 empty result indistinguishable from an empty file; 'wc -l' put a
fabricated 0 on stdout, which looks like an answer.

grep keeps GNU's rule that -q with a line selected exits 0 even after an
error, and cut keeps the files it could read. '<' is now the read side of
025673c: the shell opens the target, so the shell reports the failure and the
command does not run. EISDIR is deliberately excluded - open(2) on a directory
succeeds, so bash runs the command and the command's own read fails.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
… paths

e1028b6 taught the read paths to report the read they could not do. That is
only right for operands that are really paths, and two are not — so the fix
turned two silent successes into hard failures:

  cat < /dev/null    rc=0 ""  ->  rc=1 bash: /dev/null: No such file...
  echo x | sort -    rc=0 ""  ->  rc=2 sort: cannot read: -: No such file...
  echo x | uniq -    rc=0 ""  ->  rc=1 uniq: -: ...
  echo x | grep x -  rc=1 ""  ->  rc=2 grep: -: ...

`cmd < /dev/null` is how a command is handed a closed stdin, and `sort -` is
how a pipeline names its own input; both now abort, and under `set -e` or
`pipefail` they take the pipeline with them.

/dev/null was special-cased only on the write side of redirection. The two
directions now share one @null_device and agree: `> /dev/null` discards, so
`< /dev/null` reads empty.

`-` is fixed by reading stdin, not by suppressing the diagnostic — rc=0 with
no output is the silent-success class #70 exists to remove. The audit was
driven off the registry rather than by hand, which reached nineteen more
commands that had the same gap before the sweep and were merely quieter about
it: cut, head, tail, wc, nl, fold, expand, base64, md5sum, od, xxd, comm,
diff, sed, awk, jq and paste/tac/rev, which already worked. Ten of them never
reached a read path at all — their arg parsers called a bare `-` an invalid
option, which POSIX says it never is.

The per-operand label follows GNU, which does not agree with itself: grep
prints "(standard input)", head and tail print "standard input", wc and
md5sum print "-".

The test enumerates the whole registry and asserts every command is either a
`-`-reads-stdin row or explicitly listed as not one, so a command added later
has to be classified. Each row is checked against the same command reading a
file with the same bytes, rather than a transcript: the invariant is that the
operand names a source. Removing the `-` arm of StdinOperand.read/4 turns 18
of them red; removing the /dev/null arm turns 4 red.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
d59d84d taught nineteen commands that a bare `-` names stdin, and enumerated
the registry to prove it. The enumeration only ever built a *single*-operand
invocation, and that is blind to the bug it was meant to catch: a parser that
deletes `-` from the operand list still looks correct when `-` is the only
operand, because deleting it leaves the command falling back to stdin anyway.
With a file beside it, four commands the fix note called already-correct still
resolved `-` as a path:

  printf 'SSS\n' | shasum - /f     rc=1  shasum: -: No such file or directory
  printf 'SSS\n' | sha256sum - /f  rc=1  sha256sum: -: No such file...
  printf 'SSS\n' | tac - /f        rc=0  "FFF\n"      -- stdin dropped
  printf 'SSS\n' | rev - /f        rc=0  "FFF\n"      -- stdin dropped

Both known shapes. `shasum`/`sha256sum` guard with `files == [] or files ==
["-"]` and hand anything else to `FS.read_file/2`, so `shasum - /f` is exactly
the failure signature the `-` fix existed to remove. `tac`/`rev` do
`Enum.reject(args, &String.starts_with?(&1, "-"))`, the same shape as the
`base64 - /f` bug d59d84d already found. Byte-identical at bb55214 and
e1028b6: pre-existing, not a regression.

That reject is not an option/operand split. It also deletes every operand
after `--`, which is the only way to name a file whose name begins with a
dash, so `tac -- -f` read nothing and exited 0. Both deletions are silent.
`StdinOperand.operands/1` is the split done properly: `-` is an operand, `--`
ends the options, everything else is as before.

`tac` needed more than the split. GNU reverses each operand's lines on its own
and writes the operands in the order given -- `tac a b` is `tac a; tac b`, not
`cat a b | tac` -- and this `tac` concatenated first, so it also reversed the
operands against each other. `rev` is line-local, so operand-order
concatenation already places each operand's lines where they belong.

    $ printf 'SSS\n' | shasum - /f
    ba3189fb712d534af1ea2fe27ea107dbdc296775  -
    fb5d5f20adab4de2ac06bedb9f5d0768186157db  /f
    $ printf '1\n2\n' | tac - /b          # /b holds 3\n4\n
    2
    1
    4
    3

matching `shasum - f`, `gsha256sum - f` and `gtac - b` byte for byte, in both
operand positions.

The hole in the test is closed the way the rows are already checked: against
the same command reading a real file, not against a transcript. Every row now
also runs as `cmd - FILE` and `cmd FILE -`, with `/s` -- a real file holding
the stdin bytes -- standing in for `-` in the reference run. Because the
reference is itself a two-operand run, a command that reads only its first
operand compares equal to itself: what is under test is the classification of
`-`, not multi-operand support. The matrix is red on exactly these four
against d59d84d, and green on the other twenty-two rows.

Removing the `-` arm of `StdinOperand.read/4` now turns 40 rows red, up from
18; removing `operands/1`'s `-` clause turns 3 red; removing its `--` clause
turns 1 red.

Left alone, and not about `-`: `tac`/`rev` still swallow an unrecognised option
rather than diagnosing it (`rev -x f` exits 0 where GNU and BSD exit 1), `tac`
still ignores `-b`/`-r`/`-s`, `sort` still reads only its first operand, and
`base64` concatenates its operands where GNU calls the second one an extra
operand.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant