Skip to content

fix: a flag a command does not implement is an error, not a filename - #72

Merged
davydog187 merged 9 commits into
mainfrom
fix/issue-68-strict-unknown-flags
Aug 7, 2026
Merged

fix: a flag a command does not implement is an error, not a filename#72
davydog187 merged 9 commits into
mainfrom
fix/issue-68-strict-unknown-flags

Conversation

@davydog187

@davydog187 davydog187 commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Closes #68

What was broken

JustBash.FlagParser turned a flag it did not recognise into a positional
operand (lib/just_bash/flag_parser.ex:75), so the flag became a filename.
For sort and uniq — which silently treat an unreadable file as empty —
that produced the worst available outcome: the file's contents dropped, and
exit 0 to say it worked.

bash = JustBash.new(files: %{"/f.txt" => "a\nb\nc\n"})

for cmd <- ["sort -Q /f.txt", "uniq -Z /f.txt", "head -q /f.txt", "ls -Q /"] do
  {r, _} = JustBash.exec(bash, cmd)
  IO.puts("$ #{cmd}\n   exit=#{r.exit_code} stdout=#{inspect(r.stdout)} stderr=#{inspect(r.stderr)}")
end
command before after GNU coreutils 9
sort -Q /f.txt exit 0, stdout "", stderr "" exit 2, sort: invalid option -- 'Q' + Try 'sort --help'… identical
uniq -Z /f.txt exit 0, stdout "", stderr "" exit 1, uniq: invalid option -- 'Z' + Try 'uniq --help'… identical
head -q /f.txt exit 1, head: cannot open '-q' for reading… exit 1, head: invalid option -- 'q' + Try 'head --help'… exit 0 (-q is implemented there)
ls -Q / exit 1 and still printed the listing exit 2, ls: invalid option -- 'Q' + Try 'ls --help'… exit 0 (-Q is implemented there)

head -q and ls -Q are real coreutils flags that just are not implemented
here; rejecting them names the actual problem instead of blaming a file that
was never on the command line.

Three more, found on the way:

$ echo abc | tr -dX b     => "ac"   exit 0     (X silently dropped)
$ echo abc | tr -x b      => "abc"  exit 0     (-x became the from-set)
$ sort -t 1 -k2 /f.txt    => ** (FunctionClauseError) String.split/2 given the integer 1

What the fix does

1. FlagParser rejects instead of demoting. parse/2 now returns
{:ok, flags, rest} or {:error, {:unknown_flag, flag}} /
{:error, {:missing_value, flag}}. format_error/3 renders it in GNU's
shape, checked against GNU coreutils 9 on this machine:

  • short option named by its character (invalid option -- 'Q'), because that
    is the unit getopt stopped on — out of a cluster, -rQ reports Q;
  • long option named in full (unrecognized option '--jb-not-a-flag'), because
    invalid option -- '-' identifies nothing;
  • option requires an argument -- 'n' for a value flag with nothing after it,
    which used to become a file named -n;
  • the Try '<cmd> --help' for more information. line, and coreutils' exit
    codes: 2 for sort, ls and grep, 1 for cp, head, tail, uniq.
    Only grep prints a Usage: line on a bad option; coreutils does not, so
    neither do we.

All seven callers (cp, grep, head, ls, sort, tail, uniq) surface
it. -- still ends option parsing and a lone - is still an operand.

A cluster is parsed the way getopt parses one: booleans accumulate and a value
flag takes the rest of the cluster as its argument, or the next argument when
it is the last character, so sort -nk2 is -n -k 2 and sort -rt: -k2
sorts. The scan stops on the first character the spec does not describe at
all
, so the diagnostic never names a flag the command implements.

A flag declared :integer is validated. head -n abc, head -nq and
head -n '' used to raise a FunctionClauseError out of JustBash.exec/2 and
head -c xy printed the whole file at exit 0; all four are now
head: invalid number of lines: 'abc' at exit 1, GNU's wording, with no
Try --help line because GNU prints none for this one. The noun comes from a
:value_labels spec key, required of every :integer flag.

--help answers. Every rejection prints
Try '<cmd> --help' for more information., so that command has to work.
FlagParser.help/2 renders the usage out of the spec — it lists exactly what
the parser accepts and cannot drift from it:

$ sort --help
Usage: sort [OPTION]... [FILE]...
Options this shell implements:
  -f
  -k VALUE
  -n
  -r
  -t VALUE
  -u

All eight commands answer at exit 0. -- still wins, so sort -- --help is an
operand. --version stays an unrecognized option, and the advice it prints now
leads somewhere.

2. tr's own copy. The combined-flag reducer's catch-all discarded unknown
characters while consuming the token, and a single-character unknown flag fell
past the byte_size(flags) > 1 guard into the operand clause and became a
character set. Both now report tr: invalid option -- 'x' at exit 1. tr also
learned --, so echo a-b | tr -- '-' '_' gives a_b as in GNU rather than
empty output.

tr also refuses to translate with one set. One set and no -d/-s used to
fall past every run/2 head into defp run(_input, _opts), do: "", so
echo abc | tr -- -d and echo abc | tr x answered with empty stdout at exit
0 — the failure this PR exists to remove. Both are now GNU's
tr: missing operand after 'x' + Two strings must be given when translating.
at exit 1.

3. parse_value/1 no longer coerces. Only flags a spec declares
:integer are converted; head and tail declare :n and :c. Everything
else stays a string, so sort -t 1 -k2 is a delimiter of "1" and sorts
instead of crashing. (The issue cites cut -d 1 for this — cut has its own
hand-rolled parser and was never affected; sort -t is the real victim.)

4. sort reports a file it cannot read. get_content/3 mapped every
FS.read_file error to {"", fs}, so sort -- -Q was still the exact symptom
of the issue: empty stdout, empty stderr, exit 0. It is now
sort: cannot read: -Q: No such file or directory at exit 2, as in GNU, and a
lone - reads standard input rather than naming a file that does not exist.

Per the issue's warning, the ~45 hand-rolled parse_args commands are not
migrated onto FlagParser here.

New tests

test/commands/unknown_flags_test.exs (22 tests) — the repro from the issue as
exact-string assertions for all eight commands, cluster and long-option
wording, missing values, -- and lone -, and the two value-coercion cases.

test/flag_parser_test.exs — updated to the new return shape, plus 12 new
tests covering every error path and the :integer opt-in.

The registry-wide probe. test/commands/unknown_flags_test.exs enumerates
all 92 names in Commands.Registry against cmd --jb-not-a-flag and cmd -Z
and compares the observed behaviour of the whole registry to a classification
table, so a new command cannot be added without being classified and none can
change category unnoticed. A second test asserts every rejecting command names
itself in the diagnostic, and a third pins the eight shared-parser commands to
:strict so the table cannot be edited to demote one.

A non-zero exit is not evidence that a flag was rejected: demoting it to a
filename and failing to open that file also exits non-zero, and leaves the
command's real output on stdout. :strict therefore requires empty stdout and
a diagnostic that names the option. Three of the five categories are honest
about the state of the tree:

  • :strict (36) — the flag was rejected: non-zero exit, nothing on stdout, and
    invalid option / unrecognized option in the diagnostic. Where every
    command should be, and where the eight fixed here now are.
  • :misblamed (21) — still wrong: non-zero, but the diagnostic is about a
    file the flag was turned into. cat -Z /f.txt says
    cat: -Z: No such file or directory and prints the file anyway.
  • :quiet (exit, false, read) — non-zero, no diagnostic; bash is equally
    silent, they parse no options.
  • :operand (:, echo, test, true, yes) — exit 0 is correct, bash also
    treats the argument as data.
  • :absorbed (27) — still wrong: exits 0 with the flag ignored, exactly the
    way sort -Q did. These are the hand-rolled parsers the issue defers. They
    are listed by name rather than waved at, so fixing one forces an edit to the
    table and the list can only shrink.

Run against 82ee9e5 the classifier puts all eight shared-parser commands
outside :strictcp/head/ls/tail :misblamed, sort/uniq
:exit_zero, grep :quiet, tr both — so the matrix fails on the baseline
for every caller this PR claims to fix.

No exception list was needed for "-Z legitimately implemented" — no command
in the registry implements -Z.

Gates

$ mix compile --warnings-as-errors
(no output)

$ mix format --check-formatted
(no output; exit 0)

$ mix credo --strict
  Refactoring opportunities
┃ [F] ↘ Avoid `apply/2` and `apply/3` when the number of arguments is known.
┃       test/support/banned_fixture_apply.ex:4:16 #(BannedCallTracer.Fixture.Apply.run)

Analysis took 2.8 seconds (0.3s to load, 2.5s running 68 checks on 230 files)
5149 mods/funs, found 1 refactoring opportunity.

(the single finding is the pre-existing intentional test fixture — identical on main)

$ mix test
Finished in 32.6 seconds (32.2s async, 0.4s sync)
2 doctests, 62 properties, 4818 tests, 0 failures (5 excluded)

$ mix dialyzer
Total errors: 13, Skipped: 13, Unnecessary Skips: 0
done (passed successfully)

4752 tests before, 4818 after, no regressions.

Left out

Nothing from the issue's scope. Three adjacent gaps in tr are noted and not
touched, because none is reachable through anything this PR adds: tr -cs a
returns "" at exit 0 where GNU prints abc, and tr a b c / tr -d a b
reach run/2's catch-all where GNU says extra operand.

@davydog187 davydog187 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two lenses were applied to this PR: a behavioural-parity pass (run each claim against the branch and against 82ee9e5, compare to GNU coreutils 9) and a test-strength pass (does the new test suite actually distinguish fixed from unfixed?). Eleven candidate findings went in; seven survive after merging duplicates and refuting the rest. Everything below was reproduced in a worktree at 52562f1, with 82ee9e5 as the baseline.

The core of the change is right, and the GNU wording is faithful. The findings are about the shapes the fix does not cover and about the tests not being able to see them.


1. A cluster containing an implemented value flag is rejected, and the diagnostic names a flag the command supports

majorlib/just_bash/flag_parser.ex:213 (offender search at :219)

try_attached_value_flag/4 only peels an attached value off character 0 of the cluster, so a getopt cluster of booleans followed by a value option falls through parse_combined_flags/4 and try_numeric_flag/4 into unknown_flag/2. There the offender is picked with:

Enum.find(graphemes, flag_str, fn char -> Map.get(lookup, char) not in spec.boolean end)

The predicate treats any non-boolean character as the offender. Real getopt stops on the first character not in the option string at all; a value flag inside a cluster consumes the remainder as its argument. So the message names a flag the command does implement — the same "blames something that was never wrong" shape the PR set out to fix for head -q.

Verified in the PR worktree, with /f.txt:

command this branch baseline 82ee9e5 GNU coreutils 9
sort -nk2 /f.txt exit 2, sort: invalid option -- 'k' exit 0, empty exit 0, sorted
sort -rk1 /f.txt exit 2, sort: invalid option -- 'k' exit 0, empty exit 0, sorted
sort -rt: -k2 /f.txt exit 2, sort: invalid option -- 't' exit 0, empty exit 0, sorted
sort -t: -rk2 /f.txt exit 2, sort: invalid option -- 'k' exit 0, empty exit 0, sorted

Both -t and -k are implemented by this sort. sort is the only FlagParser caller with both boolean and value flags, so it is the whole blast radius — but every combination is affected (-nk, -rk, -uk, -fk, -rt, -nt). Not a correctness regression (these were silently absorbed before), but the diagnostic now actively misinforms: an agent reading invalid option -- 'k' will drop -k and get differently sorted output rather than learning that clustered value flags are unsupported.

Suggested fix: scan past leading boolean characters in try_attached_value_flag/4 before looking for an attached value, and handle the "value flag is last in the cluster, argument is the next argv" case in parse_flag/4. Failing that, at minimum restrict the offender search to characters that appear in no part of the spec, so the message never names an implemented flag.

Test gap: every new cluster test (sort -rQ, -alQ, -Qal) uses a boolean-only spec. Nothing exercises a cluster containing a :value or :multi_value flag, which is why sort -rt: -k2 ships green.


2. The registry matrix's :strict bucket cannot distinguish rejecting a flag from turning it into a filename

majortest/commands/unknown_flags_test.exs:367

probe/2 classifies purely on exit code and stderr emptiness:

cond do
  result.exit_code == 0 -> :exit_zero
  result.stderr == "" -> :quiet
  true -> :strict
end

It never checks that stdout is empty or that the diagnostic is about an option. So a command that demotes the unknown flag to an operand and then fails to open it is indistinguishable from one that rejects the flag — even though the table comment calls :strict "What every option-parsing command should do."

Concretely, in the PR worktree:

  • cat -Z /f.txt → exit 1, stdout "b:2\na:1\n", stderr cat: -Z: No such file or directory → classified :strict
  • wc -Z /f.txt → exit 1, stdout " 2 2 8 /f.txt\n…", stderr wc: -Z: No such file or directory → classified :strict

GNU for cat -Z is cat: invalid option -- 'Z'. Around twenty :strict entries are strict only because the demoted flag names a file that does not exist (cat, wc, sha256sum, shasum, rm, ./source, cd, markdown/md), and comm, env, nl, readlink, which emit invalid option -- '-Z' with GNU's dash duplicated.

The sharper demonstration: I ran the same classification logic on baseline 82ee9e5, before this fix:

ls   -Z  => :strict  (ls: cannot access '-Z': No such file or directory)
head -Z  => :strict  (head: cannot open '-Z' for reading: …)
tail -Z  => :strict  (tail: cannot open '-Z' for reading: …)
cp   -Z  => :strict  (cp: missing destination file operand after '-Z')

Four of the seven FlagParser callers already classified :strict on the unfixed tree. Only sort/uniq (:exit_zero) and grep (:quiet) change category. So the matrix certifies as correct exactly the head -q / ls -Q half of issue #68 — the half the PR describes as "the message names the wrong problem."

Suggested fix: have probe/2 also require result.stdout == "" and result.stderr =~ ~r/invalid option|unrecognized option|illegal option/ for :strict, and add a fourth honest bucket (:misblamed?) for the demote-then-fail commands. That keeps the list-can-only-shrink property the table is built around and makes the comment true.


3. tr's new -- clause opens a fresh exit-0 / empty-output path

majorlib/just_bash/commands/tr.ex:38 (catch-all at :138)

The new clause pushes everything after -- into sets without checking the arity tr needs:

defp parse_args(["--" | rest], opts) do
  parse_args([], %{opts | sets: Enum.reverse(rest) ++ opts.sets})
end

When that leaves exactly one set and neither -d nor -s is set, execution falls through every run/2 head into defp run(_input, _opts), do: "" — empty stdout, exit 0, no stderr. That is precisely the failure class issue #68 is about.

Verified:

$ echo abc | tr -- -d     branch: exit 0, stdout "", stderr ""     baseline: exit 0, stdout "abc\n"
$ echo abc | tr -- x      branch: exit 0, stdout "", stderr ""     baseline: exit 0, stdout "abc\n"

GNU coreutils 9: exit 1, tr: missing operand after ‘-d’ / Two strings must be given when translating.

So the diff replaced wrong-but-visible output with the silent-empty result, in the same PR whose thesis is that silent-empty is the worst available outcome. The root cause is pre-existing — plain tr abc also reaches the catch-all — but the new -- clause is a new route into it, and the registry matrix's "tr" => :strict classification only holds for the two probe flags it happens to test.

Suggested fix: guard on set count in parse_args/2 — one set with no -d/-s is tr: missing operand after 'X' + Two strings must be given when translating. at exit 1; zero sets is the existing tr: missing operand. That closes the pre-existing hole and the new one together.


4. :integer is declared but never validated, so head -nq raises out of JustBash.exec/2

majorlib/just_bash/flag_parser.ex:242 (spec at lib/just_bash/commands/head.ex:12, test at test/flag_parser_test.exs:172)

defp parse_value(value, flag_atom, spec) do
  with true <- flag_atom in Map.get(spec, :integer, []),
       {num, ""} <- Integer.parse(value) do
    num
  else
    _ -> value
  end
end

The else conflates "this flag is not an integer flag" with "this integer flag got garbage", handing the raw string back to a caller that has just declared the flag numeric. :integer is the one place in the new design that knows the value must be a count, and it declines to say so.

Verified on both branches:

$ head -n abc /f.txt   ** (FunctionClauseError) no function clause matching in Enum.take/2
$ head -nq  /f.txt     ** (FunctionClauseError) no function clause matching in Enum.take/2
$ head -n '' /f.txt    ** (FunctionClauseError) …
$ head -c xy /f.txt    exit 0, prints the whole file        (the quiet twin)

An unhandled exception escaping JustBash.exec/2 — out of a library whose purpose is safely running untrusted shell. GNU: exit 1, head: invalid number of lines: 'abc'.

The crash pre-exists on main, so this is not a regression. Two things make it in-scope anyway: -nq is reachable straight from this PR's own theme (an unimplemented head flag clustered after -n), and the PR rewrites this exact function while adding test "leaves a non-numeric value alone even when declared :integer" (test/flag_parser_test.exs:172) asserting {:ok, %{c: "all"}, []} — which locks the crashing behaviour in as intended.

Suggested fix: distinguish the two else cases and return {:error, {:invalid_value, flag, value}}, rendered as GNU words it. It is the third getopt error class alongside the two the PR did add, and the channel now exists at zero extra cost. Then flip the test at :172 to assert the error.


5. --help and --version are hard errors on all eight commands, and the advice they print is a command that also fails

minorlib/just_bash/flag_parser.ex:106

$ sort --help
exit 2
sort: unrecognized option '--help'
Try 'sort --help' for more information.

Followed literally, the advice produces the same error again. Identical for --version and for ls, head, tail, uniq, cp, grep, tr. Baseline was not better (sort --help was exit 0/empty; ls --help blamed a missing file), so this is not a regression.

But issue #68's stated motivation is the agent feedback loop — "the model can only learn what a command supports by running it." This PR closes the first dead end and creates a second one, and the new Try '<cmd> --help' line is what advertises it.

Suggested fix: intercept --help in FlagParser.parse/2 (or add a shared FlagParser.usage/2 that renders the spec's own flag list) and exit 0. It costs little, makes the emitted advice honest, and turns the spec into the discoverability surface the issue is asking for.


6. A new test passes on a run that returns exit 0, empty stdout and empty stderr

minortest/commands/unknown_flags_test.exs:129

test "an option-shaped operand after -- is not rejected as a flag" do
  {result, _} = JustBash.exec(bash(), "sort -- -Q")
  refute result.stderr =~ "invalid option"
end

sort -- -Q actually returns exit 0, stdout "", stderr "" — because get_content/3 (lib/just_bash/commands/sort.ex:58) maps {:error, _} from FS.read_file to {"", bash.fs}. GNU: exit 2, sort: cannot read: -Q: No such file or directory.

The unreadable-file behaviour is a genuinely separate bug and the PR discloses it under "Left out" — not asking for it here. The problem is the test: the headline symptom of issue #68 is still reproducible one -- away from the command the issue quotes, and a test added by this PR walks straight over it without noticing. A refute … =~ against a run that produced no stderr at all asserts nothing.

Suggested fix: assert the positive — that -Q reached sort as an operand, e.g. by pointing it at a file that exists (sort -- /f.txt with the expected sorted stdout), so the test fails if -- handling breaks.


7. sort's parse_key_spec/1 integer clause is now unreachable

minorlib/just_bash/commands/sort.ex:111

defp parse_key_spec(spec) when is_integer(spec), do: {spec, nil}

This existed because the old parse_value/1 coerced every integer-looking value, so sort -k 2 produced [2]. With coercion now gated on :integer and sort's @flag_spec declaring no :integer key, :k values are always binaries; try_numeric_flag/4 only ever writes :n, which sort declares boolean. Verified: sort -k 2 /f.txt and sort -k2 /f.txt both now reach the is_binary clause.

Dialyzer will not flag it (the guard is satisfiable in principle) and no test covers it, so it will sit there implying sort still accepts integer key specs. It also returns {spec, nil}, which the caller indexes as modifiers[:numeric] — safe only by accident, since nil[:numeric] would raise if it were ever reached.

Suggested fix: delete the clause in this PR, since this PR is what made it dead.


Considered and dismissed

"Per-command rejection tests are hand-written one case per command rather than enumerated" (test/commands/unknown_flags_test.exs:15) — dismissed as duplicative. The complaint is real in the abstract (growth-by-example, per issue #70), but every concrete cell it names is already reported above with a traced failure: the missing cluster-with-a-value-flag row is finding 1, and the weak enumeration is finding 2. Restating the shape of the table adds no new information, and "these seven tests should have been a table" without a distinct failing input is a structural preference. Verified separately that the coverage gaps it speculates about are not live bugs: cp -R /f.txt /g.txt and cp --recursive /f.txt /h.txt both exit 0 through the new :aliases merge, so the alias path is untested but not broken.

Two duplicate pairs were merged rather than dropped. Both lenses independently found the cluster-offender bug (flag_parser.ex:213) — merged into finding 1, keeping the sort -rt: -k2 reproduction, which is sharper than sort -nk2 because both flags in it are implemented. Both independently found the registry-matrix weakness — merged into finding 2, keeping the stdout-non-empty evidence and adding the baseline classification run, which is the decisive proof. Both independently found the :integer non-validation — merged into finding 4, keeping the observation that a new test blesses the fallback.

Nothing was refuted outright as factually wrong: every behavioural claim reproduced exactly as described, on the branch and on 82ee9e5.

@davydog187

Copy link
Copy Markdown
Collaborator Author

All seven findings reproduced on 52562f1 before anything was touched, and all seven are fixed. Nothing disputed. Six commits, one per finding or tight group; every fix has a test that fails on 52562f1 and passes now.

# Finding Resolution Commit
1 Cluster containing an implemented value flag is rejected, diagnostic names an implemented flag fixed 8c95435
2 Registry matrix's :strict bucket cannot tell rejection from renaming the flag to a file fixed ef72579
3 tr's new -- clause opens a fresh exit-0 / empty-output path fixed 55342c1
4 :integer declared but never validated, head -nq raises out of exec/2 fixed 3dcbd0e
5 --help is a hard error on all eight commands, and the advice printed is a command that fails fixed f2f4629
6 New test passes on a run that returns exit 0, empty stdout and empty stderr fixed cf3f9f8
7 sort's parse_key_spec/1 integer clause is unreachable dead code fixed cf3f9f8

1 — clusters (8c95435)

parse_cluster/5 now walks a cluster the way getopt does: booleans accumulate, the first value flag takes the rest of the cluster as its argument or the next argv when it is the last character, and the scan stops on the first character the spec does not describe at all. The diagnostic can no longer name a flag the command implements.

command before now GNU
sort -nk2 /f.txt exit 2, invalid option -- 'k' exit 0, a 1\nb 2\n exit 0, sorted
sort -rt: -k2 /f.txt exit 2, invalid option -- 't' exit 0, sorted exit 0, sorted
sort -rk 2 /f.txt exit 2 exit 0 (value flag ends cluster, takes next argv) same
sort -nQk2 /f.txt exit 2, invalid option -- 'Q' same

New tests use specs that carry :value/:multi_value flags, which the old cluster tests never did.

2 — registry matrix (ef72579)

probe/2 now requires empty stdout and a diagnostic that names an option (invalid|unrecognized|illegal option) before it calls a command :strict. A fourth bucket, :misblamed, holds the ~20 commands that exit non-zero only because the demoted flag named a file that does not exist.

The decisive check the review asked for — the strengthened classifier run against 82ee9e5:

cp   => :misblamed     head => :misblamed     ls   => :misblamed     tail => :misblamed
sort => :exit_zero     uniq => :exit_zero     grep => :quiet         tr   => [:misblamed, :exit_zero]

All eight are outside :strict on the unfixed tree, where the old classifier certified four of them. The matrix test fails on 82ee9e5 for every caller the branch claims to fix. A new test also pins those eight names to :strict in the table, so the table cannot be edited to quietly demote one.

3 — tr arity (55342c1)

One set with neither -d nor -s is now an error rather than a silent fall-through to defp run(_input, _opts), do: "". This closes the new -- route and the pre-existing one (tr x) together:

$ echo abc | tr -- -d
tr: missing operand after '-d'
Two strings must be given when translating.
Try 'tr --help' for more information.          (exit 1, matching GNU)

tr with no sets at all now prints the Try --help line too, as GNU does.

4 — :integer validation (3dcbd0e)

A bad count is now the third getopt error class, {:invalid_value, label, value}, worded as GNU words it, with no Try --help line because GNU prints none for this one:

$ head -n abc /f.txt     head: invalid number of lines: 'abc'   (exit 1)
$ head -nq  /f.txt       head: invalid number of lines: 'q'     (exit 1)
$ head -n '' /f.txt      head: invalid number of lines: ''      (exit 1)
$ head -c xy /f.txt      head: invalid number of bytes: 'xy'    (exit 1)

All three of the first ones previously raised a FunctionClauseError out of JustBash.exec/2; the fourth printed the whole file at exit 0. The noun comes from a new :value_labels spec key, required of every :integer flag. test/flag_parser_test.exs:172 — the test that locked the old fallback in — now asserts the error.

5 — --help (f2f4629)

FlagParser.parse/2 returns :help for --help, and FlagParser.help/2 renders the usage from the spec, so it lists exactly what the parser accepts and cannot drift:

$ sort --help
Usage: sort [OPTION]... [FILE]...
Options this shell implements:
  -f
  -k VALUE
  -n
  -r
  -t VALUE
  -u

All eight commands answer at exit 0, tr included. -- still wins, so sort -- --help is an operand. --version stays an unrecognized option, and the advice it prints is now honest.

6 — the sort -- -Q test (cf3f9f8)

The test was a refute ... =~ against a run with no stderr at all, so it asserted nothing. Fixing it required fixing sort: get_content/3 mapped every read error to {"", fs}.

$ sort -- -Q      sort: cannot read: -Q: No such file or directory   (exit 2, matching GNU)
$ sort -- /f.txt  exit 0, sorted

A lone - is now standard input rather than a file named -, which is both GNU's behaviour and what keeps printf 'b\na\n' | sort - meaningful; that test now asserts the sorted output instead of only the exit code.

7 — dead clause (cf3f9f8)

parse_key_spec/1's is_integer clause removed. Verified unreachable (sort -k 2 and sort -k2 both reach the is_binary clause), and it returned {spec, nil} where the caller does modifiers[:numeric], so it was wrong as well as dead.


Gates

mix compile --warnings-as-errors   Compiling 168 files (.ex) / Generated just_bash app   (no warnings)
mix format --check-formatted       clean
mix credo --strict                 5149 mods/funs, found 1 refactoring opportunity
                                   (test/support/banned_fixture_apply.ex - the intentional fixture)
mix test                           2 doctests, 62 properties, 4818 tests, 0 failures (5 excluded)
mix dialyzer                       Total errors: 13, Skipped: 13, Unnecessary Skips: 0 - passed

Noted, not fixed (out of scope for these findings)

  • echo abc | tr -cs a still returns "" at exit 0 (GNU prints abc), and tr a b c / tr -d a b still reach run/2's catch-all where GNU says extra operand. Pre-existing, not reachable through the -- clause this branch added, so left alone.
  • test/property_test.exs:423 ("an option date does not implement never exits 0") is flaky on this branch, independent of these changes: date -Rf exits 1 with the usage: date [-u] ... block, which does not contain the date: prefix the property asserts. Reproduced identically on 52562f1 before any of this work. date does not use FlagParser; belongs to the fix: date rejects unknown arguments instead of returning today at exit 0 #66 line of work.
  • comm, env, nl, readlink and which classify :strict but word the short option as invalid option -- '-Z', duplicating GNU's dash. Genuine rejections, so the bucket is right; the wording is a separate nit.

@davydog187

Copy link
Copy Markdown
Collaborator Author

Verification of review fixes

Independently verified at ef72579 (origin/fix/issue-68-strict-unknown-flags), fresh checkout, all
repros re-run, all oracles re-checked against GNU coreutils 9.11 (gsort/ghead/gtail/gtr) on
this machine. All seven findings hold up. One new minor problem introduced by the fix commits.

Per-finding

# Finding Verdict Evidence
1 Cluster containing an implemented value flag is rejected fixed sort -nk2 /f2.txtexit=0 out="y 2\nx 10\nz 33\n" err=""; sort -rt: -k2 /f3.txtexit=0 out="a:3\nc:2\nb:1\n"; sort -rk 2 /f2.txtexit=0 out="z 33\ny 2\nx 10\n" — byte-identical to gsort on the same fixtures. sort -nQk2 /f2.txtexit=2 "sort: invalid option -- 'Q'", same as gsort (exit=2, same text). Mutation: replacing the flag_atom in value_flags(spec) branch of parse_cluster/5 with an unreachable guard → 12 failures in flag_parser_test.exs + unknown_flags_test.exs.
2 Matrix :strict bucket cannot tell rejection from demotion fixed Exported 82ee9e5 with git archive, ran the new probe/2 logic there: cp:misblamed (cp: missing destination file operand after '-Z'), head/tail:misblamed (cannot open '-Z' for reading), ls:misblamed (cannot access '-Z'), grep:quiet (rc=1, empty stderr), sort/uniq:exit_zero, tr:misblamed/:exit_zero. None of the eight is :strict on baseline — the review's decisive test. Copied the new test file into the baseline tree: 3 tests, 2 failures. Mutation on HEAD: weakening rejected_the_option?/1 back to stdout == "" or stderr != "" → the whole-registry equality test goes red. The classifier is load-bearing.
3 tr's -- clause opens a fresh exit-0 / empty-output path fixed echo abc | tr -- -dexit=1 err="tr: missing operand after '-d'\nTwo strings must be given when translating.\nTry 'tr --help' for more information.\n"; echo abc | tr -- x and echo abc | tr x identical (with x). gtr prints the same three lines at rc=1 (only difference: GNU's curly quotes). echo abc | tr -d bexit=0 "ac\n", so one set still suffices when deleting. Mutation: guarding the new one-set clause off → 2 failures.
4 :integer declared but never validated → head -nq raises fixed Nothing raises. head -n abc /f.txtexit=1 "head: invalid number of lines: 'abc'\n"; head -nq'q'; head -n ''''; head -c xy"head: invalid number of bytes: 'xy'\n"; tail -n abc"tail: invalid number of lines: 'abc'\n". ghead/gtail print exactly these at rc=1 (curly quotes) with no Try --help line — matched. head -n 2 /f.txt still "a\nb\n". Mutation: parse_count/3 returning {:ok, value} on garbage → 6 failures.
5 --help/--version are hard errors advising a failing command fixed (as scoped) All eight exit 0 with a real usage block: sort --help"Usage: sort [OPTION]...\nOptions this shell implements:\n -f\n -k VALUE\n -n\n -r\n -t VALUE\n -u\n". sort -Q /f.txt still advises Try 'sort --help', and following that advice now works. sort -- --help stays an operand (exit=2 "sort: cannot read: --help: No such file or directory"). --version is still unrecognized option — the fixer says so explicitly; GNU exits 0 there. Mutation: disabling the -help branch → 4 failures.
6 New test passed on exit 0 / empty stdout / empty stderr fixed sort -- -Qexit=2 out="" err="sort: cannot read: -Q: No such file or directory\n" — byte-identical to gsort -- -Q. The test now asserts exit code, stdout and the exact stderr. printf 'b\na\n' | sort -exit=0 "a\nb\n" (was "" at 82ee9e5). Mutation: restoring {:error, _} -> {:ok, "", bash.fs} in get_content/3 → 2 failures.
7 sort's parse_key_spec/1 integer clause is dead code fixed Clause gone (git diff 82ee9e5..HEAD -- lib/just_bash/commands/sort.ex). @flag_spec still declares no :integer, so :k values stay binaries. Exercised through both spellings and clusters: sort -k 2, sort -k2, sort -nk2, sort -rk 2, sort -k2,2n, sort -ruk1,1n all exit 0 with the expected bytes.

Gates (re-run here, not reported)

mix compile --warnings-as-errors  Compiling 168 files (.ex) / Generated just_bash app   PASS
mix format --check-formatted      exit 0                                                PASS
mix credo --strict                5149 mods/funs, found 1 refactoring opportunity       PASS
                                  (only test/support/banned_fixture_apply.ex:4 — the
                                   intentional fixture)
mix test                          2 doctests, 62 properties, 4818 tests, 0 failures
                                  (5 excluded)                                          PASS
mix dialyzer                      Total errors: 13, Skipped: 13, Unnecessary Skips: 0   PASS

New problem introduced by the fix commits

[minor] sort <dir> now reports an existing directory as nonexistent (cf3f9f8,
lib/just_bash/commands/sort.ex:73). get_content/3 maps every FS.read_file error to
"No such file or directory".

sort /d   (with /d/inner present)
  HEAD:     exit=2 err="sort: cannot read: /d: No such file or directory\n"
  gsort:    exit=2 err="gsort: read failed: dtest: Is a directory"
  82ee9e5:  exit=0 out="" err=""

The exit code is now right and this is a clear improvement over swallowing the error, but the
diagnostic asserts something false about a path that exists — the same "blame something that isn't
the problem" failure mode this PR is about. sort /nope is correct (cannot read: /nope: No such file or directory, matching gsort exactly); only the directory case is wrong.

Noted, not counted against the fix

echo abc | tr -- a b cexit=0 out="" err="" (GNU: tr: extra operand 'c', rc=1). The new
arity check covers one set but not three. Not a regression: echo abc | tr a b c gives the same
exit-0/empty at 82ee9e5, so the -- clause is landing on a pre-existing hole rather than opening
one. Worth a follow-up since it is the same shape as finding 3.

@davydog187

Copy link
Copy Markdown
Collaborator Author

Closed the last open item: sort reported an existing directory as "No such file or directory".

What was wrong

cf3f9f8 made sort report an operand it cannot read (good — it used to sort nothing at exit 0), but get_content/3 mapped every FS.read_file error onto the ENOENT wording. Exit 2 was already correct; the diagnostic was the problem — it stated something false about a path that exists, which is its own kind of misleading feedback for an agent trying to learn what went wrong.

Before (ef72579)

$ sort /d          # /d/inner exists
sort: cannot read: /d: No such file or directory     (exit 2)

$ sort /f.txt/sub
sort: cannot read: /f.txt/sub: No such file or directory     (exit 2)

After (f624597)

$ sort /d
sort: read failed: /d: Is a directory                        (exit 2)

$ sort /nope
sort: cannot read: /nope: No such file or directory          (exit 2)

$ sort /f.txt/sub
sort: cannot read: /f.txt/sub: Not a directory               (exit 2)

Oracle

GNU coreutils 9 gsort, run directly rather than taken from the review summary — coreutils genuinely uses two templates, and which one it picks says where the failure happened:

$ gsort dtest        ->  rc=2  gsort: read failed: dtest: Is a directory
$ gsort /nope        ->  rc=2  gsort: cannot read: /nope: No such file or directory
$ gsort ff.txt/sub   ->  rc=2  gsort: cannot read: ff.txt/sub: Not a directory
$ gsort perm/f.txt   ->  rc=2  gsort: cannot read: perm/f.txt: Permission denied   (chmod 000)

A directory opens fine and only fails once sort reads it, so it gets read failed:; everything else fails at open and gets cannot read:. All three of our outputs now match byte for byte modulo the program name.

The change

lib/just_bash/commands/sort.ex — one case arm plus a two-clause read_error/2:

defp read_error(file, %VFS.Error{kind: :eisdir} = error),
  do: "sort: read failed: #{file}: #{FS.strerror(error)}\n"

defp read_error(file, error), do: "sort: cannot read: #{file}: #{FS.strerror(error)}\n"

The reason comes from FS.strerror/1 in both templates, so a kind we have not enumerated (:eacces is not reachable from the in-memory read_file today) still reports itself accurately instead of being relabelled ENOENT. Deliberately kept to the diagnostic wording only, to limit the conflict with #74, which is extending this same read path on its own branch.

test/commands/text_processing_test.exs — the three error kinds FS.read_file can return are now one table asserted as a whole, not three separate cases, so a fourth kind cannot arrive wearing the wrong template:

@unreadable_operands [
  {:enoent, "/nope", "sort: cannot read: /nope: No such file or directory\n"},
  {:eisdir, "/d", "sort: read failed: /d: Is a directory\n"},
  {:enotdir, "/file.txt/sub", "sort: cannot read: /file.txt/sub: Not a directory\n"}
]

Test-first: the table failed on :eisdir and :enotdir against ef72579 before the fix landed.

Gates

mix compile --warnings-as-errors   clean
mix format --check-formatted       clean
mix credo --strict                 1 finding — test/support/banned_fixture_apply.ex, the intentional fixture
mix test                           2 doctests, 62 properties, 4819 tests, 0 failures (5 excluded)
mix dialyzer                       Total errors: 13, Skipped: 13, Unnecessary Skips: 0 — done (passed successfully)

@davydog187

Copy link
Copy Markdown
Collaborator Author

Final verification

Independent re-verification of the third round on fix/issue-68-strict-unknown-flags.
Head verified: f624597 ("sort names the real reason an operand could not be read").
Compared against the branch's previous head ef72579 and against origin/main (82ee9e5).
Oracle: GNU coreutils 9.11 gsort on this machine — re-derived from scratch, not taken from the summary.

Per-item result

# Item Verdict Evidence
1 sort reports an existing directory as "No such file or directory" fixed sort /d (with /d/inner present) → rc=2, stderr sort: read failed: /d: Is a directory\n. Oracle gsort dtestrc=2, gsort: read failed: dtest: Is a directory. Byte-for-byte match modulo the program name.

The GNU templates, re-derived here

I checked each error kind against gsort myself rather than trusting the claim. The claimed rule
(open failure → cannot read:, read failure → read failed:, reason always from strerror) holds:

input gsort JustBash at f624597 match
directory rc=2 gsort: read failed: dtest: Is a directory rc=2 sort: read failed: /d: Is a directory yes
missing rc=2 gsort: cannot read: /nope: No such file or directory rc=2 sort: cannot read: /nope: No such file or directory yes
ENOTDIR rc=2 gsort: cannot read: ff.txt/sub: Not a directory rc=2 sort: cannot read: /f.txt/sub: Not a directory yes
unreadable file (chmod 000) rc=2 gsort: cannot read: perm/f.txt: Permission denied n/a in the in-memory FS; the fallback clause would produce the same wording via FS.strerror/1
symlink cycle rc=2 gsort: cannot read: loopa: Too many levels of symbolic links rc=2 sort: cannot read: /loopa: Too many levels of symbolic links yes
dangling symlink rc=2 gsort: cannot read: dang: No such file or directory rc=2 sort: cannot read: /dang: No such file or directory yes
symlink → directory rc=2 gsort: read failed: dlink: Is a directory rc=2 sort: read failed: /dlink: Is a directory yes
. and / rc=2 gsort: read failed: .: Is a directory rc=2 sort: read failed: .: Is a directory (same for /, //d, /d/../d) yes

The symlink-cycle row is worth calling out: :eloop is a fourth kind FS.read_file can return
(lib/just_bash/fs/memory.ex:474), not one of the three the new test enumerates. The generic
read_error/2 clause handles it correctly and matches gsort exactly — the design of routing
through FS.strerror/1 is doing its job. See the minor note below about the enumeration comment.

The new test can fail

Mutated the fix out twice and confirmed test/commands/text_processing_test.exs:955 goes red each time.

  1. Whole fix reverted (read_error/2 collapsed back to the hardcoded ENOENT string) — red on two
    rows at once:
    left:  enotdir: {2, "", "sort: cannot read: /file.txt/sub: No such file or directory\n"},
           eisdir:  {2, "", "sort: cannot read: /d: No such file or directory\n"}
    right: enotdir: {2, "", "sort: cannot read: /file.txt/sub: Not a directory\n"},
           eisdir:  {2, "", "sort: read failed: /d: Is a directory\n"}
    
  2. Only the :eisdir clause deleted, FS.strerror/1 kept — still red, on precisely the row that
    distinguishes the two templates:
    left:  eisdir: {2, "", "sort: cannot read: /d: Is a directory\n"}
    right: eisdir: {2, "", "sort: read failed: /d: Is a directory\n"}
    
    This is the important one: it proves the test pins the template choice, not just the strerror
    reason. A fix that only got the reason right would not pass.

Working tree restored to a clean f624597 after both mutations.

Regression sweep

Ran a 78-command corpus through JustBash.exec/2 at f624597, at ef72579, and at origin/main,
and diffed the full {exit_code, stdout, stderr} triple for every command.

f624597 vs ef72579: the only differences in the entire corpus are the eight sort diagnostics
this commit intends to change.
Every exit code is byte-identical; no stdout anywhere changed; no
other command's stderr changed. Diff excerpt:

-err="sort: cannot read: /d: No such file or directory\n"
+err="sort: read failed: /d: Is a directory\n"
-err="sort: cannot read: /f.txt/sub: No such file or directory\n"
+err="sort: cannot read: /f.txt/sub: Not a directory\n"
-err="sort: cannot read: /loopa: No such file or directory\n"
+err="sort: cannot read: /loopa: Too many levels of symbolic links\n"
...

What was swept and found clean:

  • sort happy pathssort FILE, -r, -u, -n, -nr, -k1,1, -t: -k1, empty file,
    file inside a directory, through a symlink, sort - , bare sort on a pipe, sort < FILE,
    here-string. All unchanged, all exit 0.
  • sort flag surface (the rest of this PR) — sort --help, sort -Q, sort --bogus,
    sort -- -Q. Unchanged from ef72579; -- -Q correctly reaches the read path and reports
    cannot read: -Q: No such file or directory at exit 2, matching gsort.
  • - operands and stdin redirectionsort -, echo hi | sort -, sort < /f.txt,
    sort < /d, sort < /nope. Unchanged (get_content/3's [] and ["-" | _] clauses are
    untouched, and the sweep confirms it empirically).
  • Pipelines and exit-code propagationsort /d | wc -l, sort /f.txt | head -1,
    cat /f.txt | sort | uniq | wc -l, x=$(sort /d 2>/dev/null), sort /f.txt > /out.txt,
    sort /d > /out2.txt. Unchanged.
  • set -e and conditionalsset -e; sort /d; echo AFTER halts at exit 2 with no AFTER
    (same for /nope); sort /d || echo fallback, sort /d && echo ok, if sort /d; then...,
    sort /d; echo rc=$?. All identical to ef72579.
  • Every neighbouring command that takes a file operandcat, head, tail, wc, uniq,
    cut, grep, tac, nl, sed, awk, tr, ls, cp, rm, tee, each on a directory, a
    missing path and a good file. Byte-identical across all three heads. (Several of these have their
    own pre-existing divergences from GNU — head/tail/wc/tac/nl say "No such file or
    directory" for a directory, uniq/cut swallow it at exit 0 — but they are unchanged by this
    commit and out of scope here.)
  • Special paths/dev/null, /dev/stdin (both ENOENT in the in-memory FS, unchanged at all
    three heads), /, ., //d, /d/../d, trailing slash.

One thing the sweep surfaced that is not attributable to this PR, recorded so it is not
re-discovered as a regression: cmd >/dev/null 2>&1 does not suppress stderr — it leaks into
stdout. Reproduced at origin/main with commands this branch never touches:

if cat /nope >/dev/null 2>&1; then echo yes; else echo no; fi
  -> out="cat: /nope: No such file or directory\nno\n"
if ls  /nope >/dev/null 2>&1; then echo yes; else echo no; fi
  -> out="ls: cannot access '/nope': No such file or directory\nno\n"

It only looks new for sort because before cf3f9f8 sort /d printed nothing at all. Pre-existing
redirection bug, separate issue.

Gates (re-run at f624597, clean tree)

mix compile --warnings-as-errors --force   Generated just_bash app          (clean)
mix format --check-formatted               exit 0
mix credo --strict                         5151 mods/funs, found 1 refactoring opportunity
                                           (test/support/banned_fixture_apply.ex — the intentional fixture)
mix test                                   Finished in 25.1 seconds (24.8s async, 0.2s sync)
                                           2 doctests, 62 properties, 4819 tests, 0 failures (5 excluded)
mix dialyzer                               Total errors: 13, Skipped: 13, Unnecessary Skips: 0 — done (passed successfully)

Two minors, neither blocking

  1. The test's enumeration is not exhaustive, and its comment says it is. The comment above
    @unreadable_operands reads "the three error kinds FS.read_file can return"; :eloop is a
    fourth, reachable from sort today:

    ln -s /loopb /loopa; ln -s /loopa /loopb; sort /loopa
      -> rc=2, "sort: cannot read: /loopa: Too many levels of symbolic links\n"
    gsort loopa
      -> rc=2, "gsort: cannot read: loopa: Too many levels of symbolic links"
    

    The behaviour is correct and matches GNU — this is a coverage/wording gap, not a defect. A
    fourth row {:eloop, ...} in the table (and dropping "three" from the comment) would make the
    matrix actually closed, which was the stated point of using a table. One line, no conflict risk
    with fix: filesystem commands name the actual error kind, and the Oils spec parser reads the format #74.

  2. sort '' moved further from GNU, via a pre-existing root cause.

    sort ''      f624597 -> rc=2, "sort: read failed: : Is a directory\n"
                 ef72579 -> rc=2, "sort: cannot read: : No such file or directory\n"
    gsort ''             -> rc=2, "gsort: cannot read: '': No such file or directory"
    

    The reason string used to be accidentally right and is now wrong. The cause is not this commit's
    logic: FS.resolve_path(cwd, "") resolves an empty operand to the cwd, so the FS genuinely
    answers :eisdir and sort now reports it faithfully. cat '' already says
    cat: : Is a directory at origin/main, so this is a shared resolve_path issue that the more
    accurate diagnostic merely exposes — sort is now consistent with cat. Fixing it belongs in the
    path layer, not in sort.ex, and doing it here would violate the "keep it narrow for fix: filesystem commands name the actual error kind, and the Oils spec parser reads the format #74" constraint.

Conclusion

The reported finding is genuinely fixed against an independently re-derived GNU oracle, the new test
provably fails without the fix (including under the subtle single-clause mutation), the change is
narrow — 11 lines in sort.ex plus one test block — and a full behavioural diff against the previous
head shows nothing changed except the eight intended diagnostics. All five gates pass.

@davydog187

Copy link
Copy Markdown
Collaborator Author

Round 4: :eloop was missing from a table that claimed to be closed

Last remaining item from the verifier's review. 29ad8ff, test-only — no production code touched.

What was wrong

The comment above @unreadable_operands said the table held "the three error kinds FS.read_file
can return"
. There is a fourth: :eloop, constructed at lib/just_bash/fs/memory.ex:474 and
reachable from sort today.

The behaviour was already correct — the generic read_error/2 clause plus FS.strerror/1
handle it, and it matches GNU exactly:

$ ln -s /loopb /loopa; ln -s /loopa /loopb; sort /loopa
  rc=2  "sort: cannot read: /loopa: Too many levels of symbolic links\n"

$ gsort loopa                                    # GNU coreutils 9, same seed on disk
  gsort: cannot read: loopa: Too many levels of symbolic links
  rc=2

The defect was in the test: a table that asserts exhaustiveness while missing a reachable kind is
exactly the "absent case is indistinguishable from a passing case" problem #70 was filed about.

What changed

Adding the row alone would leave the fifth kind to the same hand-count that lost the fourth, so the
kind list is now derived, not transcribed. Two readings out of the compiled beams:

  1. the literal atoms in FS.strerror/1's clause heads (every kind that has a message), and
  2. every literal atom the memory backend — JustBash.FS.Memory and its VFS.Mountable impl,
    located via impl_for! rather than named — passes to VFS.Error.new/2.

A new test fails unless every kind in that union is either a row in @unreadable_operands or listed
in @kinds_read_file_cannot_return with what does produce it. Kinds sort cannot be handed are
named with the reason rather than omitted:

kind why read_file/2 cannot return it
:eexist mkdir/symlink/link refusing a name that is already taken
:enotempty rmdir on a directory that still has entries
:eacces Memory.link/3 hard-linking a non-file; no read path consults mode bits
:erofs a read-only mount, which this FS has no way to declare
:enotsup the POSIX shim's symlink/link stubs on backends that have neither
:einval readlink on a non-symlink, and the backend's one computed-kind Error.new/2
:exdev a rename that crosses a mount boundary
:eio strerror names it for completeness; no backend here constructs it

The backend has exactly one Error.new/2 whose kind is computed rather than literal — stream_read/3
forwarding a VFS.StreamOptions rejection, which read_file/2 cannot trigger because it passes no
options. The test pins that count at 1, so a second such site fails loudly instead of silently
shrinking what the derivation can see.

Before / after

The new test is genuinely red without the row — it was written first and watched fail:

BEFORE (:eloop row absent, derivation in place)
  1) test sort command the unreadable-operand table accounts for every error kind that exists
     unaccounted: MapSet.new([:eloop]), stale: MapSet.new([])
     code: assert MapSet.union(tabled, accounted) == universe
  12 tests, 1 failure

AFTER (:eloop row added)
  12 tests, 0 failures

Mutations — the derivation itself is load-bearing

Three mutations, each reverted after:

1. New strerror clause, nothing else     def strerror(:enospc), do: "No space left on device"
   -> RED: unaccounted: MapSet.new([:enospc])

2. Backend constructs an unlisted kind   Error.new(:eisdir,…) -> Error.new(:enospc,…) in stream_read/3
   -> RED: unaccounted: MapSet.new([:enospc])      (arm 2 works without arm 1)

3. A second computed-kind call site      Error.new(:enotdir,…) -> Error.new(hd([:enotdir]),…)
   -> RED: assert Enum.count(constructed, &(&1 == :computed_at_runtime)) == 1
           left: 2  right: 1

Mutation 2 matters most: it fails on a kind strerror has no clause for, so the two derivations are
independently effective rather than one shadowing the other.

Gates — all five, at 29ad8ff, clean tree

mix compile --warnings-as-errors --force   Compiling 168 files (.ex) / Generated just_bash app
mix format --check-formatted               exit 0
mix credo --strict                         5160 mods/funs, found 1 refactoring opportunity
                                           (test/support/banned_fixture_apply.ex — the intentional fixture)
mix test                                   2 doctests, 62 properties, 4820 tests, 0 failures (5 excluded)
                                           Finished in 22.4 seconds
mix dialyzer                               Total errors: 13, Skipped: 13, Unnecessary Skips: 0
                                           done (passed successfully)

4819 -> 4820 tests: the one new closure test. Credo's first run flagged
Nested modules could be aliased on the new JustBash.FS.Memory reference; fixed with an alias, and
the run above is back to the pre-existing baseline finding only.

Deliberately not fixed here

sort '' -> "sort: read failed: : Is a directory" where GNU says
"cannot read: '': No such file or directory". Root cause is FS.resolve_path(cwd, "") resolving an
empty operand to the cwd — pre-existing and shared (cat '' already misreports at origin/main), so
it belongs in the path layer and would collide with #74.

@davydog187

Copy link
Copy Markdown
Collaborator Author

Final verification

Independent re-run at 29ad8ff in a clean worktree. Nothing taken on the fixer's word — every repro
re-executed, every claim of test strength re-derived by mutating it out.

Per-item

# Item Claimed Verified Evidence
1 :eloop missing from a table that claims exhaustiveness fixed fixed Behaviour and coverage both confirmed; five mutations prove the new test is load-bearing (below)
2 sort '' -> read failed: : Is a directory not_done dispute upheld Pre-existing and shared; identical misreport at origin/main. Out of scope per the task

1. :eloop

Oracle, GNU coreutils 9.11 (gsort), in a dir holding a two-hop cycle loopa -> loopb -> loopa:

$ gsort loopa
gsort: cannot read: loopa: Too many levels of symbolic links
rc=2

JustBash at 29ad8ff, cycle seeded with ln -s /loopb /loopa; ln -s /loopa /loopb:

$ sort /loopa
  rc=2 out="" err="sort: cannot read: /loopa: Too many levels of symbolic links\n"

Byte-identical modulo the operand spelling. The other three rows were re-checked against gsort in
the same run and all match:

operand gsort JustBash
/nope cannot read: …: No such file or directory, rc=2 same
/d (dir) read failed: …: Is a directory, rc=2 same
/file.txt/sub cannot read: …: Not a directory, rc=2 same
/loopa cannot read: …: Too many levels of symbolic links, rc=2 same

The derivation is real, not decorative. I independently reproduced the two readings the new test
performs: FS.strerror/1 has 12 literal clauses (lib/just_bash/fs/fs.ex:269-280), and
lib/just_bash/fs/memory.ex has 22 Error.new/2 sites of which exactly one (:603, stream_read/3
forwarding a VFS.StreamOptions rejection) computes its kind. 4 tabled + 8 excused = 12 = the
universe. I also grepped every Error.new/2 outside the memory backend (fs.ex, posix.ex,
cp.ex, redirection.ex): the kinds are eisdir, enoent, eloop, einval, enotdir, exdev, enotsup
all already inside the derived universe, so the derivation is not narrower than reality.

Mutation testing — five independent mutations, each red, each reverted:

Mutation Result
Delete the :eloop row from @unreadable_operands RED: unaccounted: MapSet.new([:eloop])
Delete the @symlink_cycle seed from the behaviour test RED: observed …: No such file or directory vs expected …: Too many levels of symbolic links — the row is exercised, not decorative
Add def strerror(:enospc), … to JustBash.FS RED: unaccounted: MapSet.new([:enospc])
Make the backend construct a kind strerror has no clause for (:eloop -> :emlink at memory.ex:474) RED on both tests: unaccounted: MapSet.new([:emlink])
Add a second computed-kind call site (Error.new(hd([:eisdir]), …)) RED: assert Enum.count(constructed, &(&1 == :computed_at_runtime)) == 1 — left 2, right 1

Worth stating the one boundary the derivation does have: it reads JustBash.FS.Memory and its
VFS.Mountable impl. A kind constructed only by some other backend and absent from strerror/1
would slip through. No such kind exists today (checked above), and strerror/1's is_atom catch-all
means such a kind would render as a bare atom rather than a wrong sentence.

2. sort '' — dispute upheld

PR head:      $ sort ''  ->  rc=2, "sort: read failed: : Is a directory\n"
origin/main:  $ cat ''   ->  misreports identically

FS.resolve_path(cwd, "") resolves the empty operand to the cwd, so the FS honestly answers
:eisdir. Path-layer bug, shared by every command, would collide with #74. Correctly left alone.

Gates (all re-run here, not quoted)

Gate Result
mix compile --warnings-as-errors --force pass, 0 warnings
mix format --check-formatted pass
mix credo --strict 1 finding: test/support/banned_fixture_apply.ex:4:16 — the intentional fixture, present at origin/main too
mix test 2 doctests, 62 properties, 4820 tests, 0 failures (5 excluded)Finished in 26.2 seconds
mix dialyzer pass — Total errors: 13, Skipped: 13, Unnecessary Skips: 0

Sweep — what was covered, including what came back clean

Test-file timing, the stated concern for a test-only change:

tests wall
parent f624597 185 0.3s
head 29ad8ff 186 0.3s

One test added, no measurable runtime change, no shared fixture: the new symlink cycle lives in a
per-test in-memory FS, so async: true neighbours cannot see /loopa.

Operand-surface sweep: 25 invocations run at both head and origin/main and diffed — multi-operand
reads, operands starting with -, --, --help/--nosuch, and stdin-vs-file ordering across
sort, cat, head, tail, wc, uniq, tac, nl, tr, cut, paste. Every difference between the two is an
improvement this PR intended (sort /nope, sort -Q, sort --help, sort --nosuch,
echo x | sort -). Zero regressions.

Three things came back identical to origin/main and so are not findings against this PR, recorded
so the empty result is legible rather than assumed:

  • sort /a.txt /nope -> rc=0, sorts /a.txt, says nothing (GNU: rc=2 + diagnostic). sort reads only
    its first operand — get_content(bash, [file | _], _stdin) is the same shape at origin/main.
    Same for echo x | sort - /nope.
  • uniq /nope, cut -f1 /nope -> rc=0, silent. Identical at origin/main; the issue-68 family of
    fixes has not reached those commands yet.
  • tr a b /nope -> rc=0, silent (GNU: extra operand). Identical at origin/main.

Verdict: all clear. Item 1 is genuinely fixed and the coverage genuinely cannot pass without it;
item 2's deferral is correct.

JustBash.FlagParser demoted an unrecognised flag to a positional operand, so
`sort -Q /f.txt` and `uniq -Z /f.txt` read a file named `-Q`, found nothing,
and exited 0 with empty stdout and empty stderr — the file's contents dropped
and the caller told it succeeded.

parse/2 now returns {:ok, flags, rest} or {:error, {:unknown_flag, flag}} /
{:error, {:missing_value, flag}}, and all seven callers (cp, grep, head, ls,
sort, tail, uniq) render it with format_error/3 in GNU's shape: a short option
named by its character, a long option in full, followed by the Try --help line
and coreutils' exit code (2 for sort, ls and grep; 1 for the rest).

tr had its own copy: the combined-flag reducer discarded characters it did not
know, so `tr -dX b` deleted b and reported success, and a single unknown flag
like `-x` fell past the flag clauses and became a character set. It now rejects
both, and honours `--` so a dash-shaped character set still works.

parse_value/1 coerced every integer-looking value, which made `sort -t 1` a
delimiter of 1 and crashed String.split/2. Only flags declared :integer are
converted now; head and tail declare :n and :c.

Per the issue, the ~45 hand-rolled parse_args commands are left alone. The new
registry-wide probe enumerates every name in Commands.Registry against
`--jb-not-a-flag` and `-Z` and pins each one as :strict, :quiet, :operand or
:absorbed, so the list of commands that still swallow a flag can only shrink.
`sort -nk2` is `-n -k 2`, and `sort -rt: -k2` is `-r -t : -k 2`. The parser
only peeled an attached value off character 0 of a cluster, so both fell past
every clause into the unknown-flag path, where the offender was picked as "the
first character that is not a boolean". That named `k` and `t` - flags sort
implements - and rejected a command line GNU sorts without complaint.

`parse_cluster/5` now walks the cluster the way getopt does: booleans
accumulate, a value flag takes the rest of the cluster as its argument or the
next argument when it is the last character, and the scan stops on the first
character the spec does not describe at all. The diagnostic can no longer name
a flag the command has.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
`head -n abc`, `head -nq` and `head -n ''` raised a FunctionClauseError out of
`JustBash.exec/2` - an unhandled exception escaping a library whose job is
running untrusted shell safely. `head -c xy` was the quiet twin: exit 0 with
the whole file printed. `parse_value/3` conflated "this flag is not an integer
flag" with "this integer flag got garbage" and handed the raw string back to a
caller that had just declared the value a count.

`:integer` now validates. A bad count is the third getopt error class,
`{:invalid_value, label, value}`, worded as GNU words it:

    head: invalid number of lines: 'abc'

with no `Try --help` line, because GNU prints none for this one. The noun comes
from a new `:value_labels` spec key, required of every `:integer` flag, so the
message says what was being counted.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
The `--` clause pushed everything after the marker into `sets` without checking
how many sets tr needs, so `echo abc | tr -- -d` and `echo abc | tr -- x` left
one set with no -d/-s, fell past every `run/2` head into
`defp run(_input, _opts), do: ""`, and answered with empty stdout at exit 0.
That is the silent-empty result this branch exists to remove, reintroduced by
the clause that was meant to make `--` work.

One set with neither -d nor -s is now GNU's error, which also closes the
pre-existing route to the same catch-all (`echo abc | tr x`):

    tr: missing operand after 'x'
    Two strings must be given when translating.
    Try 'tr --help' for more information.

`tr` with no sets at all now prints the `Try --help` line too, as GNU does.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
`get_content/3` mapped every `FS.read_file` error to `{"", fs}`, so
`sort -- -Q` answered with empty stdout, empty stderr and exit 0 - issue #68's
exact symptom, still reachable one `--` away from the command the issue quotes.
The test added for that case only did `refute stderr =~ "invalid option"`
against a run that produced no stderr at all, so it asserted nothing.

sort now reports the unreadable operand the way GNU does, at GNU's exit code:

    sort: cannot read: -Q: No such file or directory     (exit 2)

A lone `-` is standard input rather than a file named `-`, which is both what
GNU does and what keeps `printf 'b\na\n' | sort -` meaningful; that test now
asserts the sorted output instead of only the exit code.

Also drops `parse_key_spec/1`'s `is_integer` clause. It was reachable only
while `parse_value/1` coerced every integer-looking value; with coercion gated
on `:integer` and sort declaring none, `:k` values are always binaries. It also
returned `{spec, nil}`, which the caller indexes as `modifiers[:numeric]`, so
it would have raised if anything had reached it.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
Every rejection this branch added ends with
`Try '<cmd> --help' for more information.`, and following that advice produced
the same class of error again:

    $ sort --help
    sort: unrecognized option '--help'
    Try 'sort --help' for more information.

Issue #68's motivation is the agent feedback loop - "the model can only learn
what a command supports by running it" - so closing one dead end and printing
directions to a second one is not much of a fix.

`FlagParser.parse/2` now returns `:help` for `--help`, and `help/2` renders the
usage out of the spec, which means it lists exactly what the parser accepts and
cannot drift from it:

    $ sort --help
    Usage: sort [OPTION]... [FILE]...
    Options this shell implements:
      -f
      -k VALUE
      -n
      -r
      -t VALUE
      -u

All eight commands answer, `tr` included. `--` still wins, so `sort -- --help`
is an operand. `--version` remains an unrecognized option, and the advice it
prints is now honest.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
…a failure

`probe/2` classified on exit code and stderr emptiness alone, so a command that
demoted the unknown flag to a filename and then failed to open it was
indistinguishable from one that rejected the flag - even though the table's own
comment calls :strict "what every option-parsing command should do".

Running that classifier against 82ee9e5, before any of this branch:

    ls   -Z  => :strict  (ls: cannot access '-Z': No such file or directory)
    head -Z  => :strict  (head: cannot open '-Z' for reading: ...)
    tail -Z  => :strict  (tail: cannot open '-Z' for reading: ...)
    cp   -Z  => :strict  (cp: missing destination file operand after '-Z')

Four of the seven FlagParser callers were already certified on the unfixed
tree, so the matrix passed for exactly the `head -q` / `ls -Q` half of issue
#68 - the half described as "the message names the wrong problem".

:strict now requires empty stdout and a diagnostic that names an option
(invalid / unrecognized / illegal option), and a fourth bucket, :misblamed,
holds the ~20 commands that are non-zero only because the demoted flag named a
file that does not exist. `cat -Z /f.txt` is the shape: exit 1,
`cat: -Z: No such file or directory`, and the file's contents still on stdout.

Against 82ee9e5 the strengthened classifier puts all eight shared-parser
commands outside :strict - cp/head/ls/tail :misblamed, sort/uniq :exit_zero,
grep :quiet, tr both - so the matrix now fails on the baseline for every caller
the branch claims to fix. A new test pins those eight to :strict so the table
cannot be edited to demote one.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
`get_content/3` mapped every `FS.read_file` error onto the ENOENT wording, so
`sort /d` on an existing directory answered

    sort: cannot read: /d: No such file or directory

which states something false about a path that exists. Exit 2 was already
right; only the diagnostic lied.

GNU sort uses two templates and the choice is informative: a directory opens
fine and fails when sort reads it, so coreutils says `read failed:`, while an
open failure says `cannot read:`. The reason now comes from `FS.strerror/1` in
both, so a kind we have not enumerated still reports itself accurately.

    sort /d           sort: read failed: /d: Is a directory
    sort /nope        sort: cannot read: /nope: No such file or directory
    sort /f.txt/sub   sort: cannot read: /f.txt/sub: Not a directory

All three match GNU coreutils 9 `gsort` byte for byte at exit 2, and the three
error kinds `FS.read_file` can return are asserted as one table rather than as
separate cases, so a fourth kind cannot arrive wearing the wrong template.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
…ng by hand

The comment above @unreadable_operands claimed the table held "the three
error kinds FS.read_file can return". There is a fourth. `:eloop` is
constructed at lib/just_bash/fs/memory.ex:474 and reaches sort today:

    ln -s /loopb /loopa; ln -s /loopa /loopb; sort /loopa
    -> rc=2, "sort: cannot read: /loopa: Too many levels of symbolic links"

which is byte-for-byte what GNU coreutils 9 gsort prints. The behaviour
was already right - the generic read_error/2 clause and FS.strerror/1
handle it. What was wrong was the table: it asserted exhaustiveness while
missing a reachable kind, so the absent case was indistinguishable from a
passing one (issue #70).

Adding the row alone would leave the next kind to the same hand-count, so
the kind list is now derived rather than transcribed. A new test reads two
things out of the compiled beams - the literal atoms in FS.strerror/1's
clause heads, and every literal atom the memory backend passes to
VFS.Error.new/2 - and fails unless each kind is either a row in the table
or listed in @kinds_read_file_cannot_return with what does produce it.
Kinds sort cannot be handed are named with the reason instead of omitted.

The backend has exactly one Error.new/2 whose kind is computed rather than
literal (stream_read/3 forwarding a VFS.StreamOptions rejection, which
read_file/2 cannot trigger because it passes no options); the test pins
that count at one so a second such site fails loudly rather than silently
shrinking what the derivation can see.

Test-only change; no production code touched.

Claude-Session: https://claude.ai/code/session_013UQWCDMJdSCZhJVkeFpN8z
@davydog187
davydog187 force-pushed the fix/issue-68-strict-unknown-flags branch from 29ad8ff to 9b8aee0 Compare August 6, 2026 15:38
@davydog187
davydog187 merged commit 9aab27a into main Aug 7, 2026
9 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unknown flags are silently absorbed as operands, so sort -Q and uniq -Z return empty output at exit 0

1 participant