diff --git a/CHANGELOG.md b/CHANGELOG.md index f5a1dd4e..c8f5626c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ - Per-line execution hit counts in the text report: `BASHUNIT_COVERAGE_SHOW_LINE_HITS=true` prints a `Line Hits` block listing each covered line as `:` per file. The LCOV report already carried the same counts in its `DA:,` records; those are now pinned by tests (#856) ### Changed +- Docs: [Command line](https://bashunit.com/command-line#invalid-input) now documents how bashunit rejects invalid input — unknown options, out-of-range values and unusable bootstrap/report paths — which had been shipped across several releases without a page describing it. `BASHUNIT_REPORT_TAP` and `BASHUNIT_REPORT_JSON` are documented in [Configuration](https://bashunit.com/configuration) alongside the JUnit/HTML/GHA reports they sit next to, and [Standalone](https://bashunit.com/standalone) documents the malformed-invocation exit code +- Docs: `snapshots.md` omitted the optional `["snapshot_file"]` argument that both snapshot assertions accept; the README claimed "hundreds of assertions" against an actual 71 - `bashunit doc` and the [Assertions](https://bashunit.com/assertions) reference now cover all 71 assertions instead of 64. The two snapshot assertions and all five spy assertions were documented only on their own pages, so `bashunit doc` — the command the docs tell agents to use so they do not invent names — listed no spy assertion at all - The [Assertions](https://bashunit.com/assertions) page opens with a grouped quick-reference table linking every assertion, instead of running straight into the first entry of a 1,700-line catalogue - The docs sidebar outline now lists `h3` headings as well as `h2`. Per-flag and per-pattern detail lives at `h3` (27 headings on Command line, 24 on Common patterns, 23 on Coverage) and was hidden from the "On this page" navigation diff --git a/README.md b/README.md index 96ad4b6e..a7c255e3 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ ## Why bashunit A lightweight, fast testing framework for **Bash 3.0+**, focused on developer experience. -It ships hundreds of assertions plus spies, mocks, data providers, snapshots and more. +It ships 71 assertions plus spies, mocks, data providers, snapshots and more. ## Quick start @@ -59,8 +59,8 @@ Prefer learning by doing? Run `./lib/bashunit learn` for an interactive tutorial ## Assertions at a glance -bashunit ships ~70 assertions across many families. One representative example per family -(full catalog and signatures at [bashunit.com/assertions](https://bashunit.com/assertions)): +One representative example per family — the full catalogue and every signature is at +[bashunit.com/assertions](https://bashunit.com/assertions), or run `bashunit doc` locally: | Family | Example | |---|---| @@ -82,6 +82,12 @@ Full documentation, covering installation options, every feature and examples, l Shell tab-completion for bash and zsh is available under [`completions/`](completions/) — see the [installation docs](https://bashunit.com/installation#shell-completion). +Using an AI coding agent? [bashunit.com/ai-agents](https://bashunit.com/ai-agents) covers machine-readable +results and the API traps that make generated bash tests pass for the wrong reason. The docs are also +published as plain text at [`llms.txt`](https://bashunit.com/llms.txt) and +[`llms-full.txt`](https://bashunit.com/llms-full.txt), with a drop-in skill at +[`bashunit-skill.md`](https://bashunit.com/bashunit-skill.md). + ## Contribute Issues, ideas and pull requests are welcome. diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 85a6080c..17ac27d6 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -32,7 +32,7 @@ Control benchmark behavior with comment annotations placed before the function: |------------|-------------|---------| | `@revs=N` | Number of revolutions (function calls per iteration) | 1 | | `@its=N` | Number of iterations (separate processes) | 1 | -| `@max_ms=N` | Maximum allowed average time in milliseconds | - | +| `@max_ms=N` | Maximum allowed average time in milliseconds (decimals allowed, e.g. `1.5`) | - | ::: code-group ```bash [Basic benchmark] diff --git a/docs/command-line.md b/docs/command-line.md index 2fbab7e2..bcde1402 100644 --- a/docs/command-line.md +++ b/docs/command-line.md @@ -975,6 +975,62 @@ bashunit watch --help bashunit doc --help ``` +## Invalid input + +bashunit validates its options before running anything and exits non-zero on a bad one. +It never silently ignores an option it does not understand — a typo would otherwise +produce a passing run that did something other than what you asked. + +### Unknown options + +An argument that looks like an option but matches nothing is an error, not a test path: + +```bash +bashunit --parralel tests/ +``` +```[Output] +Error: unknown option '--parralel'. Run 'bashunit test --help' to list the available options. +``` + +Without this, `--parralel` ran the suite **sequentially** and still exited `0`, and +`--filterr foo` swallowed both the flag and its value and ran the whole suite. + +### Invalid values + +`--jobs`, `--retry`, `--test-timeout`, `--coverage-min` and `--seed` require a +non-negative integer; `--output` accepts only `tap`; `--shard` requires `/`: + +```bash +bashunit --jobs abc tests/ +``` +```[Output] +Error: BASHUNIT_PARALLEL_JOBS (--jobs) must be a non-negative integer, got 'abc'. +``` + +The same check applies to the equivalent `BASHUNIT_*` environment variables, including +the env-only `BASHUNIT_COVERAGE_THRESHOLD_LOW` / `BASHUNIT_COVERAGE_THRESHOLD_HIGH`. + +### Unusable paths + +The bootstrap file must be readable, and every report destination must be writable. +Both are checked **before** the suite runs, so a bad path fails immediately rather than +after a green run has already reported success: + +```bash +bashunit --env missing.sh tests/ +bashunit --report-json /nope/dir/out.json tests/ +``` +```[Output] +Error: cannot read the bootstrap file: 'missing.sh'. +Error: BASHUNIT_REPORT_JSON cannot be written: '/nope/dir/out.json'. +``` + +::: tip +A `0` exit code means nothing *failed* — a run that was entirely skipped, incomplete or +risky also exits `0`. Add [`--fail-on-risky`](#test-options) or read the counts from +[`--report-json`](#reports). +::: + ## Related - [Configuration](/configuration) — set the same options via env vars and config files diff --git a/docs/configuration.md b/docs/configuration.md index 9bf04ab8..80903163 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -430,6 +430,35 @@ BASHUNIT_REPORT_HTML=report.html ``` ::: +## Report TAP + +> `BASHUNIT_REPORT_TAP=file` + +Write a [TAP version 13](https://testanything.org/tap-version-13-specification.html) report to the given file. Unlike `BASHUNIT_OUTPUT_FORMAT=tap`, which replaces the console output, this writes to disk and leaves the normal output alone. + +::: code-group +```bash [Example] +BASHUNIT_REPORT_TAP=report.tap +``` +::: + +## Report JSON + +> `BASHUNIT_REPORT_JSON=file` + +Write a machine-readable JSON report to the given file: a summary plus one entry per test, including the failure message and source line. Strings are escaped in pure Bash, so `jq` is not required to produce it. + +::: code-group +```bash [Example] +BASHUNIT_REPORT_JSON=report.json +``` +::: + +::: tip +The report destination is checked before the suite runs — an unwritable path fails +immediately instead of after a passing run. See [Invalid input](/command-line#invalid-input). +::: + ## Bootstrap > `BASHUNIT_BOOTSTRAP=file` diff --git a/docs/snapshots.md b/docs/snapshots.md index cb3e3e5d..b46f7c4d 100644 --- a/docs/snapshots.md +++ b/docs/snapshots.md @@ -11,11 +11,14 @@ This way, it helps maintain the expected behavior while modifications are being making the verification process more efficient and reliable. ## assert_match_snapshot -> `assert_match_snapshot "actual"` +> `assert_match_snapshot "actual" ["snapshot_file"]` Reports an error if `actual` does not match the existing snapshot file associated with the current test function. If no such file exists, a new one is created with the provided value. +Pass `snapshot_file` to point at a specific snapshot — useful to share one snapshot between tests. +By default each test gets its own, named after the test function. + ::: tip You can update the snapshot by deleting it and running its test again. ::: @@ -58,7 +61,7 @@ The first time you run them, the snapshots will be generated and the second time ::: ## assert_match_snapshot_ignore_colors -> `assert_match_snapshot_ignore_colors "actual"` +> `assert_match_snapshot_ignore_colors "actual" ["snapshot_file"]` Like `assert_match_snapshot` but ANSI escape codes in `actual` are ignored. This allows verifying the output text while disregarding its style. diff --git a/docs/standalone.md b/docs/standalone.md index 01ed3207..60583159 100644 --- a/docs/standalone.md +++ b/docs/standalone.md @@ -24,9 +24,20 @@ Execute assertions directly from the command line using the `assert` command: | Exit Code | Description | |-----------|-------------| | `0` | Assertion passed | -| `1` | Assertion failed | +| `1` | Assertion failed, or the invocation was malformed | | `127` | Non-existing function | +A malformed invocation is reported rather than passed over — an assertion called with no +arguments at all exits `1` with an error, so a typo in a deploy script cannot look like a +success: + +```bash +bashunit assert equals +``` +```[Output] +Error: assert equals requires at least one argument. +``` + ## Basic Usage ### With or Without Prefix diff --git a/docs/test-doubles.md b/docs/test-doubles.md index 742d910e..be736dab 100644 --- a/docs/test-doubles.md +++ b/docs/test-doubles.md @@ -208,7 +208,7 @@ function test_failure() { ::: ## assert_have_been_called_times -> assert_have_been_called_times "expected" "spy" +> `assert_have_been_called_times "expected" "spy"` Reports an error if `spy` is not called exactly `expected` times.