From 48c29c2a5ff0beb1f7f2bd2c2b72e63be10871a0 Mon Sep 17 00:00:00 2001 From: Daniel Polo <106583643+danielPoloWork@users.noreply.github.com> Date: Thu, 9 Jul 2026 21:49:30 +0200 Subject: [PATCH 1/4] perf(bench): add tail-latency percentiles and optional jemalloc/tcmalloc baselines MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Close the two residual spec-review (#105 §6.3) critiques left standing after ADR-0014's methodology was verified as already delivered, by extending the pool-vs-malloc microbenchmark (ADR-0045 extends ADR-0014; both additive). Tail latency: - A new opt-in `--percentiles` mode times the work PER OPERATION (one sample per op, vs the aggregate path's one sample per repeat) and emits a separate TSV table with p50/p90/p99/p999 + sample count. It covers interleaved for every allocator plus a dynamic-pool growth row whose p99/p999 surface the microsecond-scale growth spike the aggregate median averages away. - Opt-in so per-op timing overhead never perturbs the committed ADR-0014 ns/op numbers. Documented caveat: per-op resolution is the platform steady_clock tick (~1ns Linux/macOS, ~100ns Windows where it quantizes), so the columns are for tail/relative comparison and surfacing us-scale events, not absolute per-op cost — the aggregate table stays authoritative. External baselines: - jemalloc and tcmalloc are optional, CMake-feature-detected baselines (via find_library + find_path); when present they appear as extra rows in the aggregate and percentile tables, driven through mallocx/dallocx and tc_malloc/tc_free so they don't override the system malloc row. When absent (the default, and every MSVC build) the guarded code is compiled out and the output is unchanged but for a `# baselines:` disclosure line — spec §3.3's zero-external-dependency posture is preserved. A RawAllocator (name + alloc/free fn pointers) unifies system malloc and the baselines for the added rows and the percentile recorder; the dedicated malloc runners that produce the committed numbers are untouched. A Linux `bench-baselines` CI cell installs both allocators and asserts the baseline rows + percentile table are present; like every bench cell it gates on exit code 0, not numbers (ADR-0014 §8). Closes #111. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/ci.yml | 50 +++ CHANGELOG.md | 11 + ROADMAP.md | 2 +- ...crobenchmark-methodology-pool-vs-malloc.md | 6 +- ...mark-percentiles-and-external-baselines.md | 75 +++++ docs/adr/README.md | 1 + docs/specs/01_spec_cpp_memory_pool.md | 9 +- .../cpp/it/d4np/memorypool/CMakeLists.txt | 26 ++ src/bench/cpp/it/d4np/memorypool/README.md | 32 +- .../d4np/memorypool/pool_vs_malloc_bench.cpp | 308 +++++++++++++++++- 10 files changed, 507 insertions(+), 13 deletions(-) create mode 100644 docs/adr/0045-benchmark-percentiles-and-external-baselines.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1672e98..abbc7b9 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -459,6 +459,56 @@ jobs: "$bin" --iterations 10000 --repeats 3 echo "OK — bench binary ran to completion." + # --------------------------------------------------------------------------- + # ROADMAP §9.4 — external-allocator baselines + tail-latency percentiles + # (ADR-0045). Installs jemalloc + tcmalloc so the feature-detected baseline + # rows compile and run, and exercises the opt-in --percentiles table. Linux + # only (the baselines are packaged there); never touches the MSVC leg. Like + # the other bench cells this asserts exit code 0, not numbers. + # --------------------------------------------------------------------------- + bench-baselines: + name: bench / external baselines + percentiles + runs-on: ubuntu-24.04 + steps: + - name: Check out + uses: actions/checkout@v6 + + - name: Install CMake and Ninja + uses: lukka/get-cmake@latest + + - name: Install jemalloc and tcmalloc + shell: bash + run: | + sudo apt-get update + sudo apt-get install -y libjemalloc-dev libgoogle-perftools-dev + + - name: Configure (bench preset) and confirm both baselines were detected + shell: bash + run: | + set -euo pipefail + export CC=gcc CXX=g++ + cmake --preset bench 2>&1 | tee configure.log + grep -q "bench baseline: jemalloc" configure.log || { echo "FAIL: jemalloc not detected"; exit 1; } + grep -q "bench baseline: tcmalloc" configure.log || { echo "FAIL: tcmalloc not detected"; exit 1; } + + - name: Build the bench binary + shell: bash + run: cmake --build --preset bench + + - name: Run with baselines and the percentile table + shell: bash + run: | + set -euo pipefail + bin="build/bench/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench" + out="$("$bin" --scenario all --iterations 20000 --repeats 3 --percentiles)" + echo "$out" + # Assert the added rows/columns are actually present. + echo "$out" | grep -q "^# baselines: malloc jemalloc tcmalloc" || { echo "FAIL: baseline disclosure missing"; exit 1; } + echo "$out" | grep -q " jemalloc " || { echo "FAIL: jemalloc row missing"; exit 1; } + echo "$out" | grep -q " tcmalloc " || { echo "FAIL: tcmalloc row missing"; exit 1; } + echo "$out" | grep -q "p99_ns/op" || { echo "FAIL: percentile table missing"; exit 1; } + echo "OK — baselines + percentile table present and ran to completion." + # --------------------------------------------------------------------------- # thread-safety — build and run the existing (single-threaded) test suite # under each non-default PBR_MEMORY_POOL_THREAD_SAFETY policy (ADR-0020 / diff --git a/CHANGELOG.md b/CHANGELOG.md index d799dd1..ede7a3c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,6 +20,17 @@ dated version block (`## [X.Y.Z] — YYYY-MM-DD`) when a release PR closes a mil ### Added +- **Benchmark extension — tail-latency percentiles and optional jemalloc/tcmalloc baselines.** + The pool-vs-malloc microbenchmark gains an opt-in `--percentiles` mode that emits a separate + per-operation **p50/p90/p99/p999** table — the dynamic-pool growth row surfaces the + microsecond-scale growth spike the aggregate median hides — and optional **jemalloc** / + **tcmalloc** baselines that are CMake-**feature-detected** (used via `mallocx`/`dallocx` and + `tc_malloc`/`tc_free`), so they appear as extra comparison rows where installed while the + default build stays zero-external-dependency (spec §3.3). Both are strictly additive: the + [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md) aggregate ns/op table + and its committed numbers are unchanged. A Linux `bench-baselines` CI cell installs both + allocators and exercises the new paths (non-asserting on numbers, per ADR-0014 §8). + [ADR-0045](docs/adr/0045-benchmark-percentiles-and-external-baselines.md). Closes #111. - **Coverage-guided fuzzing harness for the pool surface.** A new [`pool_fuzz.cpp`](src/test/cpp/it/d4np/memorypool/pool_fuzz.cpp) drives randomized `alloc` / `free(valid)` / `free(NULL)` / `free(foreign)` sequences — over both fixed and diff --git a/ROADMAP.md b/ROADMAP.md index 840499b..758457d 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -139,7 +139,7 @@ Goal: a coherent, post-`v1.0.0` wave of **additive, ABI-compatible** capabilitie - [x] 9.1 **`std::pmr::memory_resource` adapter** (`PoolMemoryResource`) — the "door left open" in [ADR-0018](docs/adr/0018-stl-allocator-adapter.md): a `std::pmr::memory_resource` subclass binding one `Pool` so any `std::pmr`-aware container can draw from it through `std::pmr::polymorphic_allocator`, without the `PoolAllocator` per-type rebind. Deterministic `(bytes, alignment)` routing to the bound pool — over-sized / over-aligned requests delegate to a configurable upstream resource, and exhaustion of a pool-eligible request throws `std::bad_alloc` rather than falling back (preserving the deterministic deallocate routing) — with `is_equal` by `(pool, upstream)` identity, gated behind `PBR_MEMORY_POOL_HAS_PMR` where `` is available. Header-only, additive, ABI-compatible. Decided in [ADR-0042](docs/adr/0042-pmr-memory-resource-adapter.md); implemented in [`pool_memory_resource.hpp`](src/main/cpp/it/d4np/memorypool/pool_memory_resource.hpp) with [`pool_memory_resource_test.cpp`](src/test/cpp/it/d4np/memorypool/pool_memory_resource_test.cpp) (issue #107). - [x] 9.2 **Opt-in debug hardening** — freed-block poisoning, canaries, and free-list safe-linking (which also yields double-free detection); zero cost when the gate is off (issue #109). Decided in [ADR-0043](docs/adr/0043-opt-in-debug-hardening.md); implemented in [`memory_pool.cpp`](src/main/cpp/it/d4np/memorypool/memory_pool.cpp) behind the compile-time `PBR_MEMORY_POOL_HARDENING` knob (a `harden` CMake preset), with the swappable violation-handler surface in [`pool_hardening.hpp`](src/main/cpp/it/d4np/memorypool/pool_hardening.hpp) and [`pool_hardening_test.cpp`](src/test/cpp/it/d4np/memorypool/pool_hardening_test.cpp) (CTest `pool_hardening`). The "canary" is realized as one trailing **guard word** living in *added* slot stride — so the user-visible `block_size` and the ADR-0009 alignment guarantee are unchanged and the default build is byte-for-byte unchanged (the mechanism is fully compiled out). Poisoning (`0xDE`) catches use-after-free on the next allocation, the guard word catches a write past `block_size` and a double-free, and glibc-style safe-linking (`ptr XOR (slot_addr >> 12)`) protects the in-band next-link. A `harden` CI matrix cell builds and tests the hardened configuration on each Tier-1 platform. Works with fixed and dynamic pools across all three thread-safety policies. - [x] 9.3 **Coverage-guided fuzzing harness** for the pool surface — a libFuzzer target, time-boxed in CI (issue #108). Decided in [ADR-0044](docs/adr/0044-coverage-guided-fuzzing-harness.md); implemented in [`pool_fuzz.cpp`](src/test/cpp/it/d4np/memorypool/pool_fuzz.cpp) as a stateful opcode interpreter with a shadow oracle that asserts the no-alias, canary-intact, foreign/NULL-no-op ([ADR-0012](docs/adr/0012-foreign-pointer-and-out-of-range-pointer-policy.md)) and `InstrumentedPool` accounting invariants across fixed and dynamic pools. One engine-agnostic source yields both the Clang-only libFuzzer target `pool_fuzz` (opt-in `PBR_MEMORY_POOL_BUILD_FUZZERS`, a `fuzz` preset under `-fsanitize=fuzzer,address,undefined`) and the always-built standalone [`pool_fuzz_replay`](src/test/cpp/it/d4np/memorypool/pool_fuzz_corpus/) target that replays the seed corpus as a portable regression gate (CTest `pool_fuzz_replay`) on every platform, including MSVC. A dedicated `fuzz` CI job replays the corpus and fuzzes for a bounded time on every PR; a crash is filed in the bug ledger ([ADR-0039](docs/adr/0039-bug-ledger-and-triage-protocol.md)). Test-only and additive — the release build and the benchmark numbers are untouched. -- [ ] 9.4 **Benchmark extension** — external allocator baselines (jemalloc / tcmalloc) and p99 tail-latency reporting (issue #111). +- [x] 9.4 **Benchmark extension** — external allocator baselines (jemalloc / tcmalloc) and p99 tail-latency reporting (issue #111). Decided in [ADR-0045](docs/adr/0045-benchmark-percentiles-and-external-baselines.md) (extends [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md)); implemented in [`pool_vs_malloc_bench.cpp`](src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp). An opt-in `--percentiles` mode adds a separate per-operation p50/p90/p99/p999 table (the dynamic-pool growth row surfaces the amortized-growth spike the median hides); jemalloc / tcmalloc are optional, CMake-feature-detected baselines (guarded so the default build keeps zero external dependencies — spec §3.3). Both are additive: the [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md) aggregate table and its committed numbers are unchanged. A Linux `bench-baselines` CI cell installs both allocators and exercises the baseline + percentile paths (non-asserting on numbers). --- diff --git a/docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md b/docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md index 4dc18dd..824d054 100644 --- a/docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md +++ b/docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md @@ -1,9 +1,11 @@ # ADR-0014: Microbenchmark methodology — pool vs. `malloc` -- **Status:** Accepted +- **Status:** Accepted (extended by [ADR-0045](0045-benchmark-percentiles-and-external-baselines.md)) - **Date:** 2026-06-11 - **Deciders:** Daniel Polo (maintainer), Claude (architect agent) -- **Related:** spec §6.3, ROADMAP §2.9, [ADR-0005](0005-toolchain-matrix-and-supported-platforms.md) §3 (compiler matrix), [ADR-0006](0006-code-style-and-static-analysis-baseline.md) (style + lint baseline), [ADR-0009](0009-free-list-layout-block-size-constraints-and-alignment-guarantee.md) §2 (block-size constraints the benchmark must respect), [ADR-0013](0013-doxygen-for-api-markdown-for-narrative.md) (narrative goes in Markdown — bench reports too) +- **Related:** spec §6.3, ROADMAP §2.9, [ADR-0005](0005-toolchain-matrix-and-supported-platforms.md) §3 (compiler matrix), [ADR-0006](0006-code-style-and-static-analysis-baseline.md) (style + lint baseline), [ADR-0009](0009-free-list-layout-block-size-constraints-and-alignment-guarantee.md) §2 (block-size constraints the benchmark must respect), [ADR-0013](0013-doxygen-for-api-markdown-for-narrative.md) (narrative goes in Markdown — bench reports too), [ADR-0045](0045-benchmark-percentiles-and-external-baselines.md) (adds tail-latency percentiles + optional jemalloc/tcmalloc baselines) + +> **Extended by [ADR-0045](0045-benchmark-percentiles-and-external-baselines.md)** (2026-07-09): an opt-in `--percentiles` per-operation table (p50/p90/p99/p999) and optional feature-detected jemalloc/tcmalloc baselines. Both are strictly additive — the §4 aggregate statistics, the §6 output contract, and the committed numbers below are unchanged. ## Context diff --git a/docs/adr/0045-benchmark-percentiles-and-external-baselines.md b/docs/adr/0045-benchmark-percentiles-and-external-baselines.md new file mode 100644 index 0000000..0001a0e --- /dev/null +++ b/docs/adr/0045-benchmark-percentiles-and-external-baselines.md @@ -0,0 +1,75 @@ +# ADR-0045: Benchmark tail-latency percentiles and optional external allocator baselines + +- **Status:** Accepted +- **Date:** 2026-07-09 +- **Deciders:** Daniel Polo (maintainer / project architect) +- **Extends:** [ADR-0014](0014-microbenchmark-methodology-pool-vs-malloc.md) (the microbenchmark methodology — this ADR adds to it without superseding; the §1–8 contract there stays in force) +- **Related:** spec [§6.3](../specs/01_spec_cpp_memory_pool.md#63-performance-benchmark), [ADR-0004](0004-versioning-and-release-policy.md) (SemVer-neutral tooling change), [ADR-0005](0005-toolchain-matrix-and-supported-platforms.md) §3 (the platform split the baseline cell inherits), [ADR-0009](0009-free-list-layout-block-size-constraints-and-alignment-guarantee.md) §2 (block-size constraints), [ADR-0024](0024-dynamic-growth-synchronization-and-creation-surface.md) §2 (dynamic-growth support, whose events the p99 row surfaces), issue #111, origin issue #105 + +## Context + +[ADR-0014](0014-microbenchmark-methodology-pool-vs-malloc.md) delivered a rigorous hand-rolled microbenchmark — warm-up, min/median/mean/max/stddev over repeats, anti-optimization barriers, disclosed host, per-release reports, a non-asserting CI smoke, and (later) concurrent and growth scenarios. The spec review (#105 §6.3) is otherwise satisfied; **two residual critiques** stand: + +1. **No tail latency.** `median` is effectively p50; there is no p99/p999. For an allocator, the tail — e.g. the microsecond-scale spike when a dynamic pool grows — is exactly what a latency-sensitive consumer cares about, and the aggregate median averages it away. +2. **Only the system `malloc` baseline.** A reviewer compares an allocator against jemalloc / tcmalloc; without them the headline ratios are only vs glibc/MSVC `malloc`. + +This ADR extends ADR-0014 to close both, under two hard constraints: the existing aggregate ns/op numbers (the ADR-0014 committed contract) must **not** be perturbed, and the default build must keep spec §3.3's **zero external dependencies**. + +## Decision + +### 1. Tail-latency percentiles — opt-in `--percentiles`, per-operation timing, separate table + +Percentiles are only meaningful over a large sample set, so `--percentiles` switches on a **per-operation** timing path (one latency sample per op) — distinct from the aggregate path (one sample per repeat), which is left exactly as ADR-0014 froze it. It is **opt-in** so per-op timing overhead never perturbs the committed aggregate numbers. It emits a **separate TSV table** (`scenario, allocator, p50_ns/op, p90_ns/op, p99_ns/op, p999_ns/op, samples`) after the headline block, so the ADR-0014 aggregate table's 8-column schema is untouched. + +Percentiles use the **nearest-rank** method over the sorted per-op samples. The table covers the interleaved scenario (pool / `malloc` / each compiled baseline) and a **dynamic-pool growth** row whose p99/p999 expose the growth events — the headline motivation. + +**Documented caveat (honest methodology).** Per-op timing carries a fixed clock-read overhead common to every allocator, and its resolution is the platform `steady_clock` tick: on Linux/macOS (≈1 ns) the percentiles are fine-grained; on Windows (≈100 ns) they **quantize to the tick**, so p50–p99 of a single ~5–50 ns op collapse onto multiples of 100 ns. The percentile columns are therefore for **tail / relative comparison and for surfacing microsecond-scale events** (a growth spike shows as p999 ≫ p50 on every platform); for absolute per-op cost the aggregate ns/op table remains authoritative. This is stated in the bench README and the report template. + +### 2. Optional external baselines — feature-detected, never a hard dependency + +jemalloc and tcmalloc are added as **optional** baselines. CMake feature-detects each (`find_library` + `find_path` for the header); only when both are found does it define `PBR_BENCH_HAVE_JEMALLOC` / `PBR_BENCH_HAVE_TCMALLOC` and link it. When neither is present — the default, and every MSVC build — the guarded code is compiled out and the benchmark's output is byte-for-byte what ADR-0014 produced (plus one `# baselines: malloc` disclosure line). jemalloc is driven through its prefix-independent extended API (`mallocx`/`dallocx`), tcmalloc through gperftools' explicit `tc_malloc`/`tc_free`, so neither has to override the system `malloc` and all three appear as **distinct rows**. + +A small `RawAllocator` (name + alloc/free function pointers) unifies the system `malloc`, jemalloc, and tcmalloc behind one interface for the *added* baseline rows and the percentile recorder. The dedicated `malloc` runners that produce the committed ADR-0014 aggregate numbers are left untouched. + +### 3. Output contract + +Additive per ADR-0014 §6: the aggregate 8-column table is unchanged; baseline rows reuse that schema (extra rows, tagged with the allocator name); percentile data is a new, separate table with its own header; a `# baselines:` header line discloses which allocators are compiled in. No existing row or column changes, so the M7.x report-diffing tooling still parses old and new reports. + +### 4. CI + +A `bench-baselines` job (Linux) installs `libjemalloc-dev` + `libgoogle-perftools-dev`, asserts both baselines were feature-detected, builds, and runs `--scenario all --percentiles`, asserting the baseline rows and the percentile table are present. Like every bench cell it gates on **exit code 0, not numbers** (ADR-0014 §8 — shared runners are too noisy for numeric thresholds). It is Linux-only, never touching the MSVC leg (ADR-0005 §3), consistent with the sanitizer-preset split. + +## Alternatives Considered + +- **Always-on percentiles (fold p99 into the existing per-repeat `Stats`).** Rejected. p99 over ~9 per-repeat aggregates is meaningless (it is essentially the max), and per-op timing on the default path would perturb the committed ADR-0014 ns/op numbers. Per-op sampling behind an opt-in flag is the only way to get a meaningful p99 without disturbing the frozen numbers. +- **HdrHistogram (or another bucketed recorder).** Rejected for now: it is an external dependency, and for this benchmark's needs a `std::vector` of per-op samples with a nearest-rank query is sufficient and keeps the methodology inspectable (the ADR-0014 §1 pedagogy argument). Revisit if memory for the sample vector becomes a constraint at very large iteration counts. +- **`LD_PRELOAD` / link the whole binary against jemalloc.** Rejected: that *replaces* the `malloc` row with jemalloc rather than adding a distinct baseline, so pool / malloc / jemalloc cannot be compared side by side in one run. Calling the allocators' explicit APIs keeps them as separate rows. +- **A hard dependency on jemalloc/tcmalloc.** Rejected — it would break spec §3.3 and the MSVC leg. Feature-detection with a silent skip keeps the default build dependency-free. +- **CI numeric thresholds on the baselines/percentiles.** Rejected for the ADR-0014 §8 runner-noise reasons; the new cell is a build/run gate, not a performance gate. + +## Consequences + +**Positive** + +- p99/p999 tail latency is reportable, and the dynamic-pool growth row makes the amortized-growth spike visible where the median hid it — the #105 §6.3 critique is closed. +- Modern baselines (jemalloc/tcmalloc) are available for comparison wherever they are installed, feature-detected and validated in CI. +- The default build and the committed ADR-0014 numbers are unchanged; zero external dependencies preserved. + +**Negative** + +- Per-op percentiles are resolution-bound: on Windows's ~100 ns `steady_clock` tick they quantize, so they are meaningful there only for microsecond-scale tail events, not for sub-tick medians (documented; the aggregate table stays authoritative). +- The jemalloc/tcmalloc code paths build and run only where those libraries exist, so they are exercised on the Linux CI cell, not on the maintainer's MSVC box. + +**Tooling / documentation (same PR)** + +- [`pool_vs_malloc_bench.cpp`](../../src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp) — the `--percentiles` mode, the `RawAllocator` abstraction, and the guarded baselines. +- [bench `CMakeLists.txt`](../../src/bench/cpp/it/d4np/memorypool/CMakeLists.txt) — the jemalloc/tcmalloc feature-detect. +- [`.github/workflows/ci.yml`](../../.github/workflows/ci.yml) — the `bench-baselines` job. +- [bench `README.md`](../../src/bench/cpp/it/d4np/memorypool/README.md) — the new columns/baselines and the percentile caveat. +- [`ROADMAP.md`](../../ROADMAP.md) item 9.4, spec §7.1, [`CHANGELOG.md`](../../CHANGELOG.md) `Unreleased`. + +## References + +- [ADR-0014](0014-microbenchmark-methodology-pool-vs-malloc.md) — the methodology this extends. +- jemalloc `mallocx`/`dallocx` extended API; gperftools tcmalloc `tc_malloc`/`tc_free`. +- Gil Tene, *How NOT to Measure Latency* — the case for percentiles/tail over averages in latency reporting. diff --git a/docs/adr/README.md b/docs/adr/README.md index a7ad7e1..90e4120 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -77,5 +77,6 @@ Do **not** write one for purely local implementation details, formatting, or tri | 0042 | [`std::pmr::memory_resource` adapter — `PoolMemoryResource`](0042-pmr-memory-resource-adapter.md) | Accepted | | 0043 | [Opt-in debug hardening — poisoning, a guard word, and free-list safe-linking](0043-opt-in-debug-hardening.md) | Accepted | | 0044 | [A coverage-guided fuzzing harness for the pool surface](0044-coverage-guided-fuzzing-harness.md) | Accepted | +| 0045 | [Benchmark tail-latency percentiles and optional external allocator baselines](0045-benchmark-percentiles-and-external-baselines.md) | Accepted | When adding a new ADR, append a row to this table in the same PR. diff --git a/docs/specs/01_spec_cpp_memory_pool.md b/docs/specs/01_spec_cpp_memory_pool.md index 3e9dd6a..f41bdf6 100644 --- a/docs/specs/01_spec_cpp_memory_pool.md +++ b/docs/specs/01_spec_cpp_memory_pool.md @@ -204,7 +204,7 @@ valgrind --leak-check=full --show-leak-kinds=all ./test_pool ### 6.3 Performance Benchmark -`memory_pool_alloc`/`free` are compared against standard `malloc`/`free`. The methodology is fixed by [ADR-0014](../adr/0014-microbenchmark-methodology-pool-vs-malloc.md): warm-up repeat discarded, min/median/mean/max/stddev reported (not a single wall-clock loop), anti-optimization barriers, disclosed compiler flags and host, per-release reports under `docs/bench/`, and a non-asserting CI smoke run. A **concurrent** scenario (T threads on a shared pool, `MUTEX` vs `LOCKFREE`) provides the contended baseline. +`memory_pool_alloc`/`free` are compared against standard `malloc`/`free`. The methodology is fixed by [ADR-0014](../adr/0014-microbenchmark-methodology-pool-vs-malloc.md): warm-up repeat discarded, min/median/mean/max/stddev reported (not a single wall-clock loop), anti-optimization barriers, disclosed compiler flags and host, per-release reports under `docs/bench/`, and a non-asserting CI smoke run. A **concurrent** scenario (T threads on a shared pool, `MUTEX` vs `LOCKFREE`) provides the contended baseline. An opt-in `--percentiles` mode adds a per-operation **p50/p90/p99/p999 tail-latency** table (surfacing, for a dynamic pool, the microsecond-scale growth spike the median averages away), and **jemalloc / tcmalloc** are available as optional, feature-detected baselines alongside the system `malloc` — both additive and keeping the default build dependency-free ([ADR-0045](../adr/0045-benchmark-percentiles-and-external-baselines.md)). ### 6.4 Sanitizers & CI @@ -229,17 +229,18 @@ Every requirement above is realized and recorded. The table maps the spec to its | §5.3 error semantics | [ADR-0012](../adr/0012-foreign-pointer-and-out-of-range-pointer-policy.md), [ADR-0016](../adr/0016-exception-policy-at-the-c-cpp-boundary.md) | | §5.4 instrumentation / observers | [ADR-0025](../adr/0025-decorator-for-instrumented-pool.md), [ADR-0026](../adr/0026-observer-for-pool-lifecycle-events.md) | | §6.3 benchmark methodology | [ADR-0014](../adr/0014-microbenchmark-methodology-pool-vs-malloc.md) | +| §6.3 benchmark percentiles & external baselines | [ADR-0045](../adr/0045-benchmark-percentiles-and-external-baselines.md) | | §6.4 coverage-guided fuzzing harness | [ADR-0044](../adr/0044-coverage-guided-fuzzing-harness.md) | | Spec-compliance acceptance audit | [ADR-0029](../adr/0029-spec-compliance-acceptance-audit.md) | ### 7.1 Deferred / tracked -These are explicitly out of the current build and tracked as issues: - -- **Benchmark extension** — external baselines (jemalloc/tcmalloc) and p99 percentile reporting (issue #111). +Every item once deferred here has now shipped; nothing from the original spec review remains outstanding. **Opt-in debug hardening** (freed-block poisoning, a guard word for overflow + double-free detection, and free-list safe-linking), once deferred here, now ships behind the `PBR_MEMORY_POOL_HARDENING` compile-time knob — see [§4.1](#41-constraints--guarantees) and [ADR-0043](../adr/0043-opt-in-debug-hardening.md) (issue #109). +The **benchmark extension** — external allocator baselines (jemalloc/tcmalloc) and p99 tail-latency percentiles — once deferred here, now ships as the opt-in `--percentiles` table and the feature-detected baseline rows — see [§6.3](#63-performance-benchmark) / [§6.4](#64-sanitizers--ci) and [ADR-0045](../adr/0045-benchmark-percentiles-and-external-baselines.md) (issue #111). + The **coverage-guided fuzzing harness**, once deferred here, now ships as the libFuzzer target `pool_fuzz` plus a portable standalone corpus-replay gate — see [§6.4](#64-sanitizers--ci) and [ADR-0044](../adr/0044-coverage-guided-fuzzing-harness.md) (issue #108). The **C4 component diagram** of the pool internals, once deferred here, now ships in [Section 4.2](#42-component-diagram-c4) (its tooling decision is [ADR-0041](../adr/0041-mermaid-diagram-tooling.md)). diff --git a/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt b/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt index ef3cb50..fbf2c15 100644 --- a/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt +++ b/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt @@ -30,6 +30,32 @@ elseif(PBR_MEMORY_POOL_THREAD_SAFETY STREQUAL "LOCKFREE") PBR_MEMORY_POOL_THREAD_SAFETY=PBR_MEMORY_POOL_THREAD_SAFETY_LOCKFREE) endif() +# --------------------------------------------------------------------------- +# Optional external-allocator baselines (ADR-0045). Feature-detected: when the +# library AND its header are both found, the bench compiles in an extra baseline +# row (and percentile row) for that allocator. NEVER a hard dependency — the +# default build stays zero-external-dependency (spec §3.3), and a missing +# allocator is a silent skip, not an error. jemalloc is used via its extended +# mallocx/dallocx API; tcmalloc via gperftools' explicit tc_malloc/tc_free. +# --------------------------------------------------------------------------- +find_library(PBR_JEMALLOC_LIB jemalloc) +find_path(PBR_JEMALLOC_INCLUDE_DIR jemalloc/jemalloc.h) +if(PBR_JEMALLOC_LIB AND PBR_JEMALLOC_INCLUDE_DIR) + target_compile_definitions(pool_vs_malloc_bench PRIVATE PBR_BENCH_HAVE_JEMALLOC=1) + target_include_directories(pool_vs_malloc_bench PRIVATE "${PBR_JEMALLOC_INCLUDE_DIR}") + target_link_libraries(pool_vs_malloc_bench PRIVATE "${PBR_JEMALLOC_LIB}") + message(STATUS " bench baseline: jemalloc (${PBR_JEMALLOC_LIB})") +endif() + +find_library(PBR_TCMALLOC_LIB tcmalloc) +find_path(PBR_TCMALLOC_INCLUDE_DIR gperftools/tcmalloc.h) +if(PBR_TCMALLOC_LIB AND PBR_TCMALLOC_INCLUDE_DIR) + target_compile_definitions(pool_vs_malloc_bench PRIVATE PBR_BENCH_HAVE_TCMALLOC=1) + target_include_directories(pool_vs_malloc_bench PRIVATE "${PBR_TCMALLOC_INCLUDE_DIR}") + target_link_libraries(pool_vs_malloc_bench PRIVATE "${PBR_TCMALLOC_LIB}") + message(STATUS " bench baseline: tcmalloc (${PBR_TCMALLOC_LIB})") +endif() + # Property name used by IDEs that group targets; matches the layout under # src/bench/cpp/. set_target_properties(pool_vs_malloc_bench PROPERTIES diff --git a/src/bench/cpp/it/d4np/memorypool/README.md b/src/bench/cpp/it/d4np/memorypool/README.md index 4c070fc..2ae77af 100644 --- a/src/bench/cpp/it/d4np/memorypool/README.md +++ b/src/bench/cpp/it/d4np/memorypool/README.md @@ -35,10 +35,12 @@ The binary lands at `build/bench/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc ```text usage: pool_vs_malloc_bench [--iterations N] [--repeats N] [--block-size N] - [--scenario {bulk|interleaved|both}] + [--threads N] + [--scenario {bulk|interleaved|concurrent|growth|both|all}] + [--percentiles] ``` -Defaults match the spec §6.3 contract — 1,000,000 iterations, 10 repeats (first discarded as warm-up, nine measured), 64-byte block size, both scenarios. +Defaults match the spec §6.3 contract — 1,000,000 iterations, 10 repeats (first discarded as warm-up, nine measured), 64-byte block size, both the `bulk` and `interleaved` scenarios (`concurrent` and `growth` are opt-in via `--scenario`). ```bash # Full canonical run — what the committed bench reports use. @@ -87,6 +89,30 @@ interleaved malloc alloc+free 47.600 51.360 49.480 51.360 All times are in **nanoseconds per single operation** (`ns/op`), computed as `(elapsed_ns_for_repeat / iterations)`. Min / median / mean / max / stddev are across the measured repeats only (the warm-up at index 0 is dropped before statistics). +## Tail-latency percentiles (`--percentiles`, ADR-0045) + +`--percentiles` appends a **separate** TSV table timed **per operation** (one sample per op, rather than one per repeat), so p50/p90/p99/p999 are meaningful. It covers the interleaved scenario for every allocator plus a dynamic-pool **growth** row whose p99/p999 expose the microsecond-scale spike of a growth event — the tail a latency-sensitive consumer cares about, which the aggregate median averages away. + +```text +# percentile table — per-op timing (ADR-0045); tail/relative, not absolute per-op cost +scenario allocator p50_ns/op p90_ns/op p99_ns/op p999_ns/op samples +interleaved pool 3.000 4.000 6.000 31.000 1000000 +growth pool 3.000 4.000 6.000 7000.000 1000000 +``` + +**Caveat:** per-op timing carries a fixed clock-read overhead, and its resolution is the platform `steady_clock` tick — ≈1 ns on Linux/macOS, but ≈100 ns on Windows, where the columns **quantize** to the tick. Treat the percentile columns as a **tail / relative** comparison and a way to surface microsecond-scale events (a growth spike shows as p999 ≫ p50 on every platform); the aggregate ns/op table above stays authoritative for absolute per-op cost. The `--percentiles` path is opt-in precisely so its overhead never perturbs those aggregate numbers. + +## External allocator baselines (jemalloc / tcmalloc, ADR-0045) + +jemalloc and tcmalloc are **optional** baselines, CMake-feature-detected: when the library and its header are both found at configure time, the bench compiles in an extra row for that allocator (in the aggregate table and, with `--percentiles`, the percentile table). When neither is present — the default, and every MSVC build — the benchmark is dependency-free (spec §3.3) and its output is unchanged but for a `# baselines: malloc` disclosure line. On Debian/Ubuntu: + +```bash +sudo apt-get install -y libjemalloc-dev libgoogle-perftools-dev +cmake --preset bench # prints "bench baseline: jemalloc / tcmalloc" when detected +cmake --build --preset bench +./build/bench/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench --scenario all --percentiles +``` + ## Reporting Every release that closes a milestone ships a bench report under [`docs/bench/`](../../../../../../docs/bench/). One file per release × host, named `v---.md`: @@ -101,7 +127,7 @@ The file wraps the raw benchmark output in a Markdown report disclosing the full ## CI -The `bench-smoke` job in [`.github/workflows/ci.yml`](../../../../../../.github/workflows/ci.yml) builds the bench binary with the `bench` preset and runs it with `--iterations 10000 --repeats 3` — proves the binary still compiles, links, and runs to completion. It deliberately does **not** assert numeric thresholds; shared-runner noise makes that gate flaky without adding signal. ADR-0014 §8 documents the rationale. +The `bench-smoke` job in [`.github/workflows/ci.yml`](../../../../../../.github/workflows/ci.yml) builds the bench binary with the `bench` preset and runs it with `--iterations 10000 --repeats 3` — proves the binary still compiles, links, and runs to completion. A companion `bench-baselines` job (Linux) installs jemalloc + tcmalloc and runs `--scenario all --percentiles`, asserting the feature-detected baseline rows and the percentile table are present (ADR-0045). Both deliberately **not** assert numeric thresholds; shared-runner noise makes that gate flaky without adding signal. ADR-0014 §8 documents the rationale. ## Methodology snapshot diff --git a/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp b/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp index 3c9bfb9..ec4354e 100644 --- a/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp +++ b/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp @@ -58,6 +58,15 @@ #include #include +// Optional external-allocator baselines (ADR-0045), feature-detected by CMake. +// Guarded so the default build keeps spec §3.3's zero external dependencies. +#ifdef PBR_BENCH_HAVE_JEMALLOC +#include +#endif +#ifdef PBR_BENCH_HAVE_TCMALLOC +#include +#endif + namespace mem = it::d4np::memorypool; namespace { @@ -77,6 +86,7 @@ struct Config { bool run_interleaved_ = true; bool run_concurrent_ = false; // M4.5 — opt-in (single-thread fast path vs concurrent path) bool run_growth_ = false; // M5.4 — opt-in (amortized cost of dynamic growth) + bool percentiles_ = false; // M9.4 — opt-in per-op tail-latency table (ADR-0045) }; constexpr std::size_t MIN_REPEATS = 2U; @@ -119,6 +129,69 @@ inline void touch_byte(void* ptr, std::size_t loop_index) { *static_cast(ptr) = static_cast(loop_index & BYTE_MASK); } +// --------------------------------------------------------------------------- +// A raw (C-style) allocator, described by a name and an alloc/free function +// pair (ADR-0045). This unifies the system `malloc` and the optional external +// baselines (jemalloc / tcmalloc) behind one interface so the generic runners +// and the per-op percentile recorder drive them all through the same code +// path. The existing dedicated `malloc` runners that produce the committed +// ADR-0014 aggregate numbers are left untouched; this abstraction serves the +// *added* baseline rows and the percentile table only. +// --------------------------------------------------------------------------- +struct RawAllocator { + std::string_view name_; + void* (*alloc_)(std::size_t); + void (*free_)(void*); +}; + +// The system allocator, wrapped so its address can travel in a RawAllocator. +// NOLINTNEXTLINE(cppcoreguidelines-no-malloc,cppcoreguidelines-owning-memory) +void* sys_malloc(std::size_t size) { + // NOLINTNEXTLINE(cppcoreguidelines-no-malloc,cppcoreguidelines-owning-memory) + return std::malloc(size); +} +void sys_free(void* ptr) { + // NOLINTNEXTLINE(cppcoreguidelines-no-malloc,cppcoreguidelines-owning-memory) + std::free(ptr); +} + +#ifdef PBR_BENCH_HAVE_JEMALLOC +// jemalloc's extended API (always exported, prefix-independent). mallocx +// requires a non-zero size and dallocx a non-null pointer; both hold here. +void* je_alloc(std::size_t size) { + return mallocx(size != 0U ? size : 1U, 0); +} +void je_release(void* ptr) { + if (ptr != nullptr) { + dallocx(ptr, 0); + } +} +#endif + +#ifdef PBR_BENCH_HAVE_TCMALLOC +// gperftools tcmalloc's explicit, non-overriding symbols. +void* tc_alloc(std::size_t size) { + return tc_malloc(size); +} +void tc_release(void* ptr) { + tc_free(ptr); +} +#endif + +// The optional external baselines compiled into this build (empty by default — +// spec §3.3). System malloc is NOT here: it keeps its dedicated committed-number +// runners; this list drives only the added baseline rows. +std::vector external_baselines() { + std::vector list; +#ifdef PBR_BENCH_HAVE_JEMALLOC + list.push_back(RawAllocator{"jemalloc", &je_alloc, &je_release}); +#endif +#ifdef PBR_BENCH_HAVE_TCMALLOC + list.push_back(RawAllocator{"tcmalloc", &tc_alloc, &tc_release}); +#endif + return list; +} + // --------------------------------------------------------------------------- // Stats over a sample of per-iteration nanosecond costs. The first repeat is // the warm-up and is dropped before this function is called. @@ -146,6 +219,36 @@ Stats summarise(std::vector samples) { return Stats{samples.front(), samples.at(n / 2U), mean, samples.back(), stddev}; } +// --------------------------------------------------------------------------- +// Tail-latency percentiles over a per-operation sample set (M9.4 / ADR-0045). +// Unlike Stats (computed over a handful of per-repeat aggregates), these are +// computed over one sample *per operation*, so p99 / p999 are meaningful — in +// particular they surface the microsecond-scale spikes of a dynamic-pool growth +// event that the aggregate median averages away. +// --------------------------------------------------------------------------- +struct Percentiles { + double p50_ns_; + double p90_ns_; + double p99_ns_; + double p999_ns_; + std::size_t count_; +}; + +Percentiles percentiles_of(std::vector samples) { + std::sort(samples.begin(), samples.end()); + const std::size_t n = samples.size(); + // Nearest-rank on a zero-based sorted vector: index = ceil(q*n) - 1, clamped. + const auto at_quantile = [&samples, n](double q) { + if (n == 0U) { + return 0.0; + } + const auto raw_rank = static_cast(std::ceil(q * static_cast(n))); + const std::size_t rank = std::min(std::max(raw_rank, 1U), n); + return samples.at(rank - 1U); + }; + return Percentiles{at_quantile(0.50), at_quantile(0.90), at_quantile(0.99), at_quantile(0.999), n}; +} + // --------------------------------------------------------------------------- // Timed runners. Each returns the elapsed nanoseconds per iteration for ONE // repeat. The outer loop (run_* runners below) takes care of warm-up + stats. @@ -244,6 +347,111 @@ double time_malloc_interleaved(const Config& cfg) { return ns_per_iter(t0, t1, cfg.iterations_); } +// --------------------------------------------------------------------------- +// Generic aggregate runners for a RawAllocator (ADR-0045). Same timing method +// as the dedicated malloc runners above — used for the added jemalloc/tcmalloc +// baseline rows, so their numbers are directly comparable to the malloc rows. +// --------------------------------------------------------------------------- +double time_raw_bulk_alloc(const RawAllocator& allocator, const Config& cfg, std::vector& out) { + out.clear(); + out.reserve(cfg.iterations_); + const auto t0 = clock::now(); + for (std::size_t i = 0; i < cfg.iterations_; ++i) { + void* const p = allocator.alloc_(cfg.block_size_); + touch_byte(p, i); + do_not_optimize(p); + out.push_back(p); + } + const auto t1 = clock::now(); + return ns_per_iter(t0, t1, cfg.iterations_); +} + +double time_raw_bulk_free(const RawAllocator& allocator, std::vector& blocks) { + const auto t0 = clock::now(); + for (void* p : blocks) { + do_not_optimize(p); + allocator.free_(p); + } + const auto t1 = clock::now(); + return ns_per_iter(t0, t1, blocks.size()); +} + +double time_raw_interleaved(const RawAllocator& allocator, const Config& cfg) { + const auto t0 = clock::now(); + for (std::size_t i = 0; i < cfg.iterations_; ++i) { + void* const p = allocator.alloc_(cfg.block_size_); + touch_byte(p, i); + do_not_optimize(p); + allocator.free_(p); + } + const auto t1 = clock::now(); + return ns_per_iter(t0, t1, cfg.iterations_); +} + +// --------------------------------------------------------------------------- +// Per-operation timers for the percentile table (ADR-0045). Each records ONE +// latency sample per operation into `out`, so a percentile summary is +// meaningful. Per-op timing adds a fixed clock-read overhead common to every +// allocator, so the percentile columns are for tail / relative comparison and +// for surfacing microsecond-scale events (e.g. a growth) — not an absolute +// per-op cost, for which the aggregate ns/op table stays authoritative. +// --------------------------------------------------------------------------- +void perop_pool_interleaved(mem::Pool& pool, const Config& cfg, std::vector& out) { + out.clear(); + out.reserve(cfg.iterations_); + for (std::size_t i = 0; i < cfg.iterations_; ++i) { + const auto t0 = clock::now(); + void* const p = pool.try_allocate(); + touch_byte(p, i); + do_not_optimize(p); + pool.deallocate(p); + const auto t1 = clock::now(); + out.push_back(ns_per_iter(t0, t1, 1U)); + } +} + +void perop_raw_interleaved(const RawAllocator& allocator, const Config& cfg, std::vector& out) { + out.clear(); + out.reserve(cfg.iterations_); + for (std::size_t i = 0; i < cfg.iterations_; ++i) { + const auto t0 = clock::now(); + void* const p = allocator.alloc_(cfg.block_size_); + touch_byte(p, i); + do_not_optimize(p); + allocator.free_(p); + const auto t1 = clock::now(); + out.push_back(ns_per_iter(t0, t1, 1U)); + } +} + +// A dynamic pool bulk-allocated with per-op timing: the growth events show up +// in the p99 / p999 tail — the headline motivation for this feature. Returns +// false if dynamic mode is unsupported in this build (lock-free — ADR-0024 §2). +bool perop_pool_growth(const Config& cfg, std::vector& out) { + std::optional opt = mem::Pool::make_dynamic(cfg.block_size_, GROWTH_INITIAL_BLOCKS, GROWTH_FACTOR); + if (!opt.has_value()) { + return false; + } + mem::Pool& pool = *opt; + out.clear(); + out.reserve(cfg.iterations_); + std::vector slots; + slots.reserve(cfg.iterations_); + for (std::size_t i = 0; i < cfg.iterations_; ++i) { + const auto t0 = clock::now(); + void* const p = pool.try_allocate(); + const auto t1 = clock::now(); + touch_byte(p, i); + do_not_optimize(p); + slots.push_back(p); + out.push_back(ns_per_iter(t0, t1, 1U)); + } + for (void* const p : slots) { + pool.deallocate(p); // untimed cleanup so destroy is leak-free + } + return true; +} + // --------------------------------------------------------------------------- // Concurrent scenario (M4.5). T threads each run cfg.iterations_ interleaved // alloc/free against a SHARED pool, started together via a release/acquire @@ -494,7 +702,8 @@ struct ParseLoc { std::cerr << argv0 << ": " << msg << "\n"; std::cerr << "usage: " << argv0 << " [--iterations N] [--repeats N]"; std::cerr << " [--block-size N] [--threads N]"; - std::cerr << " [--scenario {bulk|interleaved|concurrent|growth|both|all}]\n"; + std::cerr << " [--scenario {bulk|interleaved|concurrent|growth|both|all}]"; + std::cerr << " [--percentiles]\n"; std::exit(EXIT_FAILURE); } @@ -571,6 +780,8 @@ Config parse_args(int argc, char* argv[]) { } } ++i; + } else if (a == "--percentiles") { + cfg.percentiles_ = true; // M9.4 — opt-in tail-latency table (ADR-0045) } else if (a == "-h" || a == "--help") { die_with_usage(argv0, "help requested"); } else { @@ -592,7 +803,7 @@ Config parse_args(int argc, char* argv[]) { // std::ostringstream for the headline tail. // --------------------------------------------------------------------------- std::string_view compiler_name() { -#if defined(__clang__) +#ifdef __clang__ return "clang"; #elif defined(__GNUC__) return "gcc"; @@ -605,7 +816,7 @@ std::string_view compiler_name() { std::string compiler_version() { std::ostringstream s; -#if defined(__clang__) +#ifdef __clang__ s << __clang_major__ << "." << __clang_minor__ << "." << __clang_patchlevel__; #elif defined(__GNUC__) s << __GNUC__ << "." << __GNUC_MINOR__ << "." << __GNUC_PATCHLEVEL__; @@ -624,6 +835,14 @@ void print_header(std::ostream& os, const Config& cfg) { os << "# hardware_concurrency: " << std::thread::hardware_concurrency() << "\n"; os << "# max_align_t: " << alignof(std::max_align_t) << " bytes\n"; os << "# thread_safety_policy: " << policy_name() << "\n"; + os << "# baselines: malloc"; + for (const RawAllocator& allocator : external_baselines()) { + os << " " << allocator.name_; + } + os << "\n"; + if (cfg.percentiles_) { + os << "# percentiles: on (per-op tail-latency table appended — ADR-0045)\n"; + } os << "# config: iterations=" << cfg.iterations_; os << " repeats=" << cfg.repeats_; os << " block_size=" << cfg.block_size_; @@ -736,6 +955,81 @@ std::string run_and_report_growth(const Config& cfg, std::ostream& body) { return tail.str(); } +// M9.4 — append rows for each optional external baseline (jemalloc / tcmalloc) +// to the main TSV table, using the same aggregate method and column schema as +// the malloc rows so they are directly comparable (ADR-0045). A no-op when no +// baseline is compiled in — the default build's table is byte-for-byte +// unchanged. Writes rows only; the existing malloc/pool headline is unchanged. +void run_and_report_baselines(const Config& cfg, std::ostream& body) { + std::vector slots; + for (const RawAllocator& allocator : external_baselines()) { + if (cfg.run_bulk_) { + std::vector alloc_samples; + std::vector free_samples; + for (std::size_t r = 0; r < cfg.repeats_; ++r) { + const double a = time_raw_bulk_alloc(allocator, cfg, slots); + const double f = time_raw_bulk_free(allocator, slots); + if (r != WARMUP_REPEAT_INDEX) { + alloc_samples.push_back(a); + free_samples.push_back(f); + } + } + print_row(body, RowKey{"bulk", allocator.name_, "alloc"}, summarise(alloc_samples)); + print_row(body, RowKey{"bulk", allocator.name_, "free"}, summarise(free_samples)); + } + if (cfg.run_interleaved_) { + std::vector samples; + for (std::size_t r = 0; r < cfg.repeats_; ++r) { + const double s = time_raw_interleaved(allocator, cfg); + if (r != WARMUP_REPEAT_INDEX) { + samples.push_back(s); + } + } + print_row(body, RowKey{"interleaved", allocator.name_, "alloc+free"}, summarise(samples)); + } + } +} + +// M9.4 — the tail-latency table (ADR-0045). A separate TSV section (its own +// header) so the ADR-0014 aggregate table's schema is untouched. Per-op timing +// for interleaved across pool / malloc / each baseline, plus a dynamic-pool +// growth row whose p99 / p999 expose the growth spikes. +void print_percentile_table_header(std::ostream& os) { + os << "scenario\tallocator\tp50_ns/op\tp90_ns/op\tp99_ns/op\tp999_ns/op\tsamples\n"; +} + +void print_percentile_row(std::ostream& os, std::string_view scenario, std::string_view allocator, + const Percentiles& p) { + os << scenario << "\t" << allocator << "\t" << p.p50_ns_ << "\t" << p.p90_ns_ << "\t" << p.p99_ns_ << "\t" + << p.p999_ns_ << "\t" << p.count_ << "\n"; +} + +void run_and_report_percentiles(const Config& cfg, std::ostream& body) { + body << "\n# percentile table — per-op timing (ADR-0045); tail/relative, not absolute per-op cost\n"; + print_percentile_table_header(body); + std::vector samples; + + std::optional pool = mem::Pool::make(cfg.block_size_, INTERLEAVED_POOL_CAPACITY); + if (pool.has_value()) { + perop_pool_interleaved(*pool, cfg, samples); + print_percentile_row(body, "interleaved", "pool", percentiles_of(samples)); + } + + const RawAllocator system_malloc{"malloc", &sys_malloc, &sys_free}; + perop_raw_interleaved(system_malloc, cfg, samples); + print_percentile_row(body, "interleaved", "malloc", percentiles_of(samples)); + for (const RawAllocator& allocator : external_baselines()) { + perop_raw_interleaved(allocator, cfg, samples); + print_percentile_row(body, "interleaved", allocator.name_, percentiles_of(samples)); + } + + if (perop_pool_growth(cfg, samples)) { + print_percentile_row(body, "growth", "pool", percentiles_of(samples)); + } else { + body << "# percentile: growth skipped (dynamic mode unsupported in this build — ADR-0024 §2)\n"; + } +} + } // namespace // NOLINTNEXTLINE(bugprone-exception-escape,cppcoreguidelines-avoid-c-arrays,modernize-avoid-c-arrays,hicpp-avoid-c-arrays) @@ -762,6 +1056,14 @@ int main(int argc, char* argv[]) { if (cfg.run_growth_) { tail += run_and_report_growth(cfg, std::cout); } + // Added external-baseline rows (jemalloc / tcmalloc) go into the same table, + // before the headline tail; a no-op when none are compiled in (ADR-0045). + run_and_report_baselines(cfg, std::cout); std::cout << "\n" << tail; + // The opt-in per-op tail-latency table is a separate section after the + // headlines so the ADR-0014 aggregate table is untouched (ADR-0045). + if (cfg.percentiles_) { + run_and_report_percentiles(cfg, std::cout); + } return 0; } From a3e869e6ad9791f361be4a35ee4e6fbd3c991036 Mon Sep 17 00:00:00 2001 From: Daniel Polo <106583643+danielPoloWork@users.noreply.github.com> Date: Thu, 9 Jul 2026 21:59:55 +0200 Subject: [PATCH 2/4] fix(bench): load jemalloc/tcmalloc via dlopen instead of linking them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first implementation linked both allocators (`-ljemalloc -ltcmalloc`), which crashed the bench at runtime (SIGSEGV): both export strong malloc / operator new symbols, so co-linking makes their initializers fight over the global allocator, and even linking one interposes the process malloc — which would silently turn the "malloc" row into that allocator. Load each at run time via dlopen(RTLD_LOCAL) and call only its explicit extended API (mallocx/dallocx, tc_malloc/tc_free), resolved by dlsym. With RTLD_LOCAL their symbols stay out of global resolution, so the process malloc remains the true system allocator, the two never conflict, and pool / malloc / jemalloc / tcmalloc all appear as distinct rows in one run. dlsym's void* is copied into the typed function pointer with memcpy (not a reinterpret_cast between object- and function-pointer types, which -Wpedantic rejects). The mechanism is POSIX-only (#ifdef PBR_BENCH_DLOPEN) and compiled out on Windows; the only build dependency is ${CMAKE_DL_LIBS}. The CI cell now installs the runtime shared objects (libjemalloc2, libgoogle-perftools4t64) and asserts both baselines load at run time. ADR-0045, bench README, spec, ROADMAP, and CHANGELOG updated to describe the dlopen approach. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/ci.yml | 24 ++-- CHANGELOG.md | 5 +- ROADMAP.md | 2 +- ...mark-percentiles-and-external-baselines.md | 12 +- docs/specs/01_spec_cpp_memory_pool.md | 2 +- .../cpp/it/d4np/memorypool/CMakeLists.txt | 32 ++--- src/bench/cpp/it/d4np/memorypool/README.md | 8 +- .../d4np/memorypool/pool_vs_malloc_bench.cpp | 113 +++++++++++++----- 8 files changed, 119 insertions(+), 79 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index abbc7b9..efde19b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -476,24 +476,19 @@ jobs: - name: Install CMake and Ninja uses: lukka/get-cmake@latest - - name: Install jemalloc and tcmalloc + - name: Install jemalloc and tcmalloc runtime libraries shell: bash run: | sudo apt-get update - sudo apt-get install -y libjemalloc-dev libgoogle-perftools-dev + # The runtime shared objects are what dlopen loads at run time. + sudo apt-get install -y libjemalloc2 libgoogle-perftools4t64 - - name: Configure (bench preset) and confirm both baselines were detected + - name: Configure and build (bench preset) shell: bash run: | - set -euo pipefail export CC=gcc CXX=g++ - cmake --preset bench 2>&1 | tee configure.log - grep -q "bench baseline: jemalloc" configure.log || { echo "FAIL: jemalloc not detected"; exit 1; } - grep -q "bench baseline: tcmalloc" configure.log || { echo "FAIL: tcmalloc not detected"; exit 1; } - - - name: Build the bench binary - shell: bash - run: cmake --build --preset bench + cmake --preset bench + cmake --build --preset bench - name: Run with baselines and the percentile table shell: bash @@ -502,10 +497,11 @@ jobs: bin="build/bench/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench" out="$("$bin" --scenario all --iterations 20000 --repeats 3 --percentiles)" echo "$out" - # Assert the added rows/columns are actually present. + # The baselines are dlopen'd at run time; assert both were found and + # that their rows and the percentile table are present. echo "$out" | grep -q "^# baselines: malloc jemalloc tcmalloc" || { echo "FAIL: baseline disclosure missing"; exit 1; } - echo "$out" | grep -q " jemalloc " || { echo "FAIL: jemalloc row missing"; exit 1; } - echo "$out" | grep -q " tcmalloc " || { echo "FAIL: tcmalloc row missing"; exit 1; } + echo "$out" | grep -qP "\tjemalloc\t" || { echo "FAIL: jemalloc row missing"; exit 1; } + echo "$out" | grep -qP "\ttcmalloc\t" || { echo "FAIL: tcmalloc row missing"; exit 1; } echo "$out" | grep -q "p99_ns/op" || { echo "FAIL: percentile table missing"; exit 1; } echo "OK — baselines + percentile table present and ran to completion." diff --git a/CHANGELOG.md b/CHANGELOG.md index ede7a3c..09af44a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,8 +24,9 @@ dated version block (`## [X.Y.Z] — YYYY-MM-DD`) when a release PR closes a mil The pool-vs-malloc microbenchmark gains an opt-in `--percentiles` mode that emits a separate per-operation **p50/p90/p99/p999** table — the dynamic-pool growth row surfaces the microsecond-scale growth spike the aggregate median hides — and optional **jemalloc** / - **tcmalloc** baselines that are CMake-**feature-detected** (used via `mallocx`/`dallocx` and - `tc_malloc`/`tc_free`), so they appear as extra comparison rows where installed while the + **tcmalloc** baselines **loaded at run time via `dlopen`(`RTLD_LOCAL`)** (so they never + interpose the system `malloc` and cannot conflict by co-linking; called via `mallocx`/`dallocx` + and `tc_malloc`/`tc_free`), appearing as extra comparison rows where installed while the default build stays zero-external-dependency (spec §3.3). Both are strictly additive: the [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md) aggregate ns/op table and its committed numbers are unchanged. A Linux `bench-baselines` CI cell installs both diff --git a/ROADMAP.md b/ROADMAP.md index 758457d..fff7c0d 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -139,7 +139,7 @@ Goal: a coherent, post-`v1.0.0` wave of **additive, ABI-compatible** capabilitie - [x] 9.1 **`std::pmr::memory_resource` adapter** (`PoolMemoryResource`) — the "door left open" in [ADR-0018](docs/adr/0018-stl-allocator-adapter.md): a `std::pmr::memory_resource` subclass binding one `Pool` so any `std::pmr`-aware container can draw from it through `std::pmr::polymorphic_allocator`, without the `PoolAllocator` per-type rebind. Deterministic `(bytes, alignment)` routing to the bound pool — over-sized / over-aligned requests delegate to a configurable upstream resource, and exhaustion of a pool-eligible request throws `std::bad_alloc` rather than falling back (preserving the deterministic deallocate routing) — with `is_equal` by `(pool, upstream)` identity, gated behind `PBR_MEMORY_POOL_HAS_PMR` where `` is available. Header-only, additive, ABI-compatible. Decided in [ADR-0042](docs/adr/0042-pmr-memory-resource-adapter.md); implemented in [`pool_memory_resource.hpp`](src/main/cpp/it/d4np/memorypool/pool_memory_resource.hpp) with [`pool_memory_resource_test.cpp`](src/test/cpp/it/d4np/memorypool/pool_memory_resource_test.cpp) (issue #107). - [x] 9.2 **Opt-in debug hardening** — freed-block poisoning, canaries, and free-list safe-linking (which also yields double-free detection); zero cost when the gate is off (issue #109). Decided in [ADR-0043](docs/adr/0043-opt-in-debug-hardening.md); implemented in [`memory_pool.cpp`](src/main/cpp/it/d4np/memorypool/memory_pool.cpp) behind the compile-time `PBR_MEMORY_POOL_HARDENING` knob (a `harden` CMake preset), with the swappable violation-handler surface in [`pool_hardening.hpp`](src/main/cpp/it/d4np/memorypool/pool_hardening.hpp) and [`pool_hardening_test.cpp`](src/test/cpp/it/d4np/memorypool/pool_hardening_test.cpp) (CTest `pool_hardening`). The "canary" is realized as one trailing **guard word** living in *added* slot stride — so the user-visible `block_size` and the ADR-0009 alignment guarantee are unchanged and the default build is byte-for-byte unchanged (the mechanism is fully compiled out). Poisoning (`0xDE`) catches use-after-free on the next allocation, the guard word catches a write past `block_size` and a double-free, and glibc-style safe-linking (`ptr XOR (slot_addr >> 12)`) protects the in-band next-link. A `harden` CI matrix cell builds and tests the hardened configuration on each Tier-1 platform. Works with fixed and dynamic pools across all three thread-safety policies. - [x] 9.3 **Coverage-guided fuzzing harness** for the pool surface — a libFuzzer target, time-boxed in CI (issue #108). Decided in [ADR-0044](docs/adr/0044-coverage-guided-fuzzing-harness.md); implemented in [`pool_fuzz.cpp`](src/test/cpp/it/d4np/memorypool/pool_fuzz.cpp) as a stateful opcode interpreter with a shadow oracle that asserts the no-alias, canary-intact, foreign/NULL-no-op ([ADR-0012](docs/adr/0012-foreign-pointer-and-out-of-range-pointer-policy.md)) and `InstrumentedPool` accounting invariants across fixed and dynamic pools. One engine-agnostic source yields both the Clang-only libFuzzer target `pool_fuzz` (opt-in `PBR_MEMORY_POOL_BUILD_FUZZERS`, a `fuzz` preset under `-fsanitize=fuzzer,address,undefined`) and the always-built standalone [`pool_fuzz_replay`](src/test/cpp/it/d4np/memorypool/pool_fuzz_corpus/) target that replays the seed corpus as a portable regression gate (CTest `pool_fuzz_replay`) on every platform, including MSVC. A dedicated `fuzz` CI job replays the corpus and fuzzes for a bounded time on every PR; a crash is filed in the bug ledger ([ADR-0039](docs/adr/0039-bug-ledger-and-triage-protocol.md)). Test-only and additive — the release build and the benchmark numbers are untouched. -- [x] 9.4 **Benchmark extension** — external allocator baselines (jemalloc / tcmalloc) and p99 tail-latency reporting (issue #111). Decided in [ADR-0045](docs/adr/0045-benchmark-percentiles-and-external-baselines.md) (extends [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md)); implemented in [`pool_vs_malloc_bench.cpp`](src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp). An opt-in `--percentiles` mode adds a separate per-operation p50/p90/p99/p999 table (the dynamic-pool growth row surfaces the amortized-growth spike the median hides); jemalloc / tcmalloc are optional, CMake-feature-detected baselines (guarded so the default build keeps zero external dependencies — spec §3.3). Both are additive: the [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md) aggregate table and its committed numbers are unchanged. A Linux `bench-baselines` CI cell installs both allocators and exercises the baseline + percentile paths (non-asserting on numbers). +- [x] 9.4 **Benchmark extension** — external allocator baselines (jemalloc / tcmalloc) and p99 tail-latency reporting (issue #111). Decided in [ADR-0045](docs/adr/0045-benchmark-percentiles-and-external-baselines.md) (extends [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md)); implemented in [`pool_vs_malloc_bench.cpp`](src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp). An opt-in `--percentiles` mode adds a separate per-operation p50/p90/p99/p999 table (the dynamic-pool growth row surfaces the amortized-growth spike the median hides); jemalloc / tcmalloc are optional baselines loaded at run time via `dlopen`(`RTLD_LOCAL`), so they never interpose the system `malloc` and the default build keeps zero external dependencies (spec §3.3). Both are additive: the [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md) aggregate table and its committed numbers are unchanged. A Linux `bench-baselines` CI cell installs both allocators and exercises the baseline + percentile paths (non-asserting on numbers). --- diff --git a/docs/adr/0045-benchmark-percentiles-and-external-baselines.md b/docs/adr/0045-benchmark-percentiles-and-external-baselines.md index 0001a0e..c5a53a8 100644 --- a/docs/adr/0045-benchmark-percentiles-and-external-baselines.md +++ b/docs/adr/0045-benchmark-percentiles-and-external-baselines.md @@ -25,9 +25,9 @@ Percentiles use the **nearest-rank** method over the sorted per-op samples. The **Documented caveat (honest methodology).** Per-op timing carries a fixed clock-read overhead common to every allocator, and its resolution is the platform `steady_clock` tick: on Linux/macOS (≈1 ns) the percentiles are fine-grained; on Windows (≈100 ns) they **quantize to the tick**, so p50–p99 of a single ~5–50 ns op collapse onto multiples of 100 ns. The percentile columns are therefore for **tail / relative comparison and for surfacing microsecond-scale events** (a growth spike shows as p999 ≫ p50 on every platform); for absolute per-op cost the aggregate ns/op table remains authoritative. This is stated in the bench README and the report template. -### 2. Optional external baselines — feature-detected, never a hard dependency +### 2. Optional external baselines — `dlopen`-loaded at run time, never a hard dependency -jemalloc and tcmalloc are added as **optional** baselines. CMake feature-detects each (`find_library` + `find_path` for the header); only when both are found does it define `PBR_BENCH_HAVE_JEMALLOC` / `PBR_BENCH_HAVE_TCMALLOC` and link it. When neither is present — the default, and every MSVC build — the guarded code is compiled out and the benchmark's output is byte-for-byte what ADR-0014 produced (plus one `# baselines: malloc` disclosure line). jemalloc is driven through its prefix-independent extended API (`mallocx`/`dallocx`), tcmalloc through gperftools' explicit `tc_malloc`/`tc_free`, so neither has to override the system `malloc` and all three appear as **distinct rows**. +jemalloc and tcmalloc are added as **optional** baselines, **loaded at run time via `dlopen` with `RTLD_LOCAL`** (POSIX only) rather than linked. This is deliberate and load-bearing: both libraries, if *linked*, export strong `malloc`/`operator new` symbols that **interpose the whole process's allocator** — so linking one would silently turn the "malloc" row into that allocator, and linking *both* crashes (their initializers fight over global `malloc`). Loading each with `RTLD_LOCAL` keeps its symbols out of global resolution: the process's `malloc` stays the true system allocator, jemalloc and tcmalloc never conflict, and each is reached only through the explicit extended-API entry points resolved by `dlsym` — jemalloc's `mallocx`/`dallocx`, tcmalloc's `tc_malloc`/`tc_free`. All three therefore appear as **distinct, honest rows** in one run. `dlsym` yields a `void*`; the bits are copied into the typed function pointer with `memcpy` (not a `reinterpret_cast` between object- and function-pointer types, which `-Wpedantic` rejects). On a non-POSIX host (Windows/MSVC) the whole mechanism is `#ifdef`-compiled out and the output is byte-for-byte what ADR-0014 produced, plus a `# baselines: malloc` disclosure line; a library that is not installed is a silent run-time skip. The only build dependency is the dynamic-loader library (`${CMAKE_DL_LIBS}`, empty where `dlopen` lives in libc), so spec §3.3's zero-external-dependency posture holds. A small `RawAllocator` (name + alloc/free function pointers) unifies the system `malloc`, jemalloc, and tcmalloc behind one interface for the *added* baseline rows and the percentile recorder. The dedicated `malloc` runners that produce the committed ADR-0014 aggregate numbers are left untouched. @@ -37,13 +37,13 @@ Additive per ADR-0014 §6: the aggregate 8-column table is unchanged; baseline r ### 4. CI -A `bench-baselines` job (Linux) installs `libjemalloc-dev` + `libgoogle-perftools-dev`, asserts both baselines were feature-detected, builds, and runs `--scenario all --percentiles`, asserting the baseline rows and the percentile table are present. Like every bench cell it gates on **exit code 0, not numbers** (ADR-0014 §8 — shared runners are too noisy for numeric thresholds). It is Linux-only, never touching the MSVC leg (ADR-0005 §3), consistent with the sanitizer-preset split. +A `bench-baselines` job (Linux) installs the jemalloc + tcmalloc **runtime** shared objects (`libjemalloc2`, `libgoogle-perftools4t64`), builds, and runs `--scenario all --percentiles`, asserting that both baselines loaded at run time and that their rows and the percentile table are present. Like every bench cell it gates on **exit code 0, not numbers** (ADR-0014 §8 — shared runners are too noisy for numeric thresholds). It is Linux-only, never touching the MSVC leg (ADR-0005 §3), consistent with the sanitizer-preset split. ## Alternatives Considered - **Always-on percentiles (fold p99 into the existing per-repeat `Stats`).** Rejected. p99 over ~9 per-repeat aggregates is meaningless (it is essentially the max), and per-op timing on the default path would perturb the committed ADR-0014 ns/op numbers. Per-op sampling behind an opt-in flag is the only way to get a meaningful p99 without disturbing the frozen numbers. - **HdrHistogram (or another bucketed recorder).** Rejected for now: it is an external dependency, and for this benchmark's needs a `std::vector` of per-op samples with a nearest-rank query is sufficient and keeps the methodology inspectable (the ADR-0014 §1 pedagogy argument). Revisit if memory for the sample vector becomes a constraint at very large iteration counts. -- **`LD_PRELOAD` / link the whole binary against jemalloc.** Rejected: that *replaces* the `malloc` row with jemalloc rather than adding a distinct baseline, so pool / malloc / jemalloc cannot be compared side by side in one run. Calling the allocators' explicit APIs keeps them as separate rows. +- **Link the allocators at compile time (`find_library` + `-ljemalloc`/`-ltcmalloc`), or `LD_PRELOAD`.** Rejected — this was the first implementation and it *crashed* (SIGSEGV): both libraries export strong `malloc`/`operator new` symbols, so co-linking makes their initializers fight over global `malloc`, and even linking one interposes the process allocator (turning the "malloc" row into that allocator, defeating the side-by-side comparison). `LD_PRELOAD` has the same interposition problem and cannot load two allocators at once. Loading each via `dlopen(RTLD_LOCAL)` and calling only its explicit API sidesteps interposition entirely and lets pool / malloc / jemalloc / tcmalloc appear as four distinct rows in a single run. - **A hard dependency on jemalloc/tcmalloc.** Rejected — it would break spec §3.3 and the MSVC leg. Feature-detection with a silent skip keeps the default build dependency-free. - **CI numeric thresholds on the baselines/percentiles.** Rejected for the ADR-0014 §8 runner-noise reasons; the new cell is a build/run gate, not a performance gate. @@ -58,12 +58,12 @@ A `bench-baselines` job (Linux) installs `libjemalloc-dev` + `libgoogle-perftool **Negative** - Per-op percentiles are resolution-bound: on Windows's ~100 ns `steady_clock` tick they quantize, so they are meaningful there only for microsecond-scale tail events, not for sub-tick medians (documented; the aggregate table stays authoritative). -- The jemalloc/tcmalloc code paths build and run only where those libraries exist, so they are exercised on the Linux CI cell, not on the maintainer's MSVC box. +- The `dlopen` baseline path compiles on POSIX and loads the allocators only where they are installed, so it is exercised on the Linux CI cell, not on the maintainer's MSVC box (where it is `#ifdef`-compiled out). **Tooling / documentation (same PR)** - [`pool_vs_malloc_bench.cpp`](../../src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp) — the `--percentiles` mode, the `RawAllocator` abstraction, and the guarded baselines. -- [bench `CMakeLists.txt`](../../src/bench/cpp/it/d4np/memorypool/CMakeLists.txt) — the jemalloc/tcmalloc feature-detect. +- [bench `CMakeLists.txt`](../../src/bench/cpp/it/d4np/memorypool/CMakeLists.txt) — links `${CMAKE_DL_LIBS}` for the run-time `dlopen`. - [`.github/workflows/ci.yml`](../../.github/workflows/ci.yml) — the `bench-baselines` job. - [bench `README.md`](../../src/bench/cpp/it/d4np/memorypool/README.md) — the new columns/baselines and the percentile caveat. - [`ROADMAP.md`](../../ROADMAP.md) item 9.4, spec §7.1, [`CHANGELOG.md`](../../CHANGELOG.md) `Unreleased`. diff --git a/docs/specs/01_spec_cpp_memory_pool.md b/docs/specs/01_spec_cpp_memory_pool.md index f41bdf6..0e759b6 100644 --- a/docs/specs/01_spec_cpp_memory_pool.md +++ b/docs/specs/01_spec_cpp_memory_pool.md @@ -204,7 +204,7 @@ valgrind --leak-check=full --show-leak-kinds=all ./test_pool ### 6.3 Performance Benchmark -`memory_pool_alloc`/`free` are compared against standard `malloc`/`free`. The methodology is fixed by [ADR-0014](../adr/0014-microbenchmark-methodology-pool-vs-malloc.md): warm-up repeat discarded, min/median/mean/max/stddev reported (not a single wall-clock loop), anti-optimization barriers, disclosed compiler flags and host, per-release reports under `docs/bench/`, and a non-asserting CI smoke run. A **concurrent** scenario (T threads on a shared pool, `MUTEX` vs `LOCKFREE`) provides the contended baseline. An opt-in `--percentiles` mode adds a per-operation **p50/p90/p99/p999 tail-latency** table (surfacing, for a dynamic pool, the microsecond-scale growth spike the median averages away), and **jemalloc / tcmalloc** are available as optional, feature-detected baselines alongside the system `malloc` — both additive and keeping the default build dependency-free ([ADR-0045](../adr/0045-benchmark-percentiles-and-external-baselines.md)). +`memory_pool_alloc`/`free` are compared against standard `malloc`/`free`. The methodology is fixed by [ADR-0014](../adr/0014-microbenchmark-methodology-pool-vs-malloc.md): warm-up repeat discarded, min/median/mean/max/stddev reported (not a single wall-clock loop), anti-optimization barriers, disclosed compiler flags and host, per-release reports under `docs/bench/`, and a non-asserting CI smoke run. A **concurrent** scenario (T threads on a shared pool, `MUTEX` vs `LOCKFREE`) provides the contended baseline. An opt-in `--percentiles` mode adds a per-operation **p50/p90/p99/p999 tail-latency** table (surfacing, for a dynamic pool, the microsecond-scale growth spike the median averages away), and **jemalloc / tcmalloc** are available as optional baselines (loaded at run time via `dlopen` so they never interpose the system `malloc`) alongside it — both additive and keeping the default build dependency-free ([ADR-0045](../adr/0045-benchmark-percentiles-and-external-baselines.md)). ### 6.4 Sanitizers & CI diff --git a/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt b/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt index fbf2c15..cb47844 100644 --- a/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt +++ b/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt @@ -31,30 +31,16 @@ elseif(PBR_MEMORY_POOL_THREAD_SAFETY STREQUAL "LOCKFREE") endif() # --------------------------------------------------------------------------- -# Optional external-allocator baselines (ADR-0045). Feature-detected: when the -# library AND its header are both found, the bench compiles in an extra baseline -# row (and percentile row) for that allocator. NEVER a hard dependency — the -# default build stays zero-external-dependency (spec §3.3), and a missing -# allocator is a silent skip, not an error. jemalloc is used via its extended -# mallocx/dallocx API; tcmalloc via gperftools' explicit tc_malloc/tc_free. +# Optional external-allocator baselines (ADR-0045). jemalloc / tcmalloc are NOT +# linked — they are loaded at run time via dlopen(RTLD_LOCAL) so they never +# interpose the system malloc and cannot conflict by co-linking (see the source +# for the rationale). All that needs linking is the dynamic-loader library, +# where the platform separates it from libc; ${CMAKE_DL_LIBS} is empty on +# platforms (Windows, newer glibc) that fold dlopen into the C runtime. A +# missing allocator is a silent run-time skip, so the default build stays +# zero-external-dependency (spec §3.3). # --------------------------------------------------------------------------- -find_library(PBR_JEMALLOC_LIB jemalloc) -find_path(PBR_JEMALLOC_INCLUDE_DIR jemalloc/jemalloc.h) -if(PBR_JEMALLOC_LIB AND PBR_JEMALLOC_INCLUDE_DIR) - target_compile_definitions(pool_vs_malloc_bench PRIVATE PBR_BENCH_HAVE_JEMALLOC=1) - target_include_directories(pool_vs_malloc_bench PRIVATE "${PBR_JEMALLOC_INCLUDE_DIR}") - target_link_libraries(pool_vs_malloc_bench PRIVATE "${PBR_JEMALLOC_LIB}") - message(STATUS " bench baseline: jemalloc (${PBR_JEMALLOC_LIB})") -endif() - -find_library(PBR_TCMALLOC_LIB tcmalloc) -find_path(PBR_TCMALLOC_INCLUDE_DIR gperftools/tcmalloc.h) -if(PBR_TCMALLOC_LIB AND PBR_TCMALLOC_INCLUDE_DIR) - target_compile_definitions(pool_vs_malloc_bench PRIVATE PBR_BENCH_HAVE_TCMALLOC=1) - target_include_directories(pool_vs_malloc_bench PRIVATE "${PBR_TCMALLOC_INCLUDE_DIR}") - target_link_libraries(pool_vs_malloc_bench PRIVATE "${PBR_TCMALLOC_LIB}") - message(STATUS " bench baseline: tcmalloc (${PBR_TCMALLOC_LIB})") -endif() +target_link_libraries(pool_vs_malloc_bench PRIVATE ${CMAKE_DL_LIBS}) # Property name used by IDEs that group targets; matches the layout under # src/bench/cpp/. diff --git a/src/bench/cpp/it/d4np/memorypool/README.md b/src/bench/cpp/it/d4np/memorypool/README.md index 2ae77af..cced300 100644 --- a/src/bench/cpp/it/d4np/memorypool/README.md +++ b/src/bench/cpp/it/d4np/memorypool/README.md @@ -104,13 +104,13 @@ growth pool 3.000 4.000 6.000 7000.000 1000000 ## External allocator baselines (jemalloc / tcmalloc, ADR-0045) -jemalloc and tcmalloc are **optional** baselines, CMake-feature-detected: when the library and its header are both found at configure time, the bench compiles in an extra row for that allocator (in the aggregate table and, with `--percentiles`, the percentile table). When neither is present — the default, and every MSVC build — the benchmark is dependency-free (spec §3.3) and its output is unchanged but for a `# baselines: malloc` disclosure line. On Debian/Ubuntu: +jemalloc and tcmalloc are **optional** baselines, **loaded at run time via `dlopen`** (POSIX). They are deliberately *not* linked: linking either would make its `malloc` interpose the whole process (turning the "malloc" row into that allocator), and linking both crashes. Loading each with `RTLD_LOCAL` and calling only its explicit API (`mallocx`/`dallocx`, `tc_malloc`/`tc_free`) keeps the system `malloc` intact and lets all of pool / malloc / jemalloc / tcmalloc appear as distinct rows in one run. If an allocator's shared object is not installed, it is a silent skip; on non-POSIX hosts (MSVC) the mechanism is compiled out and the output is dependency-free (spec §3.3) but for a `# baselines: malloc` disclosure line. On Debian/Ubuntu just install the runtime libraries and run — no reconfigure needed: ```bash -sudo apt-get install -y libjemalloc-dev libgoogle-perftools-dev -cmake --preset bench # prints "bench baseline: jemalloc / tcmalloc" when detected -cmake --build --preset bench +sudo apt-get install -y libjemalloc2 libgoogle-perftools4t64 +cmake --preset bench && cmake --build --preset bench ./build/bench/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench --scenario all --percentiles +# the header line "# baselines: malloc jemalloc tcmalloc" confirms both loaded ``` ## Reporting diff --git a/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp b/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp index ec4354e..459db21 100644 --- a/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp +++ b/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp @@ -58,13 +58,15 @@ #include #include -// Optional external-allocator baselines (ADR-0045), feature-detected by CMake. -// Guarded so the default build keeps spec §3.3's zero external dependencies. -#ifdef PBR_BENCH_HAVE_JEMALLOC -#include -#endif -#ifdef PBR_BENCH_HAVE_TCMALLOC -#include +// Optional external-allocator baselines (ADR-0045). They are loaded at RUN time +// via dlopen with RTLD_LOCAL (POSIX only) rather than linked, so their malloc +// symbols never interpose the process's system allocator — the "malloc" row +// stays the true system malloc, and jemalloc + tcmalloc cannot conflict by both +// trying to take over global malloc, as they would if co-linked. The default +// build keeps spec §3.3's zero external dependencies. +#if defined(__unix__) || defined(__APPLE__) +#include +#define PBR_BENCH_DLOPEN 1 #endif namespace mem = it::d4np::memorypool; @@ -155,39 +157,94 @@ void sys_free(void* ptr) { std::free(ptr); } -#ifdef PBR_BENCH_HAVE_JEMALLOC -// jemalloc's extended API (always exported, prefix-independent). mallocx -// requires a non-zero size and dallocx a non-null pointer; both hold here. +#ifdef PBR_BENCH_DLOPEN +// The external allocators' explicit extended-API signatures, resolved by dlsym. +using MallocxFn = void* (*)(std::size_t, int); // jemalloc mallocx(size, flags) +using DallocxFn = void (*)(void*, int); // jemalloc dallocx(ptr, flags) +using TcMallocFn = void* (*)(std::size_t); // tcmalloc tc_malloc(size) +using TcFreeFn = void (*)(void*); // tcmalloc tc_free(ptr) + +// Resolved once by load_external_baselines(); a C-style dlsym table +// unavoidably needs mutable globals, mirroring the do_not_optimize sink above. +// NOLINTBEGIN(cppcoreguidelines-avoid-non-const-global-variables) +MallocxFn g_je_mallocx = nullptr; +DallocxFn g_je_dallocx = nullptr; +TcMallocFn g_tc_malloc = nullptr; +TcFreeFn g_tc_free = nullptr; +// NOLINTEND(cppcoreguidelines-avoid-non-const-global-variables) + +// mallocx requires a non-zero size and dallocx a non-null pointer; both hold. void* je_alloc(std::size_t size) { - return mallocx(size != 0U ? size : 1U, 0); + return g_je_mallocx(size != 0U ? size : 1U, 0); } void je_release(void* ptr) { if (ptr != nullptr) { - dallocx(ptr, 0); + g_je_dallocx(ptr, 0); } } -#endif - -#ifdef PBR_BENCH_HAVE_TCMALLOC -// gperftools tcmalloc's explicit, non-overriding symbols. void* tc_alloc(std::size_t size) { - return tc_malloc(size); + return g_tc_malloc(size); } void tc_release(void* ptr) { - tc_free(ptr); + g_tc_free(ptr); +} + +void* dlopen_local(const char* primary, const char* fallback) { + void* handle = dlopen(primary, RTLD_NOW | RTLD_LOCAL); + if (handle == nullptr) { + handle = dlopen(fallback, RTLD_NOW | RTLD_LOCAL); + } + return handle; +} + +// Resolve a symbol to a function pointer. dlsym yields a void*; copying the +// bits with memcpy (rather than reinterpret_cast between object- and +// function-pointer types, which -Wpedantic rejects as conditionally-supported) +// is the portable, warning-free idiom. Returns nullptr if the symbol is absent. +template +Fn resolve_symbol(void* handle, const char* name) { + void* const sym = dlsym(handle, name); + Fn fn = nullptr; + if (sym != nullptr) { + std::memcpy(&fn, &sym, sizeof(fn)); + } + return fn; } -#endif -// The optional external baselines compiled into this build (empty by default — -// spec §3.3). System malloc is NOT here: it keeps its dedicated committed-number -// runners; this list drives only the added baseline rows. -std::vector external_baselines() { +std::vector load_external_baselines() { std::vector list; -#ifdef PBR_BENCH_HAVE_JEMALLOC - list.push_back(RawAllocator{"jemalloc", &je_alloc, &je_release}); -#endif -#ifdef PBR_BENCH_HAVE_TCMALLOC - list.push_back(RawAllocator{"tcmalloc", &tc_alloc, &tc_release}); + void* const je = dlopen_local("libjemalloc.so.2", "libjemalloc.so"); + if (je != nullptr) { + g_je_mallocx = resolve_symbol(je, "mallocx"); + g_je_dallocx = resolve_symbol(je, "dallocx"); + if (g_je_mallocx != nullptr && g_je_dallocx != nullptr) { + list.push_back(RawAllocator{"jemalloc", &je_alloc, &je_release}); + } + } + void* tc = dlopen_local("libtcmalloc.so.4", "libtcmalloc.so"); + if (tc == nullptr) { + tc = dlopen_local("libtcmalloc_minimal.so.4", "libtcmalloc_minimal.so"); + } + if (tc != nullptr) { + g_tc_malloc = resolve_symbol(tc, "tc_malloc"); + g_tc_free = resolve_symbol(tc, "tc_free"); + if (g_tc_malloc != nullptr && g_tc_free != nullptr) { + list.push_back(RawAllocator{"tcmalloc", &tc_alloc, &tc_release}); + } + } + return list; +} +#endif // PBR_BENCH_DLOPEN + +// The external baselines available at run time (empty by default and on +// non-POSIX hosts — spec §3.3). Resolved once. System malloc is NOT here: it +// keeps its dedicated committed-number runners; this list drives the added +// baseline rows and the percentile table only. +const std::vector& external_baselines() { +#ifdef PBR_BENCH_DLOPEN + static const std::vector list = load_external_baselines(); +#else + static const std::vector list; #endif return list; } From cca0f9ec3ae49abb75d8e1d9768359011e50c411 Mon Sep 17 00:00:00 2001 From: Daniel Polo <106583643+danielPoloWork@users.noreply.github.com> Date: Thu, 9 Jul 2026 22:15:03 +0200 Subject: [PATCH 3/4] fix(bench): measure jemalloc/tcmalloc via LD_PRELOAD, not in-process MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both the linked and the dlopen(RTLD_LOCAL) approaches crashed at runtime (SIGSEGV): jemalloc and tcmalloc are designed to BE the process allocator — they take over global malloc/operator new on load (strong symbols, and for tcmalloc a library constructor), so there is no safe way to have two of them plus the system allocator coexist in one process. Pivot to the standard, crash-free method: measure each external allocator by re-running the SAME bench under LD_PRELOAD, which swaps the whole process allocator. Under a preload the bench's malloc rows and the pool's own backing are served by that allocator, so each run is a clean pool-vs-allocator comparison across every scenario (including the percentile table). A new `# allocator:` header line (read from LD_PRELOAD; POSIX only) discloses which allocator each run measured. Consequently the bench carries NO allocator-specific code: removed the RawAllocator abstraction, the dlopen loader, the generic raw runners, and the in-table baseline rows. The default build stays byte-for-byte ADR-0014 and zero-external-dependency (spec §3.3). The percentile table now covers pool + the process malloc + the dynamic-pool growth tail. The bench-baselines CI cell installs the jemalloc/tcmalloc runtime .so and re-runs the bench under each via LD_PRELOAD, asserting the allocator disclosure + percentile table (still non-asserting on numbers). ADR-0045, bench README, spec, ROADMAP, and CHANGELOG updated to the LD_PRELOAD design (and record why in-process was rejected). Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/ci.yml | 39 +-- CHANGELOG.md | 11 +- ROADMAP.md | 2 +- ...mark-percentiles-and-external-baselines.md | 22 +- docs/specs/01_spec_cpp_memory_pool.md | 4 +- .../cpp/it/d4np/memorypool/CMakeLists.txt | 15 +- src/bench/cpp/it/d4np/memorypool/README.md | 13 +- .../d4np/memorypool/pool_vs_malloc_bench.cpp | 252 ++---------------- 8 files changed, 84 insertions(+), 274 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index efde19b..b68d53d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -461,10 +461,11 @@ jobs: # --------------------------------------------------------------------------- # ROADMAP §9.4 — external-allocator baselines + tail-latency percentiles - # (ADR-0045). Installs jemalloc + tcmalloc so the feature-detected baseline - # rows compile and run, and exercises the opt-in --percentiles table. Linux - # only (the baselines are packaged there); never touches the MSVC leg. Like - # the other bench cells this asserts exit code 0, not numbers. + # (ADR-0045). External baselines are measured the safe way: re-run the SAME + # bench under LD_PRELOAD, which swaps the whole process allocator (jemalloc / + # tcmalloc take over global malloc on load, so they cannot be linked or dlopen'd + # beside the system allocator). Exercises the opt-in --percentiles table too. + # Linux only; never touches the MSVC leg. Asserts exit code 0, not numbers. # --------------------------------------------------------------------------- bench-baselines: name: bench / external baselines + percentiles @@ -480,8 +481,7 @@ jobs: shell: bash run: | sudo apt-get update - # The runtime shared objects are what dlopen loads at run time. - sudo apt-get install -y libjemalloc2 libgoogle-perftools4t64 + sudo apt-get install -y libjemalloc2 libtcmalloc-minimal4t64 - name: Configure and build (bench preset) shell: bash @@ -490,20 +490,27 @@ jobs: cmake --preset bench cmake --build --preset bench - - name: Run with baselines and the percentile table + - name: Baseline via LD_PRELOAD + the percentile table shell: bash run: | set -euo pipefail bin="build/bench/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench" - out="$("$bin" --scenario all --iterations 20000 --repeats 3 --percentiles)" - echo "$out" - # The baselines are dlopen'd at run time; assert both were found and - # that their rows and the percentile table are present. - echo "$out" | grep -q "^# baselines: malloc jemalloc tcmalloc" || { echo "FAIL: baseline disclosure missing"; exit 1; } - echo "$out" | grep -qP "\tjemalloc\t" || { echo "FAIL: jemalloc row missing"; exit 1; } - echo "$out" | grep -qP "\ttcmalloc\t" || { echo "FAIL: tcmalloc row missing"; exit 1; } - echo "$out" | grep -q "p99_ns/op" || { echo "FAIL: percentile table missing"; exit 1; } - echo "OK — baselines + percentile table present and ran to completion." + je="$(find / -name 'libjemalloc.so.2' 2>/dev/null | head -1)" + tc="$(find / -name 'libtcmalloc_minimal.so.4' 2>/dev/null | head -1)" + echo "jemalloc=$je tcmalloc=$tc" + [ -n "$je" ] || { echo "FAIL: libjemalloc.so.2 not found"; exit 1; } + [ -n "$tc" ] || { echo "FAIL: libtcmalloc_minimal.so.4 not found"; exit 1; } + run() { # label, preload-path + echo "=== allocator: $1 ===" + out="$(LD_PRELOAD="$2" "$bin" --scenario all --iterations 20000 --repeats 3 --percentiles)" + echo "$out" + echo "$out" | grep -q "# allocator: ${3:-system malloc}" || { echo "FAIL: allocator disclosure ($1)"; exit 1; } + echo "$out" | grep -q "p99_ns/op" || { echo "FAIL: percentile table missing ($1)"; exit 1; } + } + run "system malloc" "" "system malloc" + run "jemalloc" "$je" "$je" + run "tcmalloc" "$tc" "$tc" + echo "OK — pool benchmarked against system malloc, jemalloc, and tcmalloc; percentile table present." # --------------------------------------------------------------------------- # thread-safety — build and run the existing (single-threaded) test suite diff --git a/CHANGELOG.md b/CHANGELOG.md index 09af44a..533a7d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,11 +23,12 @@ dated version block (`## [X.Y.Z] — YYYY-MM-DD`) when a release PR closes a mil - **Benchmark extension — tail-latency percentiles and optional jemalloc/tcmalloc baselines.** The pool-vs-malloc microbenchmark gains an opt-in `--percentiles` mode that emits a separate per-operation **p50/p90/p99/p999** table — the dynamic-pool growth row surfaces the - microsecond-scale growth spike the aggregate median hides — and optional **jemalloc** / - **tcmalloc** baselines **loaded at run time via `dlopen`(`RTLD_LOCAL`)** (so they never - interpose the system `malloc` and cannot conflict by co-linking; called via `mallocx`/`dallocx` - and `tc_malloc`/`tc_free`), appearing as extra comparison rows where installed while the - default build stays zero-external-dependency (spec §3.3). Both are strictly additive: the + microsecond-scale growth spike the aggregate median hides — and **jemalloc** / **tcmalloc** + baselines measured the safe way, by re-running the bench under **`LD_PRELOAD`** (those + allocators take over global `malloc` on load, so they cannot be linked or `dlopen`'d beside + the system allocator without crashing). A `# allocator:` header line discloses which allocator + each run measured; the bench carries no allocator-specific code, so the default build stays + zero-external-dependency (spec §3.3). Both are strictly additive: the [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md) aggregate ns/op table and its committed numbers are unchanged. A Linux `bench-baselines` CI cell installs both allocators and exercises the new paths (non-asserting on numbers, per ADR-0014 §8). diff --git a/ROADMAP.md b/ROADMAP.md index fff7c0d..923aed8 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -139,7 +139,7 @@ Goal: a coherent, post-`v1.0.0` wave of **additive, ABI-compatible** capabilitie - [x] 9.1 **`std::pmr::memory_resource` adapter** (`PoolMemoryResource`) — the "door left open" in [ADR-0018](docs/adr/0018-stl-allocator-adapter.md): a `std::pmr::memory_resource` subclass binding one `Pool` so any `std::pmr`-aware container can draw from it through `std::pmr::polymorphic_allocator`, without the `PoolAllocator` per-type rebind. Deterministic `(bytes, alignment)` routing to the bound pool — over-sized / over-aligned requests delegate to a configurable upstream resource, and exhaustion of a pool-eligible request throws `std::bad_alloc` rather than falling back (preserving the deterministic deallocate routing) — with `is_equal` by `(pool, upstream)` identity, gated behind `PBR_MEMORY_POOL_HAS_PMR` where `` is available. Header-only, additive, ABI-compatible. Decided in [ADR-0042](docs/adr/0042-pmr-memory-resource-adapter.md); implemented in [`pool_memory_resource.hpp`](src/main/cpp/it/d4np/memorypool/pool_memory_resource.hpp) with [`pool_memory_resource_test.cpp`](src/test/cpp/it/d4np/memorypool/pool_memory_resource_test.cpp) (issue #107). - [x] 9.2 **Opt-in debug hardening** — freed-block poisoning, canaries, and free-list safe-linking (which also yields double-free detection); zero cost when the gate is off (issue #109). Decided in [ADR-0043](docs/adr/0043-opt-in-debug-hardening.md); implemented in [`memory_pool.cpp`](src/main/cpp/it/d4np/memorypool/memory_pool.cpp) behind the compile-time `PBR_MEMORY_POOL_HARDENING` knob (a `harden` CMake preset), with the swappable violation-handler surface in [`pool_hardening.hpp`](src/main/cpp/it/d4np/memorypool/pool_hardening.hpp) and [`pool_hardening_test.cpp`](src/test/cpp/it/d4np/memorypool/pool_hardening_test.cpp) (CTest `pool_hardening`). The "canary" is realized as one trailing **guard word** living in *added* slot stride — so the user-visible `block_size` and the ADR-0009 alignment guarantee are unchanged and the default build is byte-for-byte unchanged (the mechanism is fully compiled out). Poisoning (`0xDE`) catches use-after-free on the next allocation, the guard word catches a write past `block_size` and a double-free, and glibc-style safe-linking (`ptr XOR (slot_addr >> 12)`) protects the in-band next-link. A `harden` CI matrix cell builds and tests the hardened configuration on each Tier-1 platform. Works with fixed and dynamic pools across all three thread-safety policies. - [x] 9.3 **Coverage-guided fuzzing harness** for the pool surface — a libFuzzer target, time-boxed in CI (issue #108). Decided in [ADR-0044](docs/adr/0044-coverage-guided-fuzzing-harness.md); implemented in [`pool_fuzz.cpp`](src/test/cpp/it/d4np/memorypool/pool_fuzz.cpp) as a stateful opcode interpreter with a shadow oracle that asserts the no-alias, canary-intact, foreign/NULL-no-op ([ADR-0012](docs/adr/0012-foreign-pointer-and-out-of-range-pointer-policy.md)) and `InstrumentedPool` accounting invariants across fixed and dynamic pools. One engine-agnostic source yields both the Clang-only libFuzzer target `pool_fuzz` (opt-in `PBR_MEMORY_POOL_BUILD_FUZZERS`, a `fuzz` preset under `-fsanitize=fuzzer,address,undefined`) and the always-built standalone [`pool_fuzz_replay`](src/test/cpp/it/d4np/memorypool/pool_fuzz_corpus/) target that replays the seed corpus as a portable regression gate (CTest `pool_fuzz_replay`) on every platform, including MSVC. A dedicated `fuzz` CI job replays the corpus and fuzzes for a bounded time on every PR; a crash is filed in the bug ledger ([ADR-0039](docs/adr/0039-bug-ledger-and-triage-protocol.md)). Test-only and additive — the release build and the benchmark numbers are untouched. -- [x] 9.4 **Benchmark extension** — external allocator baselines (jemalloc / tcmalloc) and p99 tail-latency reporting (issue #111). Decided in [ADR-0045](docs/adr/0045-benchmark-percentiles-and-external-baselines.md) (extends [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md)); implemented in [`pool_vs_malloc_bench.cpp`](src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp). An opt-in `--percentiles` mode adds a separate per-operation p50/p90/p99/p999 table (the dynamic-pool growth row surfaces the amortized-growth spike the median hides); jemalloc / tcmalloc are optional baselines loaded at run time via `dlopen`(`RTLD_LOCAL`), so they never interpose the system `malloc` and the default build keeps zero external dependencies (spec §3.3). Both are additive: the [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md) aggregate table and its committed numbers are unchanged. A Linux `bench-baselines` CI cell installs both allocators and exercises the baseline + percentile paths (non-asserting on numbers). +- [x] 9.4 **Benchmark extension** — external allocator baselines (jemalloc / tcmalloc) and p99 tail-latency reporting (issue #111). Decided in [ADR-0045](docs/adr/0045-benchmark-percentiles-and-external-baselines.md) (extends [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md)); implemented in [`pool_vs_malloc_bench.cpp`](src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp). An opt-in `--percentiles` mode adds a separate per-operation p50/p90/p99/p999 table (the dynamic-pool growth row surfaces the amortized-growth spike the median hides); jemalloc / tcmalloc baselines are measured by re-running the bench under `LD_PRELOAD` (they take over global `malloc` on load, so they cannot be linked or `dlopen`'d beside the system allocator without crashing — an in-process attempt was tried and rejected); a `# allocator:` header discloses which allocator each run measured. The bench carries no allocator-specific code, so the default build keeps zero external dependencies (spec §3.3) and the [ADR-0014](docs/adr/0014-microbenchmark-methodology-pool-vs-malloc.md) aggregate table and its committed numbers are unchanged. A Linux `bench-baselines` CI cell re-runs the bench under each allocator's preload and asserts the disclosure + percentile table (non-asserting on numbers). --- diff --git a/docs/adr/0045-benchmark-percentiles-and-external-baselines.md b/docs/adr/0045-benchmark-percentiles-and-external-baselines.md index c5a53a8..d74abc7 100644 --- a/docs/adr/0045-benchmark-percentiles-and-external-baselines.md +++ b/docs/adr/0045-benchmark-percentiles-and-external-baselines.md @@ -25,25 +25,25 @@ Percentiles use the **nearest-rank** method over the sorted per-op samples. The **Documented caveat (honest methodology).** Per-op timing carries a fixed clock-read overhead common to every allocator, and its resolution is the platform `steady_clock` tick: on Linux/macOS (≈1 ns) the percentiles are fine-grained; on Windows (≈100 ns) they **quantize to the tick**, so p50–p99 of a single ~5–50 ns op collapse onto multiples of 100 ns. The percentile columns are therefore for **tail / relative comparison and for surfacing microsecond-scale events** (a growth spike shows as p999 ≫ p50 on every platform); for absolute per-op cost the aggregate ns/op table remains authoritative. This is stated in the bench README and the report template. -### 2. Optional external baselines — `dlopen`-loaded at run time, never a hard dependency +### 2. Optional external baselines — measured under `LD_PRELOAD`, never a hard dependency -jemalloc and tcmalloc are added as **optional** baselines, **loaded at run time via `dlopen` with `RTLD_LOCAL`** (POSIX only) rather than linked. This is deliberate and load-bearing: both libraries, if *linked*, export strong `malloc`/`operator new` symbols that **interpose the whole process's allocator** — so linking one would silently turn the "malloc" row into that allocator, and linking *both* crashes (their initializers fight over global `malloc`). Loading each with `RTLD_LOCAL` keeps its symbols out of global resolution: the process's `malloc` stays the true system allocator, jemalloc and tcmalloc never conflict, and each is reached only through the explicit extended-API entry points resolved by `dlsym` — jemalloc's `mallocx`/`dallocx`, tcmalloc's `tc_malloc`/`tc_free`. All three therefore appear as **distinct, honest rows** in one run. `dlsym` yields a `void*`; the bits are copied into the typed function pointer with `memcpy` (not a `reinterpret_cast` between object- and function-pointer types, which `-Wpedantic` rejects). On a non-POSIX host (Windows/MSVC) the whole mechanism is `#ifdef`-compiled out and the output is byte-for-byte what ADR-0014 produced, plus a `# baselines: malloc` disclosure line; a library that is not installed is a silent run-time skip. The only build dependency is the dynamic-loader library (`${CMAKE_DL_LIBS}`, empty where `dlopen` lives in libc), so spec §3.3's zero-external-dependency posture holds. +jemalloc and tcmalloc are measured by **re-running the same bench binary under `LD_PRELOAD`**, which swaps the whole process allocator. This is deliberate and load-bearing. Those libraries are *designed to be the process allocator*: they take over global `malloc`/`operator new` — via strong symbols and, for tcmalloc, a library constructor — on load. So they cannot be linked, nor even `dlopen`'d, alongside the system allocator to produce side-by-side in-process rows: linking one silently turns the "malloc" row into that allocator, linking both crashes (their initializers fight over `malloc`), and `dlopen` does not help because the constructor still runs. Both were tried and both crashed (SIGSEGV). -A small `RawAllocator` (name + alloc/free function pointers) unifies the system `malloc`, jemalloc, and tcmalloc behind one interface for the *added* baseline rows and the percentile recorder. The dedicated `malloc` runners that produce the committed ADR-0014 aggregate numbers are left untouched. +`LD_PRELOAD` sidesteps all of that: one process, one allocator, no mixing. Under `LD_PRELOAD=libjemalloc.so.2` the bench's `malloc` rows — and the pool's own backing store — are served by jemalloc, so a preloaded run is a clean, complete `pool`-vs-*that-allocator* comparison across every scenario (including the percentile table). The bench therefore carries **no allocator-specific code at all**: the default build stays byte-for-byte what ADR-0014 produced and spec §3.3's zero-external-dependency posture holds trivially. A `# allocator:` header line (read from `LD_PRELOAD`, POSIX only; "system malloc" otherwise) discloses which allocator each run's numbers reflect, so the three reports are unambiguous. ### 3. Output contract -Additive per ADR-0014 §6: the aggregate 8-column table is unchanged; baseline rows reuse that schema (extra rows, tagged with the allocator name); percentile data is a new, separate table with its own header; a `# baselines:` header line discloses which allocators are compiled in. No existing row or column changes, so the M7.x report-diffing tooling still parses old and new reports. +Additive per ADR-0014 §6: the aggregate 8-column table is unchanged; percentile data is a new, separate table with its own header; a `# allocator:` header line discloses which allocator this run measured (the `LD_PRELOAD` basename, or "system malloc"). No existing row or column changes, so the M7.x report-diffing tooling still parses old and new reports; a baseline comparison is a set of reports, one per allocator, distinguished by that header line. ### 4. CI -A `bench-baselines` job (Linux) installs the jemalloc + tcmalloc **runtime** shared objects (`libjemalloc2`, `libgoogle-perftools4t64`), builds, and runs `--scenario all --percentiles`, asserting that both baselines loaded at run time and that their rows and the percentile table are present. Like every bench cell it gates on **exit code 0, not numbers** (ADR-0014 §8 — shared runners are too noisy for numeric thresholds). It is Linux-only, never touching the MSVC leg (ADR-0005 §3), consistent with the sanitizer-preset split. +A `bench-baselines` job (Linux) installs the jemalloc + tcmalloc **runtime** shared objects (`libjemalloc2`, `libtcmalloc-minimal4t64`), builds once, and runs `--scenario all --percentiles` three times — plain, `LD_PRELOAD=libjemalloc.so.2`, and `LD_PRELOAD=libtcmalloc_minimal.so.4` — asserting each disclosed the expected allocator, emitted the percentile table, and exited cleanly. Like every bench cell it gates on **exit code 0, not numbers** (ADR-0014 §8 — shared runners are too noisy for numeric thresholds). It is Linux-only, never touching the MSVC leg (ADR-0005 §3), consistent with the sanitizer-preset split. ## Alternatives Considered - **Always-on percentiles (fold p99 into the existing per-repeat `Stats`).** Rejected. p99 over ~9 per-repeat aggregates is meaningless (it is essentially the max), and per-op timing on the default path would perturb the committed ADR-0014 ns/op numbers. Per-op sampling behind an opt-in flag is the only way to get a meaningful p99 without disturbing the frozen numbers. - **HdrHistogram (or another bucketed recorder).** Rejected for now: it is an external dependency, and for this benchmark's needs a `std::vector` of per-op samples with a nearest-rank query is sufficient and keeps the methodology inspectable (the ADR-0014 §1 pedagogy argument). Revisit if memory for the sample vector becomes a constraint at very large iteration counts. -- **Link the allocators at compile time (`find_library` + `-ljemalloc`/`-ltcmalloc`), or `LD_PRELOAD`.** Rejected — this was the first implementation and it *crashed* (SIGSEGV): both libraries export strong `malloc`/`operator new` symbols, so co-linking makes their initializers fight over global `malloc`, and even linking one interposes the process allocator (turning the "malloc" row into that allocator, defeating the side-by-side comparison). `LD_PRELOAD` has the same interposition problem and cannot load two allocators at once. Loading each via `dlopen(RTLD_LOCAL)` and calling only its explicit API sidesteps interposition entirely and lets pool / malloc / jemalloc / tcmalloc appear as four distinct rows in a single run. +- **In-process side-by-side rows — link the allocators (`-ljemalloc`/`-ltcmalloc`) or `dlopen` them.** Rejected after *both* were implemented and *both crashed* (SIGSEGV). The libraries take over global `malloc` on load (strong symbols; tcmalloc also via a constructor), so co-linking makes their initializers fight over `malloc`, linking one silently rebinds the "malloc" row to that allocator, and `dlopen(RTLD_LOCAL)` still runs the constructor. There is no safe way to have two of these allocators plus the system allocator live in one process. `LD_PRELOAD` — one allocator per process, chosen from outside — is the standard, crash-free way to benchmark them, at the cost that a full comparison is N reports rather than N rows in one. - **A hard dependency on jemalloc/tcmalloc.** Rejected — it would break spec §3.3 and the MSVC leg. Feature-detection with a silent skip keeps the default build dependency-free. - **CI numeric thresholds on the baselines/percentiles.** Rejected for the ADR-0014 §8 runner-noise reasons; the new cell is a build/run gate, not a performance gate. @@ -52,18 +52,18 @@ A `bench-baselines` job (Linux) installs the jemalloc + tcmalloc **runtime** sha **Positive** - p99/p999 tail latency is reportable, and the dynamic-pool growth row makes the amortized-growth spike visible where the median hid it — the #105 §6.3 critique is closed. -- Modern baselines (jemalloc/tcmalloc) are available for comparison wherever they are installed, feature-detected and validated in CI. +- Modern baselines (jemalloc/tcmalloc) are available for comparison wherever they are installed, via `LD_PRELOAD`, and validated in CI. - The default build and the committed ADR-0014 numbers are unchanged; zero external dependencies preserved. **Negative** - Per-op percentiles are resolution-bound: on Windows's ~100 ns `steady_clock` tick they quantize, so they are meaningful there only for microsecond-scale tail events, not for sub-tick medians (documented; the aggregate table stays authoritative). -- The `dlopen` baseline path compiles on POSIX and loads the allocators only where they are installed, so it is exercised on the Linux CI cell, not on the maintainer's MSVC box (where it is `#ifdef`-compiled out). +- A baseline comparison is **N reports, not N rows in one** — the user (or CI) re-runs the bench under each `LD_PRELOAD`. This is exercised on the Linux CI cell; the maintainer's MSVC box has no `LD_PRELOAD`, so it reports "system malloc" only. **Tooling / documentation (same PR)** -- [`pool_vs_malloc_bench.cpp`](../../src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp) — the `--percentiles` mode, the `RawAllocator` abstraction, and the guarded baselines. -- [bench `CMakeLists.txt`](../../src/bench/cpp/it/d4np/memorypool/CMakeLists.txt) — links `${CMAKE_DL_LIBS}` for the run-time `dlopen`. +- [`pool_vs_malloc_bench.cpp`](../../src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp) — the `--percentiles` mode and the `# allocator:` header disclosure. +- [bench `CMakeLists.txt`](../../src/bench/cpp/it/d4np/memorypool/CMakeLists.txt) — no allocator link; the `# allocator:` header disclosure is the only bench code touched for baselines. - [`.github/workflows/ci.yml`](../../.github/workflows/ci.yml) — the `bench-baselines` job. - [bench `README.md`](../../src/bench/cpp/it/d4np/memorypool/README.md) — the new columns/baselines and the percentile caveat. - [`ROADMAP.md`](../../ROADMAP.md) item 9.4, spec §7.1, [`CHANGELOG.md`](../../CHANGELOG.md) `Unreleased`. @@ -71,5 +71,5 @@ A `bench-baselines` job (Linux) installs the jemalloc + tcmalloc **runtime** sha ## References - [ADR-0014](0014-microbenchmark-methodology-pool-vs-malloc.md) — the methodology this extends. -- jemalloc `mallocx`/`dallocx` extended API; gperftools tcmalloc `tc_malloc`/`tc_free`. +- jemalloc and gperftools tcmalloc — `LD_PRELOAD` drop-in allocator replacements; both take over global `malloc` on load, which is why they are measured out-of-process here. - Gil Tene, *How NOT to Measure Latency* — the case for percentiles/tail over averages in latency reporting. diff --git a/docs/specs/01_spec_cpp_memory_pool.md b/docs/specs/01_spec_cpp_memory_pool.md index 0e759b6..5441f70 100644 --- a/docs/specs/01_spec_cpp_memory_pool.md +++ b/docs/specs/01_spec_cpp_memory_pool.md @@ -204,7 +204,7 @@ valgrind --leak-check=full --show-leak-kinds=all ./test_pool ### 6.3 Performance Benchmark -`memory_pool_alloc`/`free` are compared against standard `malloc`/`free`. The methodology is fixed by [ADR-0014](../adr/0014-microbenchmark-methodology-pool-vs-malloc.md): warm-up repeat discarded, min/median/mean/max/stddev reported (not a single wall-clock loop), anti-optimization barriers, disclosed compiler flags and host, per-release reports under `docs/bench/`, and a non-asserting CI smoke run. A **concurrent** scenario (T threads on a shared pool, `MUTEX` vs `LOCKFREE`) provides the contended baseline. An opt-in `--percentiles` mode adds a per-operation **p50/p90/p99/p999 tail-latency** table (surfacing, for a dynamic pool, the microsecond-scale growth spike the median averages away), and **jemalloc / tcmalloc** are available as optional baselines (loaded at run time via `dlopen` so they never interpose the system `malloc`) alongside it — both additive and keeping the default build dependency-free ([ADR-0045](../adr/0045-benchmark-percentiles-and-external-baselines.md)). +`memory_pool_alloc`/`free` are compared against standard `malloc`/`free`. The methodology is fixed by [ADR-0014](../adr/0014-microbenchmark-methodology-pool-vs-malloc.md): warm-up repeat discarded, min/median/mean/max/stddev reported (not a single wall-clock loop), anti-optimization barriers, disclosed compiler flags and host, per-release reports under `docs/bench/`, and a non-asserting CI smoke run. A **concurrent** scenario (T threads on a shared pool, `MUTEX` vs `LOCKFREE`) provides the contended baseline. An opt-in `--percentiles` mode adds a per-operation **p50/p90/p99/p999 tail-latency** table (surfacing, for a dynamic pool, the microsecond-scale growth spike the median averages away), and **jemalloc / tcmalloc** can be measured as baselines by re-running the bench under `LD_PRELOAD` (a `# allocator:` header discloses which) — additive, with no allocator-specific code in the bench, so the default build stays dependency-free ([ADR-0045](../adr/0045-benchmark-percentiles-and-external-baselines.md)). ### 6.4 Sanitizers & CI @@ -239,7 +239,7 @@ Every item once deferred here has now shipped; nothing from the original spec re **Opt-in debug hardening** (freed-block poisoning, a guard word for overflow + double-free detection, and free-list safe-linking), once deferred here, now ships behind the `PBR_MEMORY_POOL_HARDENING` compile-time knob — see [§4.1](#41-constraints--guarantees) and [ADR-0043](../adr/0043-opt-in-debug-hardening.md) (issue #109). -The **benchmark extension** — external allocator baselines (jemalloc/tcmalloc) and p99 tail-latency percentiles — once deferred here, now ships as the opt-in `--percentiles` table and the feature-detected baseline rows — see [§6.3](#63-performance-benchmark) / [§6.4](#64-sanitizers--ci) and [ADR-0045](../adr/0045-benchmark-percentiles-and-external-baselines.md) (issue #111). +The **benchmark extension** — external allocator baselines (jemalloc/tcmalloc) and p99 tail-latency percentiles — once deferred here, now ships as the opt-in `--percentiles` table plus `LD_PRELOAD`-based allocator baselines — see [§6.3](#63-performance-benchmark) / [§6.4](#64-sanitizers--ci) and [ADR-0045](../adr/0045-benchmark-percentiles-and-external-baselines.md) (issue #111). The **coverage-guided fuzzing harness**, once deferred here, now ships as the libFuzzer target `pool_fuzz` plus a portable standalone corpus-replay gate — see [§6.4](#64-sanitizers--ci) and [ADR-0044](../adr/0044-coverage-guided-fuzzing-harness.md) (issue #108). diff --git a/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt b/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt index cb47844..e12b3b3 100644 --- a/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt +++ b/src/bench/cpp/it/d4np/memorypool/CMakeLists.txt @@ -30,17 +30,10 @@ elseif(PBR_MEMORY_POOL_THREAD_SAFETY STREQUAL "LOCKFREE") PBR_MEMORY_POOL_THREAD_SAFETY=PBR_MEMORY_POOL_THREAD_SAFETY_LOCKFREE) endif() -# --------------------------------------------------------------------------- -# Optional external-allocator baselines (ADR-0045). jemalloc / tcmalloc are NOT -# linked — they are loaded at run time via dlopen(RTLD_LOCAL) so they never -# interpose the system malloc and cannot conflict by co-linking (see the source -# for the rationale). All that needs linking is the dynamic-loader library, -# where the platform separates it from libc; ${CMAKE_DL_LIBS} is empty on -# platforms (Windows, newer glibc) that fold dlopen into the C runtime. A -# missing allocator is a silent run-time skip, so the default build stays -# zero-external-dependency (spec §3.3). -# --------------------------------------------------------------------------- -target_link_libraries(pool_vs_malloc_bench PRIVATE ${CMAKE_DL_LIBS}) +# Optional external-allocator baselines (jemalloc / tcmalloc) are obtained by +# re-running the binary under LD_PRELOAD (ADR-0045), which swaps the whole +# process allocator. The bench links nothing allocator-specific, so the default +# build keeps spec §3.3's zero external dependencies. # Property name used by IDEs that group targets; matches the layout under # src/bench/cpp/. diff --git a/src/bench/cpp/it/d4np/memorypool/README.md b/src/bench/cpp/it/d4np/memorypool/README.md index cced300..816cdfe 100644 --- a/src/bench/cpp/it/d4np/memorypool/README.md +++ b/src/bench/cpp/it/d4np/memorypool/README.md @@ -104,13 +104,16 @@ growth pool 3.000 4.000 6.000 7000.000 1000000 ## External allocator baselines (jemalloc / tcmalloc, ADR-0045) -jemalloc and tcmalloc are **optional** baselines, **loaded at run time via `dlopen`** (POSIX). They are deliberately *not* linked: linking either would make its `malloc` interpose the whole process (turning the "malloc" row into that allocator), and linking both crashes. Loading each with `RTLD_LOCAL` and calling only its explicit API (`mallocx`/`dallocx`, `tc_malloc`/`tc_free`) keeps the system `malloc` intact and lets all of pool / malloc / jemalloc / tcmalloc appear as distinct rows in one run. If an allocator's shared object is not installed, it is a silent skip; on non-POSIX hosts (MSVC) the mechanism is compiled out and the output is dependency-free (spec §3.3) but for a `# baselines: malloc` disclosure line. On Debian/Ubuntu just install the runtime libraries and run — no reconfigure needed: +jemalloc and tcmalloc are measured as baselines by **re-running the bench under `LD_PRELOAD`**, which swaps the whole process allocator. This is the only safe way: those libraries take over global `malloc`/`operator new` on load, so they cannot be linked or `dlopen`'d beside the system allocator to produce side-by-side rows (both were tried; both crashed). Under a preload, the bench's `malloc` rows — and the pool's own backing — are served by that allocator, so each run is a clean `pool`-vs-*allocator* comparison; the `# allocator:` header line discloses which. The bench carries no allocator-specific code, so the default build is dependency-free (spec §3.3). On Debian/Ubuntu: ```bash -sudo apt-get install -y libjemalloc2 libgoogle-perftools4t64 +sudo apt-get install -y libjemalloc2 libtcmalloc-minimal4t64 cmake --preset bench && cmake --build --preset bench -./build/bench/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench --scenario all --percentiles -# the header line "# baselines: malloc jemalloc tcmalloc" confirms both loaded +bin=./build/bench/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench +"$bin" --scenario all --percentiles # system malloc +LD_PRELOAD=libjemalloc.so.2 "$bin" --scenario all --percentiles # vs jemalloc +LD_PRELOAD=libtcmalloc_minimal.so.4 "$bin" --scenario all --percentiles # vs tcmalloc +# the "# allocator:" header line records which allocator each run measured ``` ## Reporting @@ -127,7 +130,7 @@ The file wraps the raw benchmark output in a Markdown report disclosing the full ## CI -The `bench-smoke` job in [`.github/workflows/ci.yml`](../../../../../../.github/workflows/ci.yml) builds the bench binary with the `bench` preset and runs it with `--iterations 10000 --repeats 3` — proves the binary still compiles, links, and runs to completion. A companion `bench-baselines` job (Linux) installs jemalloc + tcmalloc and runs `--scenario all --percentiles`, asserting the feature-detected baseline rows and the percentile table are present (ADR-0045). Both deliberately **not** assert numeric thresholds; shared-runner noise makes that gate flaky without adding signal. ADR-0014 §8 documents the rationale. +The `bench-smoke` job in [`.github/workflows/ci.yml`](../../../../../../.github/workflows/ci.yml) builds the bench binary with the `bench` preset and runs it with `--iterations 10000 --repeats 3` — proves the binary still compiles, links, and runs to completion. A companion `bench-baselines` job (Linux) installs jemalloc + tcmalloc and re-runs `--scenario all --percentiles` under each via `LD_PRELOAD`, asserting each run disclosed the expected allocator and emitted the percentile table (ADR-0045). Both deliberately **not** assert numeric thresholds; shared-runner noise makes that gate flaky without adding signal. ADR-0014 §8 documents the rationale. ## Methodology snapshot diff --git a/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp b/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp index 459db21..8aff426 100644 --- a/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp +++ b/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench.cpp @@ -58,16 +58,13 @@ #include #include -// Optional external-allocator baselines (ADR-0045). They are loaded at RUN time -// via dlopen with RTLD_LOCAL (POSIX only) rather than linked, so their malloc -// symbols never interpose the process's system allocator — the "malloc" row -// stays the true system malloc, and jemalloc + tcmalloc cannot conflict by both -// trying to take over global malloc, as they would if co-linked. The default -// build keeps spec §3.3's zero external dependencies. -#if defined(__unix__) || defined(__APPLE__) -#include -#define PBR_BENCH_DLOPEN 1 -#endif +// External-allocator baselines (jemalloc / tcmalloc) are obtained by re-running +// this binary under LD_PRELOAD, which swaps the whole process allocator — the +// only safe way, since those libraries take over global malloc on load, so they +// cannot be linked or dlopen'd alongside the system allocator without crashing. +// The bench itself therefore carries no allocator-specific code and keeps spec +// §3.3's zero external dependencies; the header discloses the active allocator. +// See ADR-0045. namespace mem = it::d4np::memorypool; @@ -131,124 +128,6 @@ inline void touch_byte(void* ptr, std::size_t loop_index) { *static_cast(ptr) = static_cast(loop_index & BYTE_MASK); } -// --------------------------------------------------------------------------- -// A raw (C-style) allocator, described by a name and an alloc/free function -// pair (ADR-0045). This unifies the system `malloc` and the optional external -// baselines (jemalloc / tcmalloc) behind one interface so the generic runners -// and the per-op percentile recorder drive them all through the same code -// path. The existing dedicated `malloc` runners that produce the committed -// ADR-0014 aggregate numbers are left untouched; this abstraction serves the -// *added* baseline rows and the percentile table only. -// --------------------------------------------------------------------------- -struct RawAllocator { - std::string_view name_; - void* (*alloc_)(std::size_t); - void (*free_)(void*); -}; - -// The system allocator, wrapped so its address can travel in a RawAllocator. -// NOLINTNEXTLINE(cppcoreguidelines-no-malloc,cppcoreguidelines-owning-memory) -void* sys_malloc(std::size_t size) { - // NOLINTNEXTLINE(cppcoreguidelines-no-malloc,cppcoreguidelines-owning-memory) - return std::malloc(size); -} -void sys_free(void* ptr) { - // NOLINTNEXTLINE(cppcoreguidelines-no-malloc,cppcoreguidelines-owning-memory) - std::free(ptr); -} - -#ifdef PBR_BENCH_DLOPEN -// The external allocators' explicit extended-API signatures, resolved by dlsym. -using MallocxFn = void* (*)(std::size_t, int); // jemalloc mallocx(size, flags) -using DallocxFn = void (*)(void*, int); // jemalloc dallocx(ptr, flags) -using TcMallocFn = void* (*)(std::size_t); // tcmalloc tc_malloc(size) -using TcFreeFn = void (*)(void*); // tcmalloc tc_free(ptr) - -// Resolved once by load_external_baselines(); a C-style dlsym table -// unavoidably needs mutable globals, mirroring the do_not_optimize sink above. -// NOLINTBEGIN(cppcoreguidelines-avoid-non-const-global-variables) -MallocxFn g_je_mallocx = nullptr; -DallocxFn g_je_dallocx = nullptr; -TcMallocFn g_tc_malloc = nullptr; -TcFreeFn g_tc_free = nullptr; -// NOLINTEND(cppcoreguidelines-avoid-non-const-global-variables) - -// mallocx requires a non-zero size and dallocx a non-null pointer; both hold. -void* je_alloc(std::size_t size) { - return g_je_mallocx(size != 0U ? size : 1U, 0); -} -void je_release(void* ptr) { - if (ptr != nullptr) { - g_je_dallocx(ptr, 0); - } -} -void* tc_alloc(std::size_t size) { - return g_tc_malloc(size); -} -void tc_release(void* ptr) { - g_tc_free(ptr); -} - -void* dlopen_local(const char* primary, const char* fallback) { - void* handle = dlopen(primary, RTLD_NOW | RTLD_LOCAL); - if (handle == nullptr) { - handle = dlopen(fallback, RTLD_NOW | RTLD_LOCAL); - } - return handle; -} - -// Resolve a symbol to a function pointer. dlsym yields a void*; copying the -// bits with memcpy (rather than reinterpret_cast between object- and -// function-pointer types, which -Wpedantic rejects as conditionally-supported) -// is the portable, warning-free idiom. Returns nullptr if the symbol is absent. -template -Fn resolve_symbol(void* handle, const char* name) { - void* const sym = dlsym(handle, name); - Fn fn = nullptr; - if (sym != nullptr) { - std::memcpy(&fn, &sym, sizeof(fn)); - } - return fn; -} - -std::vector load_external_baselines() { - std::vector list; - void* const je = dlopen_local("libjemalloc.so.2", "libjemalloc.so"); - if (je != nullptr) { - g_je_mallocx = resolve_symbol(je, "mallocx"); - g_je_dallocx = resolve_symbol(je, "dallocx"); - if (g_je_mallocx != nullptr && g_je_dallocx != nullptr) { - list.push_back(RawAllocator{"jemalloc", &je_alloc, &je_release}); - } - } - void* tc = dlopen_local("libtcmalloc.so.4", "libtcmalloc.so"); - if (tc == nullptr) { - tc = dlopen_local("libtcmalloc_minimal.so.4", "libtcmalloc_minimal.so"); - } - if (tc != nullptr) { - g_tc_malloc = resolve_symbol(tc, "tc_malloc"); - g_tc_free = resolve_symbol(tc, "tc_free"); - if (g_tc_malloc != nullptr && g_tc_free != nullptr) { - list.push_back(RawAllocator{"tcmalloc", &tc_alloc, &tc_release}); - } - } - return list; -} -#endif // PBR_BENCH_DLOPEN - -// The external baselines available at run time (empty by default and on -// non-POSIX hosts — spec §3.3). Resolved once. System malloc is NOT here: it -// keeps its dedicated committed-number runners; this list drives the added -// baseline rows and the percentile table only. -const std::vector& external_baselines() { -#ifdef PBR_BENCH_DLOPEN - static const std::vector list = load_external_baselines(); -#else - static const std::vector list; -#endif - return list; -} - // --------------------------------------------------------------------------- // Stats over a sample of per-iteration nanosecond costs. The first repeat is // the warm-up and is dropped before this function is called. @@ -404,47 +283,6 @@ double time_malloc_interleaved(const Config& cfg) { return ns_per_iter(t0, t1, cfg.iterations_); } -// --------------------------------------------------------------------------- -// Generic aggregate runners for a RawAllocator (ADR-0045). Same timing method -// as the dedicated malloc runners above — used for the added jemalloc/tcmalloc -// baseline rows, so their numbers are directly comparable to the malloc rows. -// --------------------------------------------------------------------------- -double time_raw_bulk_alloc(const RawAllocator& allocator, const Config& cfg, std::vector& out) { - out.clear(); - out.reserve(cfg.iterations_); - const auto t0 = clock::now(); - for (std::size_t i = 0; i < cfg.iterations_; ++i) { - void* const p = allocator.alloc_(cfg.block_size_); - touch_byte(p, i); - do_not_optimize(p); - out.push_back(p); - } - const auto t1 = clock::now(); - return ns_per_iter(t0, t1, cfg.iterations_); -} - -double time_raw_bulk_free(const RawAllocator& allocator, std::vector& blocks) { - const auto t0 = clock::now(); - for (void* p : blocks) { - do_not_optimize(p); - allocator.free_(p); - } - const auto t1 = clock::now(); - return ns_per_iter(t0, t1, blocks.size()); -} - -double time_raw_interleaved(const RawAllocator& allocator, const Config& cfg) { - const auto t0 = clock::now(); - for (std::size_t i = 0; i < cfg.iterations_; ++i) { - void* const p = allocator.alloc_(cfg.block_size_); - touch_byte(p, i); - do_not_optimize(p); - allocator.free_(p); - } - const auto t1 = clock::now(); - return ns_per_iter(t0, t1, cfg.iterations_); -} - // --------------------------------------------------------------------------- // Per-operation timers for the percentile table (ADR-0045). Each records ONE // latency sample per operation into `out`, so a percentile summary is @@ -467,15 +305,17 @@ void perop_pool_interleaved(mem::Pool& pool, const Config& cfg, std::vector& out) { +void perop_malloc_interleaved(const Config& cfg, std::vector& out) { out.clear(); out.reserve(cfg.iterations_); for (std::size_t i = 0; i < cfg.iterations_; ++i) { const auto t0 = clock::now(); - void* const p = allocator.alloc_(cfg.block_size_); + // NOLINTNEXTLINE(cppcoreguidelines-no-malloc,cppcoreguidelines-owning-memory) + void* const p = std::malloc(cfg.block_size_); touch_byte(p, i); do_not_optimize(p); - allocator.free_(p); + // NOLINTNEXTLINE(cppcoreguidelines-no-malloc,cppcoreguidelines-owning-memory) + std::free(p); const auto t1 = clock::now(); out.push_back(ns_per_iter(t0, t1, 1U)); } @@ -892,11 +732,19 @@ void print_header(std::ostream& os, const Config& cfg) { os << "# hardware_concurrency: " << std::thread::hardware_concurrency() << "\n"; os << "# max_align_t: " << alignof(std::max_align_t) << " bytes\n"; os << "# thread_safety_policy: " << policy_name() << "\n"; - os << "# baselines: malloc"; - for (const RawAllocator& allocator : external_baselines()) { - os << " " << allocator.name_; - } - os << "\n"; + // Disclose the active allocator behind the `malloc` rows. External baselines + // (jemalloc / tcmalloc) are measured by re-running under LD_PRELOAD, which + // swaps the whole process allocator — the only safe way, since those + // libraries take over global malloc on load (ADR-0045). This line records + // which allocator the `malloc` (and pool-backing) numbers actually reflect. + // LD_PRELOAD is POSIX-only; on Windows the allocator is always the system one. +#ifdef _MSC_VER + const char* const preload = nullptr; +#else + // NOLINTNEXTLINE(concurrency-mt-unsafe) + const char* const preload = std::getenv("LD_PRELOAD"); +#endif + os << "# allocator: " << ((preload != nullptr && preload[0] != '\0') ? preload : "system malloc") << "\n"; if (cfg.percentiles_) { os << "# percentiles: on (per-op tail-latency table appended — ADR-0045)\n"; } @@ -1012,45 +860,11 @@ std::string run_and_report_growth(const Config& cfg, std::ostream& body) { return tail.str(); } -// M9.4 — append rows for each optional external baseline (jemalloc / tcmalloc) -// to the main TSV table, using the same aggregate method and column schema as -// the malloc rows so they are directly comparable (ADR-0045). A no-op when no -// baseline is compiled in — the default build's table is byte-for-byte -// unchanged. Writes rows only; the existing malloc/pool headline is unchanged. -void run_and_report_baselines(const Config& cfg, std::ostream& body) { - std::vector slots; - for (const RawAllocator& allocator : external_baselines()) { - if (cfg.run_bulk_) { - std::vector alloc_samples; - std::vector free_samples; - for (std::size_t r = 0; r < cfg.repeats_; ++r) { - const double a = time_raw_bulk_alloc(allocator, cfg, slots); - const double f = time_raw_bulk_free(allocator, slots); - if (r != WARMUP_REPEAT_INDEX) { - alloc_samples.push_back(a); - free_samples.push_back(f); - } - } - print_row(body, RowKey{"bulk", allocator.name_, "alloc"}, summarise(alloc_samples)); - print_row(body, RowKey{"bulk", allocator.name_, "free"}, summarise(free_samples)); - } - if (cfg.run_interleaved_) { - std::vector samples; - for (std::size_t r = 0; r < cfg.repeats_; ++r) { - const double s = time_raw_interleaved(allocator, cfg); - if (r != WARMUP_REPEAT_INDEX) { - samples.push_back(s); - } - } - print_row(body, RowKey{"interleaved", allocator.name_, "alloc+free"}, summarise(samples)); - } - } -} - // M9.4 — the tail-latency table (ADR-0045). A separate TSV section (its own // header) so the ADR-0014 aggregate table's schema is untouched. Per-op timing -// for interleaved across pool / malloc / each baseline, plus a dynamic-pool -// growth row whose p99 / p999 expose the growth spikes. +// for interleaved across pool and the process `malloc` (which an external +// allocator baseline replaces under LD_PRELOAD — see the header disclosure and +// ADR-0045), plus a dynamic-pool growth row whose p99 / p999 expose the spikes. void print_percentile_table_header(std::ostream& os) { os << "scenario\tallocator\tp50_ns/op\tp90_ns/op\tp99_ns/op\tp999_ns/op\tsamples\n"; } @@ -1072,13 +886,8 @@ void run_and_report_percentiles(const Config& cfg, std::ostream& body) { print_percentile_row(body, "interleaved", "pool", percentiles_of(samples)); } - const RawAllocator system_malloc{"malloc", &sys_malloc, &sys_free}; - perop_raw_interleaved(system_malloc, cfg, samples); + perop_malloc_interleaved(cfg, samples); print_percentile_row(body, "interleaved", "malloc", percentiles_of(samples)); - for (const RawAllocator& allocator : external_baselines()) { - perop_raw_interleaved(allocator, cfg, samples); - print_percentile_row(body, "interleaved", allocator.name_, percentiles_of(samples)); - } if (perop_pool_growth(cfg, samples)) { print_percentile_row(body, "growth", "pool", percentiles_of(samples)); @@ -1113,9 +922,6 @@ int main(int argc, char* argv[]) { if (cfg.run_growth_) { tail += run_and_report_growth(cfg, std::cout); } - // Added external-baseline rows (jemalloc / tcmalloc) go into the same table, - // before the headline tail; a no-op when none are compiled in (ADR-0045). - run_and_report_baselines(cfg, std::cout); std::cout << "\n" << tail; // The opt-in per-op tail-latency table is a separate section after the // headlines so the ADR-0014 aggregate table is untouched (ADR-0045). From fe310af30d2fffb98c40a8a63b58ef610f01cc6e Mon Sep 17 00:00:00 2001 From: Daniel Polo <106583643+danielPoloWork@users.noreply.github.com> Date: Thu, 9 Jul 2026 22:19:32 +0200 Subject: [PATCH 4/4] fix(ci): resolve LD_PRELOAD baselines by soname, not `find /` The bench-baselines step died before running: `je="$(find / ... | head -1)"` under `set -euo pipefail` failed because `find /` exits non-zero on the permission-denied pseudo-filesystems, and pipefail propagated that into the command-substitution assignment (which set -e treats as fatal). Drop the filesystem search entirely: LD_PRELOAD resolves a bare soname (libjemalloc.so.2 / libtcmalloc_minimal.so.4) through the loader cache that ldconfig populated on package install. Simpler and robust. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/ci.yml | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b68d53d..d6fae9f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -495,21 +495,18 @@ jobs: run: | set -euo pipefail bin="build/bench/src/bench/cpp/it/d4np/memorypool/pool_vs_malloc_bench" - je="$(find / -name 'libjemalloc.so.2' 2>/dev/null | head -1)" - tc="$(find / -name 'libtcmalloc_minimal.so.4' 2>/dev/null | head -1)" - echo "jemalloc=$je tcmalloc=$tc" - [ -n "$je" ] || { echo "FAIL: libjemalloc.so.2 not found"; exit 1; } - [ -n "$tc" ] || { echo "FAIL: libtcmalloc_minimal.so.4 not found"; exit 1; } - run() { # label, preload-path + # LD_PRELOAD resolves a soname through the loader cache (ldconfig ran on + # install), so no filesystem search is needed. + run() { # label, LD_PRELOAD value ("" = none), expected "# allocator:" text echo "=== allocator: $1 ===" out="$(LD_PRELOAD="$2" "$bin" --scenario all --iterations 20000 --repeats 3 --percentiles)" echo "$out" - echo "$out" | grep -q "# allocator: ${3:-system malloc}" || { echo "FAIL: allocator disclosure ($1)"; exit 1; } + echo "$out" | grep -q "# allocator: $3" || { echo "FAIL: allocator disclosure ($1)"; exit 1; } echo "$out" | grep -q "p99_ns/op" || { echo "FAIL: percentile table missing ($1)"; exit 1; } } run "system malloc" "" "system malloc" - run "jemalloc" "$je" "$je" - run "tcmalloc" "$tc" "$tc" + run "jemalloc" "libjemalloc.so.2" "libjemalloc.so.2" + run "tcmalloc" "libtcmalloc_minimal.so.4" "libtcmalloc_minimal.so.4" echo "OK — pool benchmarked against system malloc, jemalloc, and tcmalloc; percentile table present." # ---------------------------------------------------------------------------