Skip to content

Populate the benchmark, stress and property-based test tiers - #90

Open
tschm wants to merge 1 commit into
alihaskar:masterfrom
tschm:fix/86-benchmark-stress-tiers
Open

Populate the benchmark, stress and property-based test tiers#90
tschm wants to merge 1 commit into
alihaskar:masterfrom
tschm:fix/86-benchmark-stress-tiers

Conversation

@tschm

@tschm tschm commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Closes #86

Problem

The Makefile wires up three test tiers this repo never filled:

  • make test passed --ignore=tests/benchmarks --ignore=tests/stress for directories that did not exist.
  • make benchmark and make stress had nothing to run — and pytest-benchmark emitted a warning on every make test run about benchmarks being disabled, for benchmarks that did not exist.
  • hypothesis shipped via .rhiza/requirements/tests.txt without a single import.

Change

tests/benchmarks/ measures the two hot paths: DataManager.get_chunk, which runs on every pan and zoom, and validate_input, which dominates plot() start-up. Measured here:

Benchmark Mean
get_chunk — 5k-bar viewport slice 218 µs
validate_input — 250k bars 373 µs
DataManager construction — 250k bars 402 µs
get_chunk — full 250k-bar serialize 14.5 ms

tests/stress/ pushes the shared _data_managers registry that the unit suite only ever touches one session at a time: 100 concurrent registrations, 200 overlapping reads of one manager, and create/delete churn.

Property-based tests state validate_input's contract as invariants rather than examples — every series normalizes to a length-n ndarray, length disagreement always raises, auto-filled high never falls below low, and list and ndarray inputs agree. They carry the property marker pytest.ini already registers, so make hypothesis-test selects them.

hypothesis is added to the test dependency group so uv run pytest works without make install having run first — matching that group's stated intent in the existing comment.

A note on the stress tests

The first draft of test_many_concurrent_sessions_stay_isolated offset only close, leaving high below it. validate_input correctly rejected that with High must be >= max(Open, Close) — a good sign for the validation path. The test now shifts the whole bar.

Verification

  • make test — 176 passed (was 172), coverage 100%
  • make stress — 4 passed, 180 deselected
  • make hypothesis-test — 4 passed, 50 examples each
  • make benchmark — 4 benchmarks, numbers above
  • check_test_layout.py — parity still holds; both new directories are exempt by design

🤖 Generated with Claude Code

The Makefile wires up three test tiers this repo never filled. `make test`
passed --ignore=tests/benchmarks --ignore=tests/stress for directories that
did not exist, `make benchmark` and `make stress` had nothing to run, and
hypothesis shipped via .rhiza/requirements/tests.txt without a single import.

tests/benchmarks/ measures the two hot paths: DataManager.get_chunk, which
runs on every pan and zoom, and validate_input, which dominates plot()
start-up. On this machine a 5k-bar viewport slice costs ~218us and a full
250k-bar serialize ~14.5ms.

tests/stress/ pushes the shared _data_managers registry the unit suite only
touches one session at a time: 100 concurrent registrations, 200 overlapping
reads of one manager, and create/delete churn.

The property tests state validate_input's contract as invariants rather than
examples — every series normalizes to a length-n ndarray, length disagreement
always raises, auto-filled high never falls below low, and list and ndarray
inputs agree. They carry the `property` marker that pytest.ini already
registers, so `make hypothesis-test` selects them.

hypothesis is added to the test dependency group so `uv run pytest` works
without `make install` having run first, matching that group's stated intent.

Both new directories are exempt from check_test_layout.py by design, so
test/source parity still holds. Coverage stays at 100%.

Closes alihaskar#86

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 17, 2026 08:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request fills out the previously-empty test tiers by adding benchmark and stress suites, plus property-based tests for validate_input, and wires hypothesis into the standard test dependency group so uv run pytest works without a separate install step.

Changes:

  • Add tests/benchmarks/ with pytest-benchmark benchmarks for validate_input and DataManager.get_chunk.
  • Add tests/stress/ to exercise the global _data_managers registry under concurrent access patterns.
  • Add Hypothesis-based property tests for validate_input and include hypothesis in the test dependency group.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
uv.lock Locks hypothesis and its dependency (sortedcontainers) and adds it to the test group.
pyproject.toml Declares hypothesis under [project.optional-dependencies].test to support uv run pytest.
tests/benchmarks/init.py Introduces the benchmarks test tier and documents exclusion from the default unit run.
tests/benchmarks/test_ingestion_benchmarks.py Adds ingestion and slicing benchmarks for the core hot paths.
tests/stress/init.py Introduces the stress test tier and documents exclusion from the default unit run.
tests/stress/test_session_registry_stress.py Adds concurrency and churn stress tests around the session registry and slicing path.
tests/pycharting/data/test_ingestion.py Adds Hypothesis property-based tests expressing validate_input invariants.
Suppressed comments (2)

tests/benchmarks/test_ingestion_benchmarks.py:44

  • This benchmark test uses the ohlc fixture whose values are not all np.ndarray (the index is a pd.DatetimeIndex). The ohlc parameter type should be loosened to avoid incorrect typing.
def test_validate_input_on_dense_ohlc(benchmark, ohlc: dict[str, np.ndarray]) -> None:

tests/benchmarks/test_ingestion_benchmarks.py:63

  • This benchmark test uses the ohlc fixture whose values are not all np.ndarray (the index is a pd.DatetimeIndex). The ohlc parameter type should be loosened to avoid incorrect typing.
def test_data_manager_construction(benchmark, ohlc: dict[str, np.ndarray]) -> None:

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +25 to +27
def ohlc() -> dict[str, np.ndarray]:
"""A dense OHLC series of ``N`` bars with a datetime index and two overlays."""
rng = np.random.default_rng(seed=0)
Comment on lines +39 to +41
def manager(ohlc: dict[str, np.ndarray]) -> DataManager:
"""A ``DataManager`` over the dense series, built once for the module."""
return DataManager(**ohlc)


def test_concurrent_chunk_reads_are_consistent(client: TestClient) -> None:
"""Overlapping viewport reads of one session all return the same bytes."""
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Populate or drop the empty benchmark and stress test tiers

2 participants