Adopt three-tier test layout with real-Postgres integration tier - #66
Merged
Conversation
- Restructure backend/tests into unit/<area>/, integration/, evals/ tiers; default pytest run is the DB-free unit tier, integration is opt-in via -m integration (CI runs both). - Add tests/integration/conftest.py: schema from the shipped Alembic migrations, per-test outer-transaction rollback, httpx ASGITransport client with the session dependency overridden. Guards refuse non-test database names and skip when Postgres is down. - Add tests/factories.py (make_* unpersisted / create_* persisted builders). - New integration tests: sessions/thread/retrieval routes, models CRUD, pgvector cosine-ordering smoke, migrations round-trip, DB health. New unit tests: title_generator, message_serialization, emotion. Coverage 43% -> 53%. - Switch async tests to anyio's pytest plugin (drop raw asyncio.run wrappers); drop unused main/additional markers. - Document the commit-owning session dependency contract in dependencies/db.py and remove redundant commits in session-context callers (title_generator, seed_sessions); services keep flush-only. - Alembic env: fileConfig(disable_existing_loggers=False) so in-process migrations (integration tier) don't silence app loggers. - CI: pgvector/pgvector:pg17 service container, POSTGRES_DB=cortexdj_test. - backend/scripts/create-test-db.sh + DEVELOPMENT.md three-tier docs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- DB-name guard is now endswith("_test"), not a substring check —
"cortexdj_latest" must never pass a guard that precedes downgrade base.
- Guards fail loudly in CI (skip only locally) so a misconfigured tier
can't silently deselect 28 tests while the job stays green.
- _migrated starts pristine (downgrade base -> upgrade head) so
empty-state assertions hold across runs; tier markers are applied
structurally by path in the integration conftest instead of per-file
pytestmark.
- Move TestModel-backed prepare_tools tests from evals/ to unit/agents/
(they run in the default tier); agent-deps fakes move to tests/fakes.py.
- Update the backend rules Testing section to describe the three tiers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Restructures
backend/tests/into three tiers and adds the infrastructure the previously-untested web/service/model layer needed:unit/<area>/— the existing fast, DB-free tests, now grouped by area (ml/,services/,agents/,scripts/,utils/). Defaultpytestrun.integration/(new, opt-in via-m integration) — real Postgres + HTTP. Schema comes from the shipped Alembic migrations (so the pgvector extension and HNSW index are exercised), each test rolls back an outer transaction, and the httpxASGITransportclient overrides the app's session dependency. Covers sessions/thread/retrieval routes, model CRUD (incl. append-only message history and the Spotify token singleton), a pgvector cosine-ordering smoke test, and a migrations round-trip.evals/— unchanged real-LLM suite (-m eval).Supporting changes:
dependencies/db.pyowns commit/rollback; services and model classmethods onlyflush(). Removed two redundant commits in session-context callers (title_generator,seed_sessions) — behavior-preserving, since the context manager already commits on clean exit. This contract is what makes per-test rollback isolation sound.asyncio.run()wrappers (no pytest-asyncio; anyio ships its own plugin). Unusedmain/additionalmarkers removed.tests/factories.py(make_*unpersisted /create_*persisted) andtests/fakes.py(agent-deps mocks).pgvector/pgvector:pg17service container,POSTGRES_DB=cortexdj_test. Guards fail loudly in CI (and skip locally) if Postgres is unreachable or the DB name doesn't end in_test— the tier refuses non-test databases because it runsalembic downgrade.fileConfig(disable_existing_loggers=False)so in-process migrations don't silence app loggers.title_generator,message_serialization,emotion; TestModel-backedprepare_toolstests moved fromevals/tounit/agents/(they were already running in the default tier).backend/scripts/create-test-db.sh+ DEVELOPMENT.md/rules docs for the new layout.Coverage: 43% → 53.45% (floor stays at 40).
Test plan
uv run --directory backend pytest— unit tier, 175 passed./backend/scripts/create-test-db.sh && POSTGRES_DB=cortexdj_test uv run --directory backend pytest -m integration— 28 passedpre-commit run --all-files(ruff, ruff-format, mypy --strict src tests)🤖 Generated with Claude Code