claude-checkpoint-tooling
634b2e39c0
docs/testing/test_inventory.md: honest, current list of what automated coverage actually exists (38 backend pytest tests across 2 files, 0 Godot tests merged yet) and what doesn't (no integration/endpoint tests at all, no coverage yet for the in-review RS256/moderation work) -- verified against a genuinely fresh checkout + fresh Postgres, not just read off the file tree. scripts/run_checkpoint_tests.py: runs the whole inventory in one shot -- throwaway Postgres, migrations, pytest, then starts the backend server and runs every scripts/smoke_test_*.gd against it headlessly. Verified working both with and without a smoke test present (temporarily tested against PC1's not-yet-merged scripts/api_client.gd + smoke_test_api.gd, removed before this commit -- that PR carries its own files). docs/testing/README.md: what running this at a Package Checkpoint review does and doesn't prove, and how to grow the inventory as real coverage gets added. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
51 lines
2.2 KiB
Markdown
51 lines
2.2 KiB
Markdown
# Testing at Package Checkpoints
|
|
|
|
A Package Checkpoint (PC1, PC2, ...) is a periodic review point — every few
|
|
real milestones, the client and server builds get packaged together and
|
|
independently reviewed before anyone calls anything actually done. See
|
|
`project_agp` in Claude's memory, or ask, for the full definition; this
|
|
doc is just the mechanical "how do I run the tests" part of that review.
|
|
|
|
## Running the suite
|
|
|
|
```
|
|
python scripts/run_checkpoint_tests.py
|
|
```
|
|
|
|
Brings up a throwaway Postgres, runs Alembic migrations against it, runs the
|
|
backend pytest suite, starts the backend server, and runs every Godot
|
|
headless smoke test (`scripts/smoke_test_*.gd`) against it. Tears the
|
|
Postgres container down afterward (`--keep-postgres` to skip that if you
|
|
want to poke at the DB after a failure).
|
|
|
|
If port 5432 is already taken on the machine, pass `--postgres-port` with
|
|
something else.
|
|
|
|
Exit code is 0 only if everything that actually ran passed — suites with
|
|
nothing to run yet (no Godot binary found, no smoke test scripts yet) are
|
|
reported SKIPPED, not treated as a failure.
|
|
|
|
## What this does and doesn't prove
|
|
|
|
Passing this script means: migrations apply cleanly to a genuinely empty
|
|
database, the backend's unit-level logic is correct, and whatever Godot
|
|
smoke tests exist actually round-trip against a live backend. It does
|
|
**not** mean full endpoint/integration coverage exists — see
|
|
`test_inventory.md` for the current, honest list of what's tested and what
|
|
isn't. A checkpoint review should still involve reading the actual diffs
|
|
for anything this script's current coverage doesn't reach, not just running
|
|
green and calling it done.
|
|
|
|
## Growing this over time
|
|
|
|
As real integration tests get added (endpoint tests hitting the FastAPI app,
|
|
more Godot smoke coverage beyond `health_check`), they should land under
|
|
`backend/tests/` / `scripts/smoke_test_*.gd` respectively — this script
|
|
picks up new files automatically, nothing to register by hand. Update
|
|
`test_inventory.md`'s tables when you do, so the inventory stays a source
|
|
of truth instead of quietly going stale.
|
|
|
|
Eventual goal (not implemented yet): gate merges to `main` on this script
|
|
passing, the same way a real CI pipeline would. For now it's a manual step
|
|
run at checkpoint review time.
|