claude-checkpoint-tooling
634b2e39c0
docs/testing/test_inventory.md: honest, current list of what automated coverage actually exists (38 backend pytest tests across 2 files, 0 Godot tests merged yet) and what doesn't (no integration/endpoint tests at all, no coverage yet for the in-review RS256/moderation work) -- verified against a genuinely fresh checkout + fresh Postgres, not just read off the file tree. scripts/run_checkpoint_tests.py: runs the whole inventory in one shot -- throwaway Postgres, migrations, pytest, then starts the backend server and runs every scripts/smoke_test_*.gd against it headlessly. Verified working both with and without a smoke test present (temporarily tested against PC1's not-yet-merged scripts/api_client.gd + smoke_test_api.gd, removed before this commit -- that PR carries its own files). docs/testing/README.md: what running this at a Package Checkpoint review does and doesn't prove, and how to grow the inventory as real coverage gets added. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2.2 KiB
Testing at Package Checkpoints
A Package Checkpoint (PC1, PC2, ...) is a periodic review point — every few
real milestones, the client and server builds get packaged together and
independently reviewed before anyone calls anything actually done. See
project_agp in Claude's memory, or ask, for the full definition; this
doc is just the mechanical "how do I run the tests" part of that review.
Running the suite
python scripts/run_checkpoint_tests.py
Brings up a throwaway Postgres, runs Alembic migrations against it, runs the
backend pytest suite, starts the backend server, and runs every Godot
headless smoke test (scripts/smoke_test_*.gd) against it. Tears the
Postgres container down afterward (--keep-postgres to skip that if you
want to poke at the DB after a failure).
If port 5432 is already taken on the machine, pass --postgres-port with
something else.
Exit code is 0 only if everything that actually ran passed — suites with nothing to run yet (no Godot binary found, no smoke test scripts yet) are reported SKIPPED, not treated as a failure.
What this does and doesn't prove
Passing this script means: migrations apply cleanly to a genuinely empty
database, the backend's unit-level logic is correct, and whatever Godot
smoke tests exist actually round-trip against a live backend. It does
not mean full endpoint/integration coverage exists — see
test_inventory.md for the current, honest list of what's tested and what
isn't. A checkpoint review should still involve reading the actual diffs
for anything this script's current coverage doesn't reach, not just running
green and calling it done.
Growing this over time
As real integration tests get added (endpoint tests hitting the FastAPI app,
more Godot smoke coverage beyond health_check), they should land under
backend/tests/ / scripts/smoke_test_*.gd respectively — this script
picks up new files automatically, nothing to register by hand. Update
test_inventory.md's tables when you do, so the inventory stays a source
of truth instead of quietly going stale.
Eventual goal (not implemented yet): gate merges to main on this script
passing, the same way a real CI pipeline would. For now it's a manual step
run at checkpoint review time.