claude-checkpoint-tooling
634b2e39c0
docs/testing/test_inventory.md: honest, current list of what automated coverage actually exists (38 backend pytest tests across 2 files, 0 Godot tests merged yet) and what doesn't (no integration/endpoint tests at all, no coverage yet for the in-review RS256/moderation work) -- verified against a genuinely fresh checkout + fresh Postgres, not just read off the file tree. scripts/run_checkpoint_tests.py: runs the whole inventory in one shot -- throwaway Postgres, migrations, pytest, then starts the backend server and runs every scripts/smoke_test_*.gd against it headlessly. Verified working both with and without a smoke test present (temporarily tested against PC1's not-yet-merged scripts/api_client.gd + smoke_test_api.gd, removed before this commit -- that PR carries its own files). docs/testing/README.md: what running this at a Package Checkpoint review does and doesn't prove, and how to grow the inventory as real coverage gets added. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
79 lines
4.2 KiB
Markdown
79 lines
4.2 KiB
Markdown
# Test Inventory
|
|
|
|
What automated coverage actually exists, what it actually verifies, and what
|
|
doesn't have coverage yet. Keep this current as tests are added — it's the
|
|
thing a Package Checkpoint review (see `docs/testing/README.md`) reads first
|
|
to know what "run the tests" actually checked.
|
|
|
|
Run everything below in one shot with `scripts/run_checkpoint_tests.py`.
|
|
|
|
Last verified against `main`: 2026-08-19 (commit `1c75367`).
|
|
|
|
## Backend — `backend/tests/` (pytest)
|
|
|
|
Run: `cd backend && python -m pytest -v` (needs `DATABASE_URL` pointed at a
|
|
reachable Postgres — see the runner script; most of these tests don't
|
|
actually touch the DB, but the app's import chain requires a valid URL to
|
|
construct).
|
|
|
|
**Current total: 38 tests, all passing on a clean `main` checkout.**
|
|
|
|
| File | Tests | What it covers |
|
|
|---|---|---|
|
|
| `test_auth_age_gate.py` | 10 | `calculate_age()` date-math correctness (exact boundary, leap years, birthday-not-yet-this-year) + `RegisterRequest` date-of-birth validation (parametrized valid/invalid date rejection). Backs the 18+ registration gate (PR #23). |
|
|
| `test_gltf_metadata.py` | 21 | glTF/GLB binary + JSON parsing: magic-byte detection, chunk-padding handling, mesh/animation/texture-reference metadata extraction, malformed-input rejection. Backs the asset-import pipeline (PR #27). |
|
|
|
|
**What these are, honestly:** pure unit tests of standalone functions
|
|
(`calculate_age`, the glTF parser, Pydantic validators). None of them spin
|
|
up the FastAPI app or hit a real endpoint — they run in ~1s and don't
|
|
actually need Postgres reachable at all (the import chain just needs a
|
|
syntactically valid `DATABASE_URL`, nothing connects during collection).
|
|
|
|
**Known coverage gaps** (real gaps, not placeholders — nothing enforces filling
|
|
these yet):
|
|
- No integration/endpoint tests at all — `/register`, `/login`, `/refresh`,
|
|
`/worlds`, `/assets/*` have zero tests that actually call them through
|
|
the FastAPI app (e.g. via `httpx.AsyncClient` + `ASGITransport`) against a
|
|
real (or test) database. Everything currently shipped works because it
|
|
was manually verified at merge time (checkpoint review), not because a
|
|
test asserts it.
|
|
- No tests for the RS256 auth upgrade (key generation, refresh-token
|
|
issuance/persistence) — PR #30, pending review as of this writing,
|
|
manually verified (correct key usage, all 38 existing tests still pass)
|
|
but not covered by an automated test of its own yet.
|
|
- No tests for the moderation schema/migration (in progress as of this
|
|
writing) beyond "the migration runs."
|
|
|
|
## Godot client — `scripts/smoke_test_*.gd`
|
|
|
|
**Nothing merged into `main` yet as of this writing** — `sandbox_probe/test_script.gd`
|
|
is the only Godot script on `main` right now, and it's **not a real
|
|
test** (a leftover manual probe, returns a hardcoded string, asserts
|
|
nothing). Don't count it toward coverage; delete or repurpose it if anyone
|
|
actually needs a sandbox probe again.
|
|
|
|
`scripts/smoke_test_api.gd` (PC1, PR #29, pending review as of this writing)
|
|
will be the first real one once merged — `ApiClient` autoload's
|
|
`health_check()` against a **live backend** (set `AGP_API_BASE_URL` to point
|
|
at one), verified by hand end-to-end before opening the PR: exercises the
|
|
real HTTP request path (`HTTPRequest` node lifecycle, JSON parse, the
|
|
result-dictionary contract), not just that the script compiles. Exits 1 on
|
|
failure, so it's CI/script-friendly.
|
|
|
|
Run (once merged): `godot --headless --path . --import` once (builds the
|
|
class cache — skip this and every `class_name`-declared class falsely
|
|
errors as undeclared), then for each `scripts/smoke_test_*.gd`:
|
|
`godot --headless --path . --script res://scripts/<name>.gd`
|
|
|
|
**Known gap once it lands:** only `health_check()` is exercised end-to-end.
|
|
`login`, `register`, `get_worlds`, `get_me` are implemented (see
|
|
`scripts/api_client.gd`) but have no smoke coverage yet — each would need a
|
|
real user/world fixture in the target backend to test against meaningfully.
|
|
|
|
## What "full coverage" should eventually mean here
|
|
|
|
Not defined precisely yet — the honest target as of 2026-08-19 is just
|
|
"more than pure-function unit tests." As real integration/endpoint tests get
|
|
added, update the tables above rather than letting this doc drift stale;
|
|
a wrong inventory is worse than no inventory.
|