modbison/docs/testing/test_inventory.md

Test Inventory

What automated coverage actually exists, what it actually verifies, and what doesn't have coverage yet. Keep this current as tests are added — it's the thing a Package Checkpoint review (see docs/testing/README.md) reads first to know what "run the tests" actually checked.

Run everything below in one shot with scripts/run_checkpoint_tests.py.

Last verified against main: 2026-08-21 — migrations + pytest + all 11 Godot smoke tests, all clean. PC1's real playtest-found bugs (login form unreachable, 1px world cards, Play → Back losing the session) are covered by the smoke tests below so they can't silently regress. PC2 (basic playable world + character + movement) landed 2026-08-20/21 — see docs/checkpoints/ once a PC2 record exists, and the three new smoke tests below (smoke_test_test_level.gd, smoke_test_player.gd, smoke_test_enter_world.gd) for what's actually verified. The avatar loader/customization system (smoke_test_avatar_system.gd) also landed 2026-08-21 — real glTF-based loading with graceful failure on missing assets, verified since assets/avatars/*.glb model files don't exist yet.

Backend — backend/tests/ (pytest)

Run: cd backend && python -m pytest -v (needs DATABASE_URL pointed at a reachable Postgres — see the runner script).

Current total: 61 tests, all passing on a clean main checkout.

File	Tests	What it covers
test_auth_age_gate.py	11	calculate_age() date-math correctness (exact boundary, leap years, birthday-not-yet-this-year — kept as a tested utility for a future real-DOB pass even though /register no longer calls it) + RegisterRequest.over_18 field tests (true accepted, false accepted, missing rejected). 2026-08-20: the register flow was simplified from collecting/validating a date of birth to a plain client-side "I am 18 or older" confirmation, so the old date-validation parametrized cases were replaced. Backs the 18+ registration gate (PR #23, revised PC1).
test_gltf_metadata.py	21	glTF/GLB binary + JSON parsing: magic-byte detection, chunk-padding handling, mesh/animation/texture-reference metadata extraction, malformed-input rejection. Backs the asset-import pipeline (PR #27).
test_auth_endpoints.py	10	Integration tests for POST /api/auth/register and POST /api/auth/login against the real FastAPI app + real Postgres. Covers: successful register (201), duplicate username (409), duplicate email (409), over_18=false rejection (400), missing over_18 field (422), successful login with correct credentials (200), wrong password (401), nonexistent user (401), form-encoded login (OAuth2), and JWT validity of returned tokens.
test_refresh_endpoint.py	8	Integration tests for POST /api/auth/refresh. Covers: successful refresh (200, new access token, same refresh token returned), refresh produces different access tokens on repeated calls, invalid refresh token (401), empty body (422), missing field (422), refresh token from login also works, returned access token is valid JWT decodable with public key, access_token_expires_minutes in response.
test_refresh_token_jti.py	6	Unit tests for refresh token uniqueness (jti claim) and model column length. Covers: two calls with same sub produce different tokens (jti uniqueness), jti is valid UUID, rapid repeated calls all produce unique jtis, jti survives decode, RefreshToken.token column is String(512), column type is String.
test_auth_expired_token.py	5	Integration tests for expired-token rejection across auth endpoints. Covers: expired access JWT rejected by GET /api/auth/me (401), garbage access token rejected by /me (401), missing bearer token returns 401, expired refresh token (DB expires_at in past) rejected by /refresh (401), expired refresh JWT (past exp claim) rejected before DB lookup (401).

What these are, honestly: a mix of pure unit tests (test_auth_age_gate.py, test_gltf_metadata.py, test_refresh_token_jti.py) and real integration tests (test_auth_endpoints.py, test_refresh_endpoint.py, test_auth_expired_token.py) that hit the FastAPI app against a real Postgres database via TestClient. The integration tests exercise the full stack: Pydantic validation → DB → JWT → response. The suite is also re-runnable without manual cleanup — backend/tests/conftest.py has a session-scoped autouse fixture that truncates all app tables once before the suite runs.

Known coverage gaps (real gaps, not placeholders — nothing enforces filling these yet):

• No endpoint tests for GET /api/worlds, GET /api/assets/*, or any world/asset CRUD routes — only auth endpoints have integration coverage (GET /api/worlds is covered indirectly via the Godot smoke tests below).
• No tests for the moderation schema/migration beyond "the migration runs."
• No tests for the GET /api/auth/me endpoint's response body (only its auth-header validation is covered via expired-token tests).
• No tests for session management (create/validate/revoke sessions).

Godot client — scripts/smoke_test_*.gd

12 smoke tests merged into main. Ten need a live backend, one (smoke_test_network_client.gd) needs a live dedicated server instead, and one (smoke_test_avatar_system.gd) needs neither — it only exercises local catalog/loadout/loader logic. Backend-dependent tests read AGP_API_BASE_URL env var, falling back to the application/agp/api_base_url project setting (default http://127.0.0.1:8000) if unset.

File	What it covers
smoke_test_api.gd	Exercises ApiClient.health_check() against a live backend. Verifies the real HTTP request path (HTTPRequest node lifecycle, JSON parse, result-dictionary contract). Exits 1 on failure.
smoke_test_auth.gd	Tests the full auth flow via ApiClient autoload: register unique user (201), duplicate username (409), over_18=false registration (400), correct login (200), wrong password (401), nonexistent user (401).
smoke_test_e2e.gd	End-to-end flow: health check → register → login → worlds list. Verifies the full request chain with a fresh user, including token storage and get_worlds() with auth header.
smoke_test_world_list.gd	Registers a user, seeds a real world via POST /api/worlds/ (without this, run_checkpoint_tests.py's always-fresh DB means the populated-card render path never actually runs in CI, only the empty-list path would), instantiates world_list.tscn as a real scene, waits for _refresh_worlds() to complete, confirms it reached the populated (not empty) state, and asserts each rendered card's real computed width is >= 200px. That last check is the one that matters: 2026-08-20, a real bug shipped where every card existed in the tree with a non-zero child count and no crash, but rendered at 1.0px wide (ScrollContainer's size_flags_horizontal/vertical used the raw integer 4 = Control.SIZE_SHRINK_CENTER instead of 3 = SIZE_EXPAND_FILL) — invisible on screen, and a child-count-only check would never have caught it. Confirmed by deliberately reintroducing the bug: card width check correctly fails at 25px.
smoke_test_main_menu_ui.gd	Instantiates main_menu.tscn, confirms the auth panel starts hidden, and confirms a real signal emission on LoginButton/RegisterButton (not calling the handler method directly) actually opens it. This is the check that would have caught 2026-08-20's real bug: the auth panel and its logic were fully correct, but nothing in the scene was ever wired to open it, so Login/Register were completely unreachable from the UI.
smoke_test_main_menu_full_flow.gd	Full click-through: open the register form, fill it exactly as a user would, submit, confirm Play actually becomes enabled, the welcome message shows, and the auth buttons hide themselves. Calls the submit handler directly (awaited) rather than via auth_button.pressed.emit() — see the note at the top of the file: a synthetic .emit() on an await-using handler doesn't get driven forward reliably in --script/SceneTree headless mode the way real GUI input event dispatch does, confirmed by comparing direct-call vs .emit() behavior. Not a bug in the actual click-driven flow, just a headless-test-harness quirk to route around when writing tests.
smoke_test_main_menu_back.gd	Regression test for Play → Back logging the user out even with a valid token still held. Registers a user, loads main_menu.tscn (asserts it restores the authenticated state via a real auth_state_ready signal wait, not a frame-count guess), clicks Play → world_list.tscn, clicks Back → main_menu.tscn, and asserts the menu is still authenticated. Checks WelcomeLabel.is_visible_in_tree(), not just .visible — the label lives inside AuthPanel, so .visible alone is a false positive if the panel itself is hidden (a real bug briefly present: _show_authenticated() hid AuthPanel while claiming the welcome label was visible, which rendered nothing).
smoke_test_test_level.gd	Loads scenes/worlds/test_level.tscn directly (the level is player-agnostic and self-contained per docs/architecture/pc2_test_level.md §4) and confirms it has real content: 5 props, a DirectionalLight3D ("Sun"), and a WorldEnvironment — not an empty void.
smoke_test_player.gd	Loads scenes/game.tscn and confirms the player exists as a real CharacterBody3D with a CollisionShape3D+shape, a CapsuleMesh, and spawns at the level's PlayerSpawn marker position. Also asserts JUMP_VELOCITY is positive and a Camera3D exists and is current — both were real bugs found 2026-08-20/21 (jump velocity was negative, pushing the player down instead of up; no camera existed at all, so the game was unplayable even though the character could technically move). Structural checks only — real movement/jump direction correctness was verified separately via a throwaway headless physics simulation during development, not by an automated test that's part of this suite.
smoke_test_enter_world.gd	The full PC2 path end to end: register → seed a real world → load world_list.tscn → click the real "Enter World" button on a real card (found by walking the card's children, not by index, so it doesn't silently pass if the card layout changes) → confirms scenes/game.tscn actually becomes current_scene with the player spawned at the level's PlayerSpawn marker → sends a real KEY_ESCAPE input event (not a direct method call) and confirms it returns to world_list.tscn. This is the test that proves "browse → enter → walk around → leave" actually works as one continuous flow, not just that each piece works in isolation.
smoke_test_network_client.gd	Real two-process integration test for scripts/network_client.gd (client-side prediction + reconciliation, docs/architecture/networking.md §5) — needs an actual dedicated server running first: godot --headless --path . scenes/server.tscn &, not the usual HTTP backend. Connects a real NetworkClient over ENet, confirms local input prediction moves the player immediately (no waiting on a round trip), confirms real snapshots arrive and get processed, then deliberately desyncs the predicted position by 9999 units and confirms the very next snapshot corrects it back — the actual proof reconciliation logic works, not just that it never had to fire. Known simplification, stated in the script's own docstring: the server's snapshot doesn't currently echo back a per-peer "last input seq processed," so reconciliation corrects against the most recent prediction rather than true tick-exact rollback-and-replay (docs/architecture/networking.md §5.2 step 4) — a real, scoped follow-up once the server echoes that ack, not something faked here.
smoke_test_avatar_system.gd	Confirms AvatarSystem is a real autoload, the committed assets/avatars/catalog.json actually parses, get_assets_for_slot("base_body") returns real entries, an AvatarLoadout can set base_body and equip a real catalog accessory, the loadout round-trips through to_json()/load_loadout_from_json(), and — since assets/avatars/*.glb model files don't exist yet — that AvatarLoader.load_base_body() against a real catalog entry with a missing file fails gracefully: returns null and emits load_error rather than crashing. Visual rendering itself can't be verified until real assets exist.
smoke_test_social_client.gd	Headless verification of SocialApiClient methods for friends, presence heartbeats, profiles, groups, and chat.
smoke_test_economy_client.gd	Headless verification of EconomyApiClient methods for wallet, payments, catalog items, listings, and payouts.
smoke_test_wallet_ui.gd	Instantiates scenes/wallet_ui.tscn, verifies balance display, transaction history list rendering, and empty states.
smoke_test_marketplace_ui.gd	Instantiates scenes/marketplace_ui.tscn, verifies keyword/price/type filter controls, active listing cards rendering, and purchase flow.
smoke_test_selling_payout_ui.gd	Instantiates scenes/selling_payout_ui.tscn, verifies catalog item picker, listing creation, and payout request submission.
smoke_test_discovery_client.gd	Headless verification of DiscoveryApiClient typed resource parsing (WorldSummary, DiscoveryResult), and search/browse methods.
smoke_test_discovery_ui.gd	Instantiates scenes/discovery_ui.tscn, verifies search input, category dropdown, sort options (newest/most-played/trending), pagination controls, and world entry integration.

Run (once, to build the class cache): godot --headless --path . --import, then for each smoke test: godot --headless --path . --script res://scripts/smoke_test_<name>.gd — no --quit, it kills these before their awaited HTTP calls finish.

Known gaps:

• No smoke tests for chunk loading/streaming (chunk.gd, chunk_scene.gd).
• No smoke tests for dynamic/static object nodes.
• All smoke tests require a live backend; no offline/unit smoke tests exist.
• smoke_test_world_list.gd and smoke_test_main_menu_*.gd are the only ones that exercise a real .tscn scene instantiation — others only test the ApiClient autoload directly.
• get_me() (GET /auth/me) is implemented in scripts/api_client.gd but has no smoke coverage yet.
• The Login path (as opposed to Register) through main_menu.gd isn't separately smoke-tested — it shares the same _on_auth_button_pressed()/ _on_auth_complete() code path already exercised for Register, but that's inference, not a direct check.

Client export

scripts/build.sh produces real Linux (agp-game.x86_64) and Windows (agp-game.exe) binaries via --export-release. Verified 2026-08-20: both exports complete clean from a fresh checkout and produce valid binaries (confirmed via file — real ELF64/PE32+, not empty or truncated). Neither export is exercised by an automated GUI test (headless Godot can't drive a window) — actually launching and clicking through the exported binary is a manual step, see docs/checkpoints/PC1.md for playtest instructions.

What "full coverage" should eventually mean here

Not defined precisely yet — the honest target as of 2026-08-20 is "integration tests for every endpoint + smoke tests for every major scene." As real integration/endpoint tests get added, update the tables above rather than letting this doc drift stale; a wrong inventory is worse than no inventory.