WIP checkpoint: Address review feedback: Implement Godot dedicated server scene + player spawning

This commit is contained in:
2026-08-20 23:31:27 +00:00
parent 1c3c57dbd0
commit 4276a89a6a
94 changed files with 9220 additions and 0 deletions
+369
View File
@@ -0,0 +1,369 @@
# Editor Architecture: Godot Editor Plugin vs. In-Game Editor
> **Status:** Proposed
> **Owner:** Platform team
> **Related:** World chunk data format & persistence, server authority + client
> prediction, content moderation & age verification, avatar component system
> **Godot version:** 4.6.2 stable
## 1. Overview
This document evaluates the two primary ways to build the **world/level
authoring tool** for **agp-game**:
- **(a) Godot editor plugin** — extend the Godot editor itself with custom
panels, dock widgets, and tools that operate on the project's scenes and
resources.
- **(b) In-game editor** — a fully custom editor built as game UI (Godot
`Control` nodes) that runs inside the running game, editing the game's own
world data.
The goal is to pick the approach that best fits the platform's constraints:
a **server-authoritative** multiplayer game where the **world is stored as
versioned chunks** (see `world.md`), content is **moderation-flagged** (see
`content_moderation.md`), and the client is a **shipped binary** that players
run (not a dev tool).
This document is a **decision record**: it lays out the trade-offs, gives a
clear recommendation, and defines the resulting architecture boundaries.
---
## 2. Context & Constraints
Before comparing the two approaches, we must fix the constraints that drive the
decision. These come from the existing architecture docs.
### 2.1 The World Is Data, Not Scenes
`world.md` defines the world as **chunk data** (JSON for authoring, binary for
runtime/network), persisted in SQLite on the server. A chunk is the atomic unit
of load, save, and stream. The **server is the only writer** to the store;
clients submit **deltas** against a `base_revision` and the server validates and
applies them.
**Implication:** The editor's output is **chunk payloads**, not `.tscn` scene
files. Whatever editor we build must serialize to the chunk contract, not to
Godot scene resources.
### 2.2 Server-Authoritative & Moderation-Aware
- The server validates every edit (schema, bounds, moderation) before
persisting (`world.md` §5.4).
- Content carries **content flags** aggregated from referenced assets/scripts
(`content_moderation.md` §3). Mature content is gated behind age verification.
- The client receives a **content manifest** and never loads content not in it.
**Implication:** An editor that runs on a **player's machine** cannot be the
authority on what gets published. Editing must go through the server's
validation pipeline, and the editor must surface moderation flags.
### 2.3 The Client Is a Shipped Binary
The game is a multiplayer product. Players run a **release build**. The Godot
editor is a **development tool** that players do not have.
**Implication:** If authoring is a first-class *player* feature (user-generated
content), the editor must ship inside the game binary. If authoring is a
*developer/platform* feature, it can live in the editor.
### 2.4 Two Distinct Audiences
The evaluation must separate two audiences, because they have opposite needs:
| Audience | Who | Where they run | What they build |
|----------|-----|----------------|-----------------|
| **Platform/dev** | The team building the game | Godot editor (dev machines) | Base worlds, rooms, templates, test levels |
| **Players (UGC)** | End users | The shipped game | Their own rooms/worlds, published to the server |
The two approaches map cleanly onto these audiences, and the right answer is
likely **both** — but with a clear primary recommendation.
---
## 3. Approach (a): Godot Editor Plugin
### 3.1 What It Is
A Godot editor plugin is a set of scripts (and optionally scenes) that extend
the editor via the `EditorPlugin` API. It can add:
- **Dock panels** (`add_control_to_dock`) — e.g., a chunk/tile palette panel.
- **Toolbar buttons** (`add_control_to_container`).
- **Inspector plugins** (`EditorInspectorPlugin`) — custom property editors.
- **Import plugins** (`EditorImportPlugin`) — custom asset importers.
- **Scene/Node plugins** (`EditorNode3DGizmoPlugin`) — custom gizmos.
- **Menu entries** (`add_tool_menu_item`).
Plugins are enabled per-project via `project.godot` (`[editor_plugins]`).
### 3.2 Pros
- **Full editor power for free:** You get the scene tree, inspector, viewport
navigation, undo/redo, gizmos, and asset pipeline — all battle-tested.
- **Native Godot editing:** Editing `.tscn` scenes, materials, and nodes is
exactly what the editor is built for. Great for authoring **base content**
(rooms, templates, avatar base bodies).
- **Fast iteration for devs:** The platform team can build and test levels
directly in the tool they already use.
- **Undo/redo, selection, snapping** come from the editor's `EditorUndoRedoManager`.
- **No runtime footprint:** Editor-only code is stripped from export builds
(`@tool` scripts and editor plugins are not shipped to players).
### 3.3 Cons
- **Not available to players.** Players don't run the Godot editor. Any
player-facing UGC authoring cannot use this approach.
- **Tied to the editor's UX.** You inherit the editor's look and interaction
model; deep customization is limited by what the editor API exposes.
- **Editor API churn.** `EditorPlugin` APIs change between Godot versions;
plugins are more brittle across upgrades than game code.
- **Scene-centric, not data-centric.** The editor is optimized for scenes and
resources. Our world is **chunk data** (`world.md`). We'd have to build a
translation layer (scene ↔ chunk) and keep it in sync — a real source of
drift.
- **Single-user, local.** The editor edits local files. It does not natively
talk to our server-authoritative chunk store or the moderation pipeline.
- **Headless/CI friction.** Validating plugin behavior in CI requires running
the editor headless, which is heavier than testing game code.
### 3.4 Fit for Our Constraints
- **Platform/dev authoring:** Excellent. This is the natural home for building
base worlds, rooms, and templates.
- **Player UGC:** Not applicable. Players cannot run the editor.
- **Server-authoritative chunk store:** Poor fit natively. The editor edits
local scenes; we'd bolt on a chunk export/import and a server sync layer.
---
## 4. Approach (b): In-Game Editor
### 4.1 What It Is
A fully custom editor built as **game UI** — Godot `Control` nodes (panels,
buttons, toolbars) that run inside the running game. It edits the game's own
world data model (chunks) and submits edits through the normal server pipeline.
### 4.2 Pros
- **Ships to players.** The editor is part of the game binary, so **player UGC
authoring is possible** — the core requirement for a user-generated-content
platform.
- **Data-native.** The editor operates directly on the **chunk model**
(`world.md`), so there is no scene↔chunk translation layer and no drift.
- **Server-authoritative by construction.** Edits go through the same
`request_chunk_edit` RPCs as gameplay, so validation, revisioning, and
moderation apply uniformly. The editor cannot bypass the server.
- **Moderation-aware.** Because edits flow through the server, content flags are
computed and enforced at the boundary (`content_moderation.md` §4). The editor
can surface flags and gate mature content.
- **One codebase, one UX.** The editor uses the same UI toolkit and art style as
the game; no separate editor theme to maintain.
- **Testable in CI.** The editor is ordinary game code; it can be exercised
headlessly and unit-tested like any other system.
### 4.3 Cons
- **We build everything.** No free scene tree, inspector, gizmos, or undo/redo.
We must implement selection, snapping, undo/redo, viewport navigation, and
property editing ourselves.
- **Runtime footprint.** Editor UI ships in the binary (though it can be gated
behind a feature flag / separate scene that is only instantiated on demand).
- **More upfront work.** A usable in-game editor is a large feature; it needs
its own architecture (tool modes, gizmos, palette, inspector).
- **Not ideal for deep asset authoring.** Modeling a mesh or authoring a
material is better done in the Godot editor / DCC tools. The in-game editor
should focus on **world composition** (placing tiles and objects), not asset
creation.
### 4.4 Fit for Our Constraints
- **Platform/dev authoring:** Workable but not ideal for deep asset work.
- **Player UGC:** The **only** viable approach. Players can only author through
the shipped game.
- **Server-authoritative chunk store:** Excellent fit. The editor is a client
that submits validated deltas.
---
## 5. Comparison Matrix
| Criterion | (a) Editor plugin | (b) In-game editor |
|-----------|-------------------|--------------------|
| Ships to players | ❌ No | ✅ Yes |
| Player UGC authoring | ❌ Impossible | ✅ Core capability |
| World data model fit | ⚠️ Scene-centric; needs translation layer | ✅ Chunk-native |
| Server-authoritative edits | ⚠️ Bolted on | ✅ Native (same RPCs) |
| Moderation enforcement | ⚠️ Manual | ✅ At the boundary |
| Free editor features (undo, gizmos, inspector) | ✅ Rich | ❌ Build ourselves |
| Runtime footprint | ✅ None (stripped from export) | ⚠️ Ships in binary (gated) |
| Dev iteration speed | ✅ Fast | ⚠️ Slower to build |
| Deep asset authoring (mesh/material) | ✅ Native | ❌ Out of scope |
| CI testability | ⚠️ Heavier (headless editor) | ✅ Ordinary game code |
| Editor API churn risk | ⚠️ Higher | ✅ None (game code) |
---
## 6. Recommendation
**Primary recommendation: build the in-game editor (b) as the platform's
authoring surface, and use a lightweight Godot editor plugin (a) only as an
internal dev convenience.**
### 6.1 Rationale
1. **Player UGC is the deciding factor.** The platform's value proposition is
user-generated content (rooms, worlds, avatars). Players cannot run the Godot
editor, so **only an in-game editor can deliver player authoring**. This is a
hard requirement that the editor-plugin approach cannot meet.
2. **The world is data, and the in-game editor is data-native.** `world.md`
defines chunks as the atomic unit. An in-game editor edits chunks directly
and submits them through the server pipeline. An editor plugin would force a
scene↔chunk translation layer that is a constant source of drift and bugs.
3. **Server authority and moderation fall out naturally.** Because in-game
edits use the same `request_chunk_edit` RPCs as gameplay, validation,
revisioning, optimistic concurrency, and content-flag enforcement
(`content_moderation.md`) apply uniformly. The editor cannot bypass the
server — which is exactly what a moderation-aware UGC platform needs.
4. **The editor-plugin's strengths are not what we need.** Its free features
(scene tree, inspector, gizmos) are for editing Godot scenes and resources.
Our authoring is world composition against a chunk contract, not scene
editing. The plugin's main win — deep asset authoring — is better served by
DCC tools and the Godot editor for the *platform team*, not by a plugin we
must maintain.
### 6.2 The Role of the Editor Plugin (Dev Convenience Only)
We still recommend a **small, optional** editor plugin for the **platform team**
to author **base content** (base rooms, templates, avatar base bodies, test
levels) directly in the Godot editor. This is a dev-time convenience, not a
product feature:
- It can export a scene/resource as a **chunk** (reusing the same encoder as the
in-game editor) so dev-authored content flows into the same pipeline.
- It is **not shipped** to players and is **not** the authoring surface for UGC.
- It should be kept thin to minimize editor-API churn risk.
### 6.3 Scope Boundary
To keep the in-game editor tractable, define its scope clearly:
- **In scope (world composition):** placing tiles, placing static/dynamic
objects, setting transforms, editing object properties, saving/loading chunks,
submitting edits to the server, showing moderation flags.
- **Out of scope (asset creation):** modeling meshes, authoring materials,
rigging avatars. These are done in DCC tools / the Godot editor and imported
as catalog assets (see `avatar.md` §7). The in-game editor **references**
catalog assets by id; it does not create them.
---
## 7. Resulting Architecture
### 7.1 Shared Editor Core
Both the in-game editor and the (optional) dev plugin should share a single
**editor core** that operates on the chunk model. This avoids two divergent
implementations.
```
EditorCore (shared, data-native)
├── ChunkEditorModel # in-memory chunk being edited (tiles, objects, flags)
├── ChunkEncoder # JSON/binary encode/decode (world.md §3)
├── EditSession # base_revision tracking, delta generation
├── EditSubmitter # request_chunk_edit RPC wrapper + conflict handling
├── UndoRedoStack # command pattern over chunk mutations
├── SelectionManager # selected tiles/objects
├── ToolRegistry # brush, object-place, select, erase, etc.
└── ModerationView # surfaces content flags; gates mature content
```
### 7.2 In-Game Editor (Primary)
```
InGameEditor (Control-based UI, shipped in binary, gated by feature flag)
├── EditorViewport # custom 2D/3D viewport with camera navigation
├── Toolbar # tool mode switcher
├── PalettePanel # tile/object palette (from catalog)
├── InspectorPanel # property editor for selected object
├── ChunkBrowser # load/save chunks, list rooms
└── EditorCore # shared core (above)
```
- Instantiated only when the player enters "build mode" (a separate scene /
feature flag), so the runtime footprint is negligible during normal play.
- Submits edits via `request_chunk_edit`; handles `chunk_edit_result` conflicts
by re-fetching and re-applying (`world.md` §5.4).
### 7.3 Dev Editor Plugin (Optional, Thin)
```
DevChunkPlugin (EditorPlugin, dev-only, not shipped)
├── Dock panel: "Chunk Export"
├── Reads the current scene / selected nodes
├── Maps scene content → chunk model (via EditorCore.ChunkEncoder)
└── Exports a .chunk.json / submits to a dev server
```
Kept thin to minimize editor-API churn. Its only job is to let the platform team
author base content in the editor and push it into the same chunk pipeline.
### 7.4 Data Flow
```
[In-game editor] --request_chunk_edit(delta, base_revision)--> [Server]
^ | validate
| | apply
+--chunk_edit_result(ok/new_revision | conflict)-----------| persist
v
SQLite (world.db)
[Dev plugin] --export chunk JSON--> [EditorCore.ChunkEncoder] --> [Server]
```
Both paths converge on the **same server validation pipeline**, so moderation
and revisioning are uniform regardless of authoring surface.
---
## 8. Risks & Mitigations
| Risk | Mitigation |
|------|------------|
| In-game editor is a large build | Scope it to world composition (§6.3); reuse EditorCore; ship behind a feature flag |
| Custom viewport/gizmos are hard | Start with a 2D tile editor (matches `CHUNK_SIZE_Z=1` default); add 3D later |
| Undo/redo complexity | Use a command pattern over chunk mutations; test headlessly |
| Editor-plugin API churn | Keep the plugin thin; treat it as dev-only convenience, not a product surface |
| Drift between dev plugin and in-game editor | Both use the same `EditorCore`/`ChunkEncoder` |
| Players bypass moderation | Impossible by construction: edits go through server validation |
---
## 9. Open Questions / Future Work
- **3D editing:** The default world is flat (`CHUNK_SIZE_Z=1`). A 3D in-game
editor (volumetric chunks) is future work (`world.md` §8).
- **Collaborative editing:** Multiple players editing the same chunk concurrently
— the optimistic-concurrency model already supports it, but the UX (live
cursors, conflict resolution) is unspecified.
- **Template library:** A curated set of room templates players can start from.
- **Publish/review flow:** How a player's authored room enters the moderation
review queue (`content_moderation.md` §6) and becomes publicly joinable.
---
## 10. Summary
| Concern | Decision |
|---------|----------|
| Primary authoring surface | **In-game editor** (ships to players, enables UGC) |
| Dev authoring convenience | Thin, optional **editor plugin** (not shipped) |
| World data model | Both operate on the **chunk model** via a shared `EditorCore` |
| Server authority | Edits go through `request_chunk_edit`; server validates |
| Moderation | Enforced at the server boundary for both paths |
| In-game editor scope | World composition only; asset creation is out of scope |
| Runtime footprint | Editor gated behind a feature flag / separate scene |
+194
View File
@@ -0,0 +1,194 @@
# PC2 Test Level & Player Spawn — Design Note
> **Status:** Decided (PC2)
> **Owner:** Architecture
> **Consumers:** the two PC2 implementation tasks — (1) build the test level,
> (2) build the player & spawn it. Both must be able to follow this note
> without further design decisions.
## 1. Decision: static scene, not chunk streaming
**The PC2 playable test level is a single, hand-authored, static Godot 3D
scene.** It is **not** built through the chunk streaming system
(`scripts/chunk_stream_manager.gd`, `chunk_scene.gd`, `chunk.gd`,
`chunk_data_decoder.gd`).
This is a deliberate decision, not a default:
- **The streaming system has never been exercised with real content.** It is a
faithful port of the `docs/architecture/world.md` format spec, but nothing in
the project produces or serves chunk payloads for it to consume. `scenes/main.tscn`
wires `chunk_stream_manager.gd` onto a bare `Node3D` with no `player` reference
and no `_server_rpc`, so at runtime it does nothing. There is no server-side
chunk authoring/streaming endpoint to integrate against.
- **PC2 scope is single-player, local-only, no networking.** The value of chunk
streaming is server-authoritative streaming around a moving player across a
network boundary. None of those preconditions exist in PC2, so exercising the
pipeline here would mean building a chunk *authoring + serving* path first —
clearly out of checkpoint scope.
- **A static scene is the lowest-risk way to get "spawn a player in a real 3D
level and walk around" working**, which is the actual PC2 outcome being tested.
Chunk streaming is **not removed and not broken by this decision** — it stays as
an optional, later milestone. The static level is the PC2 proof of the *player*
and *camera/physics* half of the problem; chunk streaming remains the later proof
of the *world data* half.
## 2. Where things live
| Artifact | Path | Owned by |
|---|---|---|
| Test level scene | `scenes/worlds/test_level.tscn` | level task |
| Player scene | `scenes/player.tscn` | player task |
| Player controller script | `scripts/player.gd` | player task |
| Game composition scene | `scenes/game.tscn` | player task |
- **`scenes/worlds/`** is the convention for all level scenes (it does not exist
yet; the level task creates it). Future levels follow `scenes/worlds/<name>.tscn`.
- `scenes/player.tscn` is **reusable** across levels — it must not hard-code any
reference to `test_level`; it discovers the spawn point at runtime (see §3).
## 3. Player spawn convention
The level scene **must** contain a `Marker3D` named exactly **`PlayerSpawn`**.
That node's world transform is the spawn point.
- `Marker3D` is a plain `Node3D` that shows a cross gizmo in the editor and
carries no gameplay behavior — it is the established Godot idiom for "position
hint." (Confirmed present in Godot 4.6.2.)
- The marker's **position** is authoritative; its rotation is ignored by the
player (spawn yaw is the player's concern, default forward is fine).
- Coordinate system: **Y-up, units = meters, 1 unit = 1 m.** The marker must sit
at least **1.0 m above the ground's top surface** (recommended `y = 1.0` where
the ground top is `y = 0`) so the player capsule does not spawn intersecting
the floor and get popped out by the physics solver.
- There is exactly **one** `PlayerSpawn` per level. (If a level ever needs
multiple spawns, that is a later convention — not PC2.)
**Lookup rule (implemented by `player.gd`, not by the level):**
```
# in player.gd _ready()
var spawn := get_tree().current_scene.find_child("PlayerSpawn", true, false)
if spawn is Node3D:
global_position = spawn.global_position
else:
push_warning("player.gd: no PlayerSpawn marker found; spawning at origin")
global_position = Vector3(0, 1.0, 0) # fallback, do not crash
```
The lookup runs from the current scene root so it finds the marker whether it is
a direct child of the level instance or nested under a `Level` wrapper. The
fallback keeps the player runnable in a scene that forgot its marker, but a
**missing `PlayerSpawn` is a level bug** and the level task's smoke test must fail
on it (see §6).
## 4. Composition (who puts level + player together)
`scenes/game.tscn` is the PC2 playable entry point:
```
Game (Node3D)
├── Level (instance of res://scenes/worlds/test_level.tscn)
└── Player (instance of res://scenes/player.tscn)
```
- `Game` is a plain `Node3D` with **no script** — composition is done by scene
instantiation, not code, so there is nothing to keep in sync.
- The **level scene is self-contained and has zero dependency on the player** —
it loads and is smoke-testable on its own. The level task does not instance or
reference `player.tscn`.
- The **player scene is self-contained and has zero dependency on any specific
level** — it only requires that the *current scene* contains a `PlayerSpawn`.
- Ownership of `game.tscn` goes to the **player task**, which depends on the
level task having landed `test_level.tscn` first (chain with `depends_on`).
## 5. Level content requirements (level task)
`scenes/worlds/test_level.tscn`, root node `TestLevel` (`Node3D`). Required
children:
| Node | Type | Notes |
|---|---|---|
| `Ground` | `StaticBody3D` | `BoxShape3D` (e.g. `40 × 1 × 40`) centered at `(0, -0.5, 0)` so the **top face is `y = 0`**; matching `MeshInstance3D` (`BoxMesh`) so it is visible and collidable. Use a `StaticBody3D` rather than a bare `PlaneMesh` so collision is explicit and reliable. |
| `Props` | `Node3D` | 3–5 simple props as `StaticBody3D` + `MeshInstance3D` using built-in primitives only (`BoxMesh`, `CylinderMesh`, `SphereMesh`). `assets/` is empty and PC2 must not require imported content. Positioned on the ground surface; scale freely. |
| `Sun` | `DirectionalLight3D` | Pitch it downward (~ -45°), enable shadows if the compatibility renderer allows; shadows are optional. |
| `WorldEnvironment` | `WorldEnvironment` | Ambient light and/or a simple background color. Keep it plain — the project uses `gl_compatibility`. |
| `PlayerSpawn` | `Marker3D` | Per §3. Recommended `(0, 1.0, 0)`. |
Notes:
- **Do not** add any `script` to the level. It is pure content. Any per-level
behavior belongs to the player (or a future game-mode script), not the level.
- Keep the whole level on the XZ plane around the origin; nothing off at extreme
coordinates, no thin walls the capsule can tunnel through.
## 6. Player contract (player task)
`scenes/player.tscn`, root `Player` (`CharacterBody3D`) + `scripts/player.gd`.
Minimum for PC2:
- **Body:** `CharacterBody3D` with a `CollisionShape3D` using `CapsuleShape3D`
(~`radius 0.3`, ~`height 1.8`) and a matching `CapsuleMesh` for visibility.
- **Camera:** a `Camera3D` child. First-person (camera at capsule head height) is
the lowest-risk choice; third-person is fine too but is more work.
- **Spawn:** per §3 — read `PlayerSpawn` from the current scene, fall back
gracefully.
- **Movement:** gravity + `move_and_slide()` so the player stands on `Ground`.
WASD/arrow locomotion is the hard requirement. Mouse-look is recommended but
may be split into its own task if the controller grows.
- **No networking, no avatar UI, no chunk code** — PC2 is local-only. Do not add
`MultiplayerAPI`, `ChunkStreamManager`, or any avatar customization here.
The player must work when instanced under `Game` alongside a level; it must not
assume it is the only node in the scene.
## 7. Entry point / how a human plays it
- `scenes/game.tscn` is the PC2 playable scene. For playtesting, load it directly
(e.g. `godot --path . scenes/game.tscn`), or temporarily set
`application/run/main_scene` to it.
- **Wiring "Play → Join a world" in the existing `world_list` UI is out of scope
for this note** and out of scope for the level/player tasks. The current menu
flow ends at browse-worlds (PC1). If a checkpoint wants a clickable path into
the test level, that navigation wiring is its own small task — do not fold it
into level or player.
## 8. Validation (both tasks must run this before "done")
```sh
# one-time, first checkout only — builds the class/import cache
godot --headless --path . --import
# then, to catch parse/load errors:
godot --headless --path . --quit
```
- **Level task:** additionally load `test_level.tscn` headlessly and assert it
(a) loads without error and (b) contains a `Marker3D` named `PlayerSpawn`
positioned ≥ 1.0 m above the ground top. A smoke script following the existing
`scripts/smoke_test_*.gd` pattern is the right vehicle.
- **Player task:** a smoke test that instantiates `game.tscn`, waits a frame,
and asserts the player's global position equals the `PlayerSpawn` position
(within a small epsilon). This is the check that proves the spawn convention
actually wired through — not just "the scene loads."
## 9. Explicitly out of scope for PC2
- Networking / multiplayer / prediction (single-player, local-only).
- Chunk streaming integration and server-side chunk authoring/serving.
- Avatar customization UI and glTF avatar loading.
- Join-a-world navigation from the world list.
- Any content moderation / age-gating logic in the level.
## 10. Summary
| Concern | Decision |
|---|---|
| Level construction | Static hand-authored Godot 3D scene (chunk streaming deferred, not removed) |
| Level location | `scenes/worlds/test_level.tscn` |
| Spawn convention | `Marker3D` named `PlayerSpawn`, ≥ 1.0 m above ground, looked up by the player |
| Player | `scenes/player.tscn` (`CharacterBody3D`) + `scripts/player.gd`, reusable across levels |
| Composition | `scenes/game.tscn` instances `Level` + `Player`; no script |
| Entry point | Load `scenes/game.tscn` directly; UI join-wiring deferred |
+159
View File
@@ -0,0 +1,159 @@
# GDScript Sandboxing for User-Written Scripts
> **Status:** Proposed
> **Owner:** Platform team
> **Related:** Content moderation & age verification (scripts are one of the
> three moderated content types), avatar component system
## 1. Overview
This document evaluates how to safely run **user-written scripts** as part of
UGC (user-generated content) in **agp-game**, and recommends an approach.
**Bottom line: Godot 4 has no built-in security sandbox for GDScript, and
none of the three options named in the original task hold up under testing.**
A script running anywhere inside the Godot process — however it's loaded,
parented, or wrapped — runs with full engine privileges: arbitrary process
execution, full filesystem read/write, and raw network access. Sandboxing
has to happen *outside* the scripting language, not through an in-engine
restriction mechanism, because no such mechanism exists.
This was verified directly against Godot 4.6.2 (`godot --headless`, evidence
in `sandbox_probe/` on this branch) rather than assumed from documentation,
some of which (see §2.1) describes APIs that don't actually exist in this
version.
## 2. Options Evaluated
### 2.1 Godot's built-in "ScriptServer" restrictions — does not exist
There is no `ScriptServer` class in Godot 4.6.2:
```
ClassDB.class_exists("ScriptServer") → false
```
There's also no project setting to disable engine subsystems for scripts —
none of `application/run/disable_os`, `disable_file_access`, or
`disable_network` exist as settings, and `ProjectSettings.get_property_list()`
returns nothing matching "sandbox", "restrict", or "security". Any plan that
assumes a flag like this exists needs to be dropped; it was likely conflated
with sandboxing features from a different engine/scripting ecosystem, or
with **export template stripping** (which controls what ships in a build,
not what a script can call at runtime).
Directly probed capabilities of an ordinary GDScript running as a normal
node script, no special setup:
| Capability | Available to any script? |
|---|---|
| `OS.execute(...)` — run arbitrary processes | ✅ Yes |
| `FileAccess.open(...)` — read/write any path the OS user can reach | ✅ Yes (read `/etc/hostname`, write `/tmp/...` both succeeded) |
| `TCPServer` / `HTTPClient` / `HTTPRequest` / `PacketPeerUDP` / `WebSocketPeer` | ✅ All instantiate and work |
| `ClassDB.instantiate(...)` — construct any engine class by name | ✅ Yes |
| Loading and running a second script from source text at runtime (`GDScript.new()` + `.reload()`) | ✅ Yes |
**Verdict: rejected.** There is nothing to configure here — the restriction
surface this option assumed doesn't exist in the engine.
### 2.2 Isolated SubViewport with a limited API — doesn't isolate scripts
`SubViewport` is a **rendering and input isolation** primitive (it gives a
scene subtree its own render target and can gate input propagation) — it is
not a security or scripting boundary. A script attached to any node inside a
`SubViewport` still runs in the same process, on the same GDScript VM, with
the exact same access to `OS`, `FileAccess`, networking classes, and
`ClassDB` as a script anywhere else in the tree. Nothing about being
"inside" a SubViewport changes what API calls are reachable from a script.
**Verdict: rejected.** This solves a rendering/UI-composition problem, not
the security problem the task is asking about. Worth keeping in mind for
other purposes (e.g. isolated preview rendering of user avatars), but it
provides zero script sandboxing.
### 2.3 Custom GDScript interpreter with an allowlist — right idea, wrong layer for "custom interpreter"
Two ways to read this option, tested separately:
**(a) Write/embed a separate GDScript parser+interpreter.** Not
practical — this means re-implementing (and then indefinitely maintaining,
in lockstep with every Godot release) a full language interpreter outside
the engine. Far more engineering than the problem justifies.
**(b) Use Godot's built-in `Expression` class as a restricted subset.**
`Expression` *looks* like a natural allowlist mechanism — it only evaluates
a single expression, no control flow, no arbitrary statements. But it isn't
safe by default: it can still reach through to unsafe engine methods if you
aren't careful about what object you hand it as a base instance:
```gdscript
var expr = Expression.new()
expr.parse("get_environment('HOME')", ["OS"])
expr.execute([], OS) # → "/home/daddy" (successfully called OS.get_environment)
```
So `Expression` is a viable building block only if the host **never passes
a base instance (or arguments) that exposes anything sensitive** — i.e. the
allowlisting work still has to happen at the call site, `Expression` itself
enforces none of it.
**Static text pattern-matching (blocklisting dangerous substrings like
`"OS.execute"` before parsing) was also probed and is not a real
defense** — it's trivially defeated by string concatenation, `call()`,
`Callable`, or any other indirection, and produces false confidence.
**Verdict: partially viable, but only as a foundation, not a solution on
its own.** A hand-built, narrow, data-driven DSL (not full GDScript, not
raw `Expression`) with an explicit allowlist of safe operations and *no
ability to reference engine singletons or arbitrary objects* is the only
version of "custom interpreter" that's actually safe. This is real
engineering effort, not a config flag.
## 3. Recommendation
**Don't run untrusted GDScript in-process at all.** Given none of the three
in-engine options provide a real boundary, the two approaches actually worth
building are, in order of recommended priority:
1. **Primary: a narrow, non-Turing-complete content DSL**, not general
scripting. Most UGC scripting needs in a game like this (custom
interactions, simple triggers, item behaviors) don't need a general
programming language — they need a small set of declarative
verbs/conditions (e.g. `on_interact: give_item("torch")`,
`on_enter: play_sound("chime")`). Expressed as data (JSON/Resource, not
code) and interpreted by trusted, hand-written GDScript on the engine
side. This has *zero* attack surface by construction — there's no
interpreter to escape because there's no general execution, just a
fixed, reviewed set of verbs. This is almost certainly sufficient for
the actual UGC use cases in this game and should be scoped against real
feature requirements before assuming more is needed.
2. **If genuinely general scripting becomes a hard requirement**, isolate it
at the OS level, not the engine level: run untrusted scripts in a
**separate process** (a stripped-down Godot instance or a dedicated
interpreter) under OS sandboxing (a restricted user account, a
container, or `firejail`/`bwrap`/seccomp), communicating with the main
game over a narrow IPC/RPC boundary that only exposes intentionally-safe
operations. This is the only approach that provides a real security
boundary for arbitrary code, because it doesn't depend on the scripting
language cooperating — the OS enforces it. Substantially more
infrastructure than option 1; only worth it if option 1's DSL genuinely
can't express what's needed.
**Do not** attempt in-process GDScript sandboxing via SubViewport,
ScriptServer, or ad hoc blocklisting — all three were directly tested and
none provide a real boundary (§2).
## 4. Open Questions
- What do actual planned UGC scripts need to *do*? (Trigger conditions,
simple item/state effects, dialogue branching?) This determines whether
the DSL in §3.1 is sufficient or whether §3.2's process isolation is
needed. Needs a concrete feature list before implementation starts.
- If §3.2 is needed later, does it run per-user-session or shared/pooled?
Affects resource cost and isolation guarantees.
- Interaction with the content moderation pipeline (`content_moderation.md`):
scripts as a moderated content type still need a moderation flag path
regardless of which execution model is chosen — a DSL makes static
analysis of *what a script can possibly do* far easier than it would be
for general code, which is an additional point in its favor.
+737
View File
@@ -0,0 +1,737 @@
# World Chunk Data Format & Persistence Architecture
> **Status:** Proposed
> **Owner:** Platform team
> **Related:** Server authority + client prediction, avatar component system, content moderation & age verification, backend auth (JWT)
## 1. Overview
This document specifies the **world chunk data format** and **persistence
schema** for **agp-game**, a Godot-based multiplayer game built by autonomous
agents. It defines:
- The **chunk data format** — both a human-readable **JSON** form (for
authoring, tooling, and debugging) and a compact **binary** form (for
runtime loading and network transfer).
- The **SQLite schema** that persists world state on the server, including
chunk metadata, chunk payloads, and the world/room hierarchy.
- The **read/write API** that the server and client use to load, save, and
stream chunks.
The goal is a **single, versioned chunk contract** that is shared across the
Godot client, the Python backend, and the persistence layer. A chunk is the
atomic unit of world geometry, gameplay objects, and persistence — the server
loads and saves whole chunks, and clients stream the chunks around them.
### 1.1 Design Goals
- **Versioned & forward-compatible:** Every chunk carries a format version so
the loader can migrate old data and reject unknown future data gracefully.
- **Dual representation:** JSON for authoring/debugging, binary for
runtime/network. Both encode the *same* logical model so there is exactly one
source of truth for the schema.
- **Server-authoritative persistence:** The server is the only writer to the
SQLite store. Clients submit *edits* (deltas) and the server applies and
persists them, mirroring the server-authoritative networking model.
- **Chunk-granular persistence:** Persistence happens at chunk granularity so
the server can save a dirty chunk without rewriting the whole world, and can
stream only the chunks a client needs.
- **Deterministic & testable:** The binary format is byte-stable for a given
logical payload, enabling checksums, caching, and reproducible tests.
- **Moderation-aware:** Chunks reference content (assets, scripts) that carry
content flags (see `content_moderation.md`); the persistence layer stores the
moderation state alongside chunk data so enforcement can happen at load time.
---
## 2. World Model & Chunking
### 2.1 World Hierarchy
The world is organized as a hierarchy that matches the moderation content graph:
```
World
└── Room
└── Chunk (the atomic persistence & streaming unit)
├── Tiles / Terrain
├── Static objects (props, geometry)
├── Dynamic objects (spawnable entities, interactables)
└── Metadata (flags, version, timestamps)
```
- A **World** is the top-level container (a named, versioned space).
- A **Room** is a bounded region within a world (a level, a building, a zone).
- A **Chunk** is a fixed-size, axis-aligned region of a room. It is the unit of
load, save, and stream.
### 2.2 Chunk Dimensions
Chunks are **fixed-size, axis-aligned boxes** in world space. The default
dimensions are a configuration constant so they can be tuned per world:
| Constant | Default | Description |
|---------------------|---------|----------------------------------------------------|
| `CHUNK_SIZE_X` | 32 | Chunk width in world units (tiles). |
| `CHUNK_SIZE_Y` | 32 | Chunk height in world units (tiles). |
| `CHUNK_SIZE_Z` | 1 | Chunk depth (1 = flat 2D world; >1 = volumetric). |
| `CHUNK_ORIGIN` | `(0,0)` | World-space origin of chunk coordinate `(0,0)`. |
A chunk is identified by its **integer chunk coordinate** `(cx, cy, cz)`,
computed from a world-space position `p` as:
```
cx = floor((p.x - CHUNK_ORIGIN.x) / CHUNK_SIZE_X)
cy = floor((p.y - CHUNK_ORIGIN.y) / CHUNK_SIZE_Y)
cz = floor((p.z - CHUNK_ORIGIN.z) / CHUNK_SIZE_Z)
```
The chunk's world-space bounding box is therefore:
```
min = (CHUNK_ORIGIN.x + cx*CHUNK_SIZE_X, ...)
max = min + (CHUNK_SIZE_X, CHUNK_SIZE_Y, CHUNK_SIZE_Z)
```
### 2.3 Chunk Coordinate Key
For storage and lookup, a chunk is addressed by a **canonical string key**:
```
"{world_id}:{room_id}:{cx}:{cy}:{cz}"
```
Example: `"w_0001:r_0001:12:-4:0"`. This key is the primary lookup key in the
SQLite `chunks` table and is used in the read/write API.
---
## 3. Chunk Data Format
### 3.1 Logical Model
Both the JSON and binary encodings serialize the same logical chunk model. The
logical model is defined here once; the two encodings are projections of it.
```
Chunk
├── version : uint (format version, currently 1)
├── key : string (canonical chunk key, see §2.3)
├── world_id : string
├── room_id : string
├── coord : { cx, cy, cz } (int)
├── bounds : { min_x, min_y, min_z, max_x, max_y, max_z } (float)
├── tiles : TileLayer[] (terrain / tile data)
├── objects : WorldObject[] (static + dynamic objects)
├── flags : ContentFlags (moderation flags, see §3.5)
├── meta : { created_at, updated_at, revision, author }
└── checksum : string (SHA-256 of canonical payload)
```
### 3.2 Tile Layer
A chunk's terrain is stored as one or more **tile layers**. Each layer is a
dense grid of tile IDs of size `CHUNK_SIZE_X × CHUNK_SIZE_Y` (× `CHUNK_SIZE_Z`
for volumetric worlds).
```
TileLayer
├── name : string (e.g. "ground", "walls", "decor")
├── layer_id : uint
├── width : uint (= CHUNK_SIZE_X)
├── height : uint (= CHUNK_SIZE_Y)
├── depth : uint (= CHUNK_SIZE_Z)
├── tiles : uint[] (flattened tile IDs, row-major, length = w*h*d)
└── palette : TileDef[] (tile ID → tile definition, shared per layer)
```
**Tile encoding:** A tile ID is a `uint16` index into the layer's `palette`.
`0` is reserved for "empty/air". The palette maps each tile ID to a `TileDef`:
```
TileDef
├── id : uint16
├── asset_id : string (reference to a tile/terrain asset)
├── solid : bool (collision)
├── flags : uint32 (bitmask: walkable, climbable, liquid, ...)
└── variants : uint8 (number of visual variants for this tile)
```
**Rationale for palette + indices:** Storing dense `uint16` indices rather than
full asset strings per cell keeps the binary payload small (2 bytes/cell) while
the palette (shared across the layer) carries the rich per-tile metadata.
### 3.3 World Objects
Objects are the non-tile entities in a chunk: static props, interactables,
spawn points, and dynamic-object templates.
```
WorldObject
├── id : string (stable object id, unique within the chunk)
├── type : string (e.g. "prop", "interactable", "spawn_point", "door")
├── asset_id : string (reference to the object's mesh/glTF asset)
├── transform : { pos: [x,y,z], rot: [x,y,z,w], scale: [x,y,z] }
├── properties : object (type-specific key/value map, JSON-typed)
├── dynamic : bool (true if this is a spawnable dynamic entity)
└── flags : ContentFlags (per-object moderation flags)
```
**Dynamic objects** are stored as *templates* in the chunk. Their *runtime
state* (position, health, inventory) is **not** stored in the chunk; it lives
in the runtime entity store and is persisted separately (see §4.4). This keeps
chunks stable and diffable — a chunk only changes when its *static* content
changes.
### 3.4 Content Flags
Each chunk (and each object) carries a **content flag set** that is the
aggregate of the moderation flags of the assets and scripts it references. This
mirrors the aggregation rules in `content_moderation.md`:
```
ContentFlags
├── categories : { violence, sexual, language, substance, gambling, horror, hate, custom }
│ (each → severity: none|mild|moderate|severe)
├── mature : bool (true if any contributing item is mature)
└── sources : string[] (flag ids that contributed, for audit)
```
The server computes and persists these flags at write time so that enforcement
(age gating, content filtering) can happen at load time without re-scanning
assets.
### 3.5 JSON Encoding
The JSON encoding is the **authoring and debugging** representation. It is
human-readable, diffable, and used by tools and tests. It is the canonical
logical model serialized directly.
```json
{
"version": 1,
"key": "w_0001:r_0001:12:-4:0",
"world_id": "w_0001",
"room_id": "r_0001",
"coord": { "cx": 12, "cy": -4, "cz": 0 },
"bounds": { "min_x": 384.0, "min_y": -128.0, "min_z": 0.0,
"max_x": 416.0, "max_y": -96.0, "max_z": 1.0 },
"tiles": [
{
"name": "ground",
"layer_id": 0,
"width": 32,
"height": 32,
"depth": 1,
"palette": [
{ "id": 0, "asset_id": "", "solid": false, "flags": 0, "variants": 1 },
{ "id": 1, "asset_id": "tiles/grass", "solid": false, "flags": 1, "variants": 4 }
],
"tiles": [0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1]
}
],
"objects": [
{
"id": "obj_0001",
"type": "interactable",
"asset_id": "props/chest",
"transform": { "pos": [400.0, -110.0, 0.0],
"rot": [0.0, 0.0, 0.0, 1.0],
"scale": [1.0, 1.0, 1.0] },
"properties": { "loot_table": "chest_basic", "locked": false },
"dynamic": false,
"flags": { "categories": { "violence": "none" }, "mature": false, "sources": [] }
}
],
"flags": { "categories": { "violence": "none" }, "mature": false, "sources": [] },
"meta": { "created_at": "2026-08-17T00:00:00Z",
"updated_at": "2026-08-17T00:00:00Z",
"revision": 3,
"author": "agent-bot" },
"checksum": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
}
```
> **Note on the example checksum:** The value shown above is illustrative only.
> In practice the `checksum` field is the SHA-256 of the canonical binary
> payload body (see §3.6.3), not of the JSON text. The value here is a
> placeholder for documentation purposes.
### 3.6 Binary Encoding
The binary encoding is the **runtime and network** representation. It is compact
and byte-stable. It uses a simple, explicitly-specified layout (no external
serialization library) so it can be produced and consumed identically by the
Godot client (GDScript) and the Python backend.
#### 3.6.1 Header
| Offset | Size | Field | Type / Encoding |
|--------|------|--------------|------------------------------------------|
| 0 | 4 | magic | ASCII `"AGPC"` (0x41 0x47 0x50 0x43) |
| 4 | 2 | version | `uint16` LE (currently `1`) |
| 6 | 2 | header_len | `uint16` LE (bytes of header after this) |
| 8 | 4 | flags | `uint32` LE (bitmask, reserved) |
| 12 | 4 | payload_len | `uint32` LE (bytes of payload body) |
| 16 | 32 | checksum | SHA-256 of payload body (hex, 32 bytes) |
Total header size: **48 bytes**.
#### 3.6.2 Payload Body
The payload body is a sequence of typed sections. Each section has a small
header:
| Offset | Size | Field | Type / Encoding |
|--------|------|------------|----------------------------|
| 0 | 1 | section_id | `uint8` (see table below) |
| 1 | 4 | section_len| `uint32` LE (bytes of section data) |
| 5 | ... | data | section-specific |
Section IDs:
| section_id | Section | Data layout |
|------------|----------------|-------------------------------------------------------------------|
| `0x01` | metadata | world_id, room_id (length-prefixed strings), coord (3×int32), bounds (6×float32) |
| `0x02` | tile layer | name, layer_id, dims (3×uint16), palette, dense tile indices |
| `0x03` | object | one `WorldObject` (repeated; one section per object) |
| `0x04` | content flags | category severities (8×uint8), mature (uint8), sources (string list) |
| `0x05` | meta | created_at, updated_at (int64 unix), revision (uint32), author (string) |
**String encoding:** length-prefixed UTF-8 — `uint16` LE byte length followed by
the UTF-8 bytes.
**Tile layer data layout (section `0x02`):**
```
name : string
layer_id : uint16 LE
width : uint16 LE
height : uint16 LE
depth : uint16 LE
palette_count : uint16 LE
palette : palette_count × TileDef
tile_count : uint32 LE
tiles : tile_count × uint16 LE (dense, row-major)
```
```
TileDef:
id : uint16 LE
asset_id : string
solid : uint8
flags : uint32 LE
variants : uint8
```
**Object data layout (section `0x03`):**
```
id : string
type : string
asset_id : string
pos : 3 × float32
rot : 4 × float32
scale : 3 × float32
dynamic : uint8
properties : JSON string (length-prefixed)
flags : ContentFlags (see 0x04 layout)
```
**Content flags layout (section `0x04`):**
```
violence, sexual, language, substance, gambling, horror, hate, custom
: 8 × uint8 (severity enum: 0=none,1=mild,2=moderate,3=severe)
mature : uint8
source_count: uint16 LE
sources : source_count × string
```
#### 3.6.3 Byte-Stability
For a given logical payload, the binary encoding is **deterministic**:
- All integers are little-endian.
- All strings are UTF-8, length-prefixed, and never null-terminated.
- Maps (e.g. `properties`) are serialized as a **canonical JSON string** with
keys sorted lexicographically.
- The `checksum` is the SHA-256 of the payload body bytes.
This guarantees that two encoders produce identical bytes, enabling content
addressing, caching, and reproducible tests.
---
## 4. SQLite Persistence Schema
The server persists world state in a **SQLite** database. SQLite is chosen for
its zero-configuration, single-file, transactional, and embedded nature — ideal
for a server that owns the world store and needs ACID guarantees without a
separate database process.
### 4.1 Database Layout
The database file is `world.db` (path configurable). It contains the following
tables:
```
worlds
rooms
chunks
chunk_objects
content_flags
revisions
```
### 4.2 `worlds`
Top-level world containers.
```sql
CREATE TABLE worlds (
id TEXT PRIMARY KEY, -- e.g. 'w_0001'
name TEXT NOT NULL,
description TEXT,
chunk_size_x INTEGER NOT NULL DEFAULT 32,
chunk_size_y INTEGER NOT NULL DEFAULT 32,
chunk_size_z INTEGER NOT NULL DEFAULT 1,
origin_x REAL NOT NULL DEFAULT 0,
origin_y REAL NOT NULL DEFAULT 0,
origin_z REAL NOT NULL DEFAULT 0,
created_at TEXT NOT NULL, -- ISO-8601 UTC
updated_at TEXT NOT NULL
);
```
### 4.3 `rooms`
Rooms within a world.
```sql
CREATE TABLE rooms (
id TEXT PRIMARY KEY, -- e.g. 'r_0001'
world_id TEXT NOT NULL REFERENCES worlds(id) ON DELETE CASCADE,
name TEXT NOT NULL,
bounds_min_x REAL NOT NULL,
bounds_min_y REAL NOT NULL,
bounds_min_z REAL NOT NULL DEFAULT 0,
bounds_max_x REAL NOT NULL,
bounds_max_y REAL NOT NULL,
bounds_max_z REAL NOT NULL DEFAULT 1,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE (world_id, name)
);
CREATE INDEX idx_rooms_world ON rooms(world_id);
```
### 4.4 `chunks`
The core persistence table. One row per chunk. The **payload** is stored as a
BLOB in the **binary format** (§3.6); the JSON form is derived on demand for
authoring/debugging and is not stored (to avoid dual-write drift).
```sql
CREATE TABLE chunks (
key TEXT PRIMARY KEY, -- canonical key 'world:room:cx:cy:cz'
world_id TEXT NOT NULL REFERENCES worlds(id) ON DELETE CASCADE,
room_id TEXT NOT NULL REFERENCES rooms(id) ON DELETE CASCADE,
cx INTEGER NOT NULL,
cy INTEGER NOT NULL,
cz INTEGER NOT NULL DEFAULT 0,
format_version INTEGER NOT NULL DEFAULT 1,
payload BLOB NOT NULL, -- binary chunk encoding (§3.6)
payload_len INTEGER NOT NULL,
checksum TEXT NOT NULL, -- SHA-256 of payload
revision INTEGER NOT NULL DEFAULT 0,
dirty INTEGER NOT NULL DEFAULT 0, -- 1 = needs flush to disk
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE (world_id, room_id, cx, cy, cz)
);
CREATE INDEX idx_chunks_world_room ON chunks(world_id, room_id);
CREATE INDEX idx_chunks_coord ON chunks(cx, cy, cz);
```
**Design notes:**
- The `payload` BLOB is the single source of truth for chunk content. The
`checksum` guards against corruption and enables content addressing.
- `revision` increments on every write; it is used for optimistic concurrency
and change detection (see §5.4).
- `dirty` marks chunks that have been modified in memory but not yet flushed,
enabling a write-behind persistence strategy (see §5.5).
### 4.5 `chunk_objects`
A denormalized, queryable index of the objects inside each chunk. This is
**derived** from the chunk payload and kept in sync by the write path. It exists
so the server can answer queries like "which chunks contain a door?" without
parsing every payload BLOB.
```sql
CREATE TABLE chunk_objects (
id TEXT NOT NULL, -- object id (unique within chunk)
chunk_key TEXT NOT NULL REFERENCES chunks(key) ON DELETE CASCADE,
type TEXT NOT NULL, -- 'prop', 'interactable', ...
asset_id TEXT,
dynamic INTEGER NOT NULL DEFAULT 0,
pos_x REAL NOT NULL,
pos_y REAL NOT NULL,
pos_z REAL NOT NULL DEFAULT 0,
PRIMARY KEY (chunk_key, id)
);
CREATE INDEX idx_chunk_objects_type ON chunk_objects(type);
CREATE INDEX idx_chunk_objects_asset ON chunk_objects(asset_id);
```
**Sync mechanism:** `chunk_objects` is maintained **explicitly by the write
path** — it is not a SQL trigger. Whenever `put_chunk` (or `delete_chunk`)
commits a chunk, the service layer performs a **delete-and-reinsert** of that
chunk's object rows in the same transaction: it deletes all rows for
`chunk_key`, then inserts fresh rows parsed from the new payload. This keeps the
index transactionally consistent with the payload and avoids the complexity and
fragility of SQL triggers over a binary BLOB. Because the index is derived, it
can always be rebuilt from `chunks.payload` if it ever drifts (a `rebuild_index`
utility is provided for this).
### 4.6 `content_flags`
Persists the aggregated moderation flags per chunk (and optionally per object).
This lets the server enforce age/content gates at load time without parsing
payloads.
```sql
CREATE TABLE content_flags (
chunk_key TEXT NOT NULL REFERENCES chunks(key) ON DELETE CASCADE,
object_id TEXT NOT NULL DEFAULT '', -- '' = chunk-level flag (sentinel)
violence INTEGER NOT NULL DEFAULT 0,
sexual INTEGER NOT NULL DEFAULT 0,
language INTEGER NOT NULL DEFAULT 0,
substance INTEGER NOT NULL DEFAULT 0,
gambling INTEGER NOT NULL DEFAULT 0,
horror INTEGER NOT NULL DEFAULT 0,
hate INTEGER NOT NULL DEFAULT 0,
custom INTEGER NOT NULL DEFAULT 0,
mature INTEGER NOT NULL DEFAULT 0,
sources TEXT NOT NULL DEFAULT '[]', -- JSON array of flag ids
PRIMARY KEY (chunk_key, object_id)
);
CREATE INDEX idx_content_flags_mature ON content_flags(mature);
```
Severity is stored as `0=none, 1=mild, 2=moderate, 3=severe`.
**Chunk-level vs. object-level flags:** Chunk-level flags use the **sentinel
value `''` (empty string)** in `object_id` rather than `NULL`. This is
deliberate: in SQLite, `NULL` values in a composite primary key are **not**
unique-constrained, so two rows with the same `chunk_key` and `object_id = NULL`
could be inserted accidentally. Using `''` as the sentinel means the composite
`PRIMARY KEY (chunk_key, object_id)` enforces that there is **exactly one**
chunk-level flag row per chunk. Object-level flags use the object's actual `id`
(which is never empty). The application layer treats `object_id = ''` as
"chunk-level" and any non-empty value as "object-level".
### 4.7 `revisions`
An append-only audit log of chunk writes. Enables rollback, auditing, and
replication of changes.
```sql
CREATE TABLE revisions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
chunk_key TEXT NOT NULL, -- NOT a FK: survives chunk deletion
revision INTEGER NOT NULL,
actor TEXT NOT NULL, -- user / agent id
action TEXT NOT NULL, -- 'create' | 'update' | 'delete'
checksum TEXT NOT NULL, -- SHA-256 of payload
payload BLOB, -- full binary payload at this revision
created_at TEXT NOT NULL
);
CREATE INDEX idx_revisions_chunk ON revisions(chunk_key);
```
**Rollback support:** The `payload` BLOB stores the **full binary chunk payload**
at each revision. This is what makes rollback possible: to restore a chunk to a
previous revision, the server reads the `payload` of the target revision and
writes it back to `chunks.payload` (incrementing `revision` and appending a new
`revisions` row). Without the payload, only the checksum would be available,
which is insufficient to reconstruct the chunk.
**Audit trail preservation:** `chunk_key` is **not** a foreign key to `chunks`.
This means deleting a chunk does **not** cascade-delete its revision history —
the audit trail is preserved even after a chunk is removed. (The `chunk_key`
string is still stored so the history can be correlated to a chunk that existed
at that time.) If a chunk is later recreated with the same key, its history is
still intact.
### 4.8 Schema Versioning & Migrations
The database carries a schema version in a `meta` table:
```sql
CREATE TABLE meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
);
-- seed: INSERT INTO meta(key, value) VALUES ('schema_version', '1');
```
Migrations are applied in order at startup. Each migration is a numbered SQL
script; the server tracks the applied version in `meta` and applies any pending
migrations transactionally. This keeps the schema forward-compatible as the
chunk format evolves.
---
## 5. Read/Write API
The API is exposed as a **Python service layer** on the backend (the same
process that owns the SQLite store) and consumed by the Godot client over the
network transport (see `networking.md`). The service layer is the **only**
writer to the database; clients never touch SQLite directly.
### 5.1 Service Interface (Python)
```python
class WorldStore:
"""Owns the SQLite connection and chunk read/write operations."""
def get_chunk(self, key: str) -> Chunk | None: ...
def get_chunk_binary(self, key: str) -> bytes | None: ...
def put_chunk(self, chunk: Chunk, actor: str) -> Chunk: ...
def delete_chunk(self, key: str, actor: str) -> None: ...
def list_chunks(self, world_id: str, room_id: str | None = None,
bounds: Bounds | None = None) -> list[str]: ...
def get_chunk_flags(self, key: str) -> ContentFlags: ...
def flush_dirty(self) -> int: ...
def rollback_chunk(self, key: str, revision: int, actor: str) -> Chunk: ...
def rebuild_index(self, chunk_key: str | None = None) -> int: ...
def migrate(self) -> None: ...
```
### 5.2 Encoding Helpers
The service layer provides symmetric encode/decode functions shared by the
client and server:
```python
def encode_binary(chunk: Chunk) -> bytes: ...
def decode_binary(data: bytes) -> Chunk: ...
def encode_json(chunk: Chunk) -> str: ...
def decode_json(text: str) -> Chunk: ...
def checksum(payload: bytes) -> str: ...
```
These are the single implementation of the format spec (§3), so the Godot
client and Python backend never drift.
### 5.3 Network RPCs
The chunk API is exposed to the client via RPCs (matching the networking
architecture). All chunk traffic is **reliable** (chunks are discrete,
must-arrive data).
| RPC | Direction | Payload | Notes |
|----------------------------|-----------|-------------------------------------------|-------|
| `request_chunk` | C → S | `{ key }` | Client asks for a chunk |
| `chunk_data` | S → C | `{ key, binary_payload, checksum }` | Server streams a chunk |
| `request_chunk_edit` | C → S | `{ key, base_revision, delta }` | Client submits an edit |
| `chunk_edit_result` | S → C | `{ key, success, new_revision, error? }` | Acknowledges an edit |
| `chunk_invalidated` | S → C | `{ key, new_revision }` | Server pushes a change |
| `request_chunk_flags` | C → S | `{ key }` | Age/content gate check |
### 5.4 Edit & Concurrency Model
- **Read:** Client requests a chunk; server returns the binary payload and its
`revision`.
- **Edit:** Client submits a **delta** against a known `base_revision`. The
server applies the delta, increments `revision`, persists, and returns the new
revision.
- **Optimistic concurrency:** If the chunk's current revision differs from the
client's `base_revision`, the server rejects the edit with a conflict error
and the client re-fetches and re-applies. This prevents lost updates when two
clients edit the same chunk.
- **Deltas:** A delta is a JSON patch (RFC 6902 style) applied to the chunk's
logical model. The server validates the patched chunk (schema, bounds,
moderation) before persisting.
### 5.5 Write-Behind Persistence
To amortize disk I/O, the server keeps a **dirty set** of chunks modified in
memory and flushes them to SQLite in batches:
1. `put_chunk` updates the in-memory chunk and marks it `dirty`.
2. A background flusher periodically calls `flush_dirty()`, which writes all
dirty chunks in a single transaction and clears the `dirty` flag.
3. On graceful shutdown, `flush_dirty()` is called to ensure durability.
This keeps the hot path (in-memory chunk access) fast while guaranteeing
durability at shutdown and on a configurable interval.
**Recovery after unclean shutdown:** The `dirty` column in the `chunks` table
exists specifically to support recovery if the process crashes before a flush
completes. On startup, the server scans for rows where `dirty = 1` and treats
them as **unflushed in-memory state that was lost** — it re-reads the last
persisted payload from `chunks.payload` (the last durable state) and clears the
`dirty` flag. In other words, `dirty = 1` at startup means "the last write was
not durable; discard the in-memory delta and fall back to the persisted
payload." This guarantees that the on-disk state is always a consistent,
previously-committed snapshot, never a partially-written chunk. (The in-memory
dirty set is authoritative during normal operation; the `dirty` column is only a
recovery marker.)
### 5.6 Streaming & Caching
- The server streams only the chunks a client needs (those within the client's
view radius), using `list_chunks` with a bounds query.
- Chunks are cached in memory (LRU) keyed by `key`; the cache is invalidated on
write.
- The binary payload is content-addressed by `checksum`, so unchanged chunks can
be served from cache without re-reading SQLite.
---
## 6. Format Versioning & Migration
- The chunk format `version` field (currently `1`) is checked on decode. A
decoder that encounters a higher version than it supports **rejects** the
chunk with a clear error rather than misreading it.
- The SQLite `schema_version` in `meta` drives database migrations (§4.8).
- Chunk payloads are migrated **lazily**: when a chunk is read and its
`format_version` is older than the current one, the server migrates it to the
current version in memory and marks it dirty so it is rewritten on the next
flush. This avoids a costly full-world migration at startup.
---
## 7. Security & Integrity
- **Server-authoritative writes:** Only the `WorldStore` writes to SQLite;
clients submit deltas that are validated before application.
- **Checksum verification:** Every chunk payload is checksummed (SHA-256) on
write and verified on read. Corruption is detected and the chunk is re-fetched
or flagged.
- **Bounds validation:** Chunk coordinates and object positions are validated
against the world's chunk dimensions and room bounds before persistence.
- **Moderation enforcement:** `content_flags` are checked at load time; mature
chunks are gated behind the age-verification flow (see `content_moderation.md`).
- **Input validation:** All string fields are length-limited and validated
against allowed character sets to prevent injection and oversized payloads.
---
## 8. Open Questions / Future Work
- **Volumetric worlds:** The schema and format already support `CHUNK_SIZE_Z >
1`, but the tooling and streaming for 3D chunks are not yet specified.
- **Chunk compression:** For very large worlds, the binary payload could be
compressed (e.g. zstd) before storage/transfer. The header reserves a `flags`
bitmask for a compression flag.
- **Multi-server sharding:** If a single SQLite file becomes a bottleneck, the
`chunks` table could be sharded by `world_id` across multiple databases. The
key design already supports this (world_id is a leading key component).
- **Delta compression for streaming:** Instead of sending full chunks, the
server could send only the delta between revisions to reduce bandwidth for
frequently-updated chunks.
+72
View File
@@ -0,0 +1,72 @@
# Package Checkpoint 1 (PC1)
Validated 2026-08-20 against `main` commit `927f9bd`, from a genuinely fresh
clone (not a reused agent workspace).
## Automated validation
Run with `scripts/run_checkpoint_tests.py` — see
`docs/testing/test_inventory.md` for exactly what each suite covers.
| Check | Result |
|---|---|
| Alembic migrations, fresh DB | PASS |
| Backend pytest | 68/68 PASS |
| Godot smoke tests (4 files) | ALL PASS |
| Linux export (`agp-game.x86_64`) | Builds, verified as a real ELF64 binary |
| Windows export (`agp-game.exe`) | Builds, verified as a real PE32+ binary |
## Problems found during this checkpoint (fixed, tracked as tasks)
- Roadmap showed the Godot Engine Scaffold feature amber despite 6/6 tasks
done — a historical data gap (a review-feedback follow-up task's
supersession link was never recorded, predating when that field started
being tracked). Backfilled in the PM database; confirmed green.
- `docs/testing/test_inventory.md` was stale (said 38 backend tests,
actually 68; said no integration tests existed, 4 real ones had landed).
Fixed via task `398bf1a22a2d` / PR #40.
## What's actually testable right now
Login/register/browse-worlds only — there's no joinable gameplay yet. That's
the real scope of PC1; don't expect more when playtesting.
- Register a new account (18+ only, rejected otherwise)
- Log out, log back in with the same credentials
- Play → Browse Worlds — loads a (currently empty) world list with no errors
## Manual playtest: backend on cloudbison, client on pinkbison
As of 2026-08-20 this is dashboard-controlled — no terminal commands, no
env vars, no editing files. Foolproof enough to hand to someone else.
### 1. On cloudbison — the AGP dashboard
Open the dashboard (`http://cloudbison:7070`, or its Tailscale address) and
go to **PC1** in the nav bar (`/checkpoint/PC1`). Click **Start Backend** —
it pulls the latest `main`, stands up Postgres, runs migrations, and starts
the API server, all in one step. The page shows the LAN address it's
reachable at once it's up (under a minute).
When you're done playtesting, click **Stop & Clean Up** on the same page —
it tears down everything that step started (API server, Postgres
container, its volume and network). Nothing is left running or lying
around afterward, even if the dashboard itself gets restarted in between.
### 2. On pinkbison — download and run the .exe
Same dashboard page: click **Build .exe** the first time (or **Rebuild** to
pick up newer changes) — this pulls the latest `main` and exports a Windows
binary with cloudbison's backend address already baked in. Click
**Download agp-game.exe**, copy it to pinkbison, and just run it. No
environment variable, no config file, no build step on pinkbison at all.
### 3. Tear down when done
**Stop & Clean Up** on the dashboard page. That's it.
## Status
`config/checkpoint_focus_feature_ids` in the AGP framework (`~/agp`) still
restricts agent claiming to the 5 PC1-scope features. Agents will not move
on to post-PC1 work until that's widened.
+41
View File
@@ -0,0 +1,41 @@
# MVP Vertical Slice: Client-Server Wiring — Plan
## Goal
Prove the real Godot client and the real backend can talk: **a user can register/log in
and see a list of worlds from the game client**. Nothing more. Gameplay, join-a-world,
chat, avatar, Discovery search/tags/trending, Social/Economy/Trust & Safety are all
explicitly out of scope.
## Verified backend contract (as of this writing)
- `POST /api/auth/register` — JSON body `{username, email, password, date_of_birth}`.
`date_of_birth` is a `YYYY-MM-DD` string. Returns `201 {access_token, token_type}`.
Rejects under-18 with `400`.
- `POST /api/auth/login` — OAuth2 **form-encoded** (`application/x-www-form-urlencoded`)
`username` + `password` (this is NOT a JSON endpoint). Returns `{access_token, token_type}`.
- `GET /api/worlds` — **requires auth** (`CurrentUser`), returns a JSON array of
`{id, name, description, visibility, owner_id, created_at}`. There is no unauthenticated
public-worlds path, so the worlds list screen can only be shown after login/register.
- `GET /api/health` — unauthenticated, `{status, app}`.
- Base URL: uvicorn default `http://127.0.0.1:8000` (must be configurable in the client).
- CORS is irrelevant to the native Godot `HTTPRequest` client (no browser origin).
## Task breakdown
| # | Type | Task | Depends on |
|---|------|------|-----------|
| 1 | architecture | `ApiClient` autoload: HTTP + JWT storage + error contract | — |
| 2 | implementation | Main-menu **login + register** UI built & wired (form-encoded + JSON/age gate) | 1 |
| 3 | implementation | **World list** screen consuming `GET /api/worlds` | 1 |
| 4 | testing | End-to-end headless smoke test + manual verify | 1,2,3 |
## Design decisions captured (resolved in task 1)
- Token storage: in-memory singleton property for the MVP; a `save_to_disk` seam is left
open but not required yet (no re-login on restart in this slice).
- Error contract: every call returns a discriminated result (`ok`/`error`) so callers
never crash on network failure; non-2xx surfaces `{status, detail}` from the backend.
- Base URL: read from a ProjectSetting, defaulting to `http://127.0.0.1:8000`.
- Login is form-encoded; register is JSON — the client helper must support both content types.
## Notes
- The main-menu auth UI does **not** exist yet (main_menu.gd currently has only Play/Quit
stubs). Task 2 must **build** the login/register UI, not merely wire an existing one.
- The world-list screen also needs to be **built** (a new scene), not just wired.
+295
View File
@@ -0,0 +1,295 @@
# Economy — Currency, Marketplace & Creator Payouts — Plan
> **Status:** Planned
> **Owner:** Platform team
> **Feature ID:** `economy`
> **Godot version:** 4.6.2 stable
> **Backend:** FastAPI + SQLAlchemy (async) + PostgreSQL + Redis
## 1. Overview
This document breaks the **Economy** feature into user stories and right-sized
implementation tasks. The feature delivers:
- **Virtual currency ("Credits")** — a single, server-authoritative currency
with an immutable, append-only transaction ledger.
- **Premium purchase** — buying Credits with real money through a pluggable
payment-provider hook.
- **Marketplace** — user-generated content (UGC) listings where creators sell
items (assets, avatar items, experience passes) to players for Credits.
- **Creator payouts** — accumulating creator earnings and paying them out to
external accounts through a pluggable payout-provider hook.
- **Godot client** — wallet/transaction history, marketplace browse/buy/sell,
and creator payout screens.
The economy is the monetization backbone of the platform. Because it moves
virtual value between real people, correctness (no double-spend, no lost funds,
no balance drift) and safety (fraud prevention, moderation, audit) are treated
as **blocking requirements**, not add-ons.
## 2. Why correctness is the differentiator
The platform is a creator economy (players buy UGC from other players). The
economy is the highest-risk surface after chat: a single arithmetic or
concurrency bug can mint or destroy value and irreversibly erode trust.
Correctness is therefore designed in from the start:
- **One currency, integer units.** Credits are whole-number integers. There are
no floats, no rounding, and no client-computed totals.
- **Append-only ledger.** Every balance change is a row in a `transactions`
table with a signed amount, a `balance_after` snapshot, a reference to the
event that caused it, and an **idempotency key** so retries cannot double
credit or double debit.
- **Atomic, locked mutations.** Balance changes run in a single database
transaction with row locking (`SELECT ... FOR UPDATE`) so concurrent
purchases/payouts cannot race.
- **Escrow hold.** Marketplace proceeds are held in `pending_escrow` for a
configurable window before becoming available to the seller, so fraudulent
or moderated sales can be reversed without clawing back paid-out funds.
- **Server-authoritative fees.** The platform commission is computed
server-side from a configured rate; the client never supplies the fee or the
seller's net.
## 3. Existing constraints (do not re-build)
- **Auth:** JWT bearer tokens; `get_current_user` / `CurrentUser` dependency in
`app/dependencies.py`. All economy endpoints reuse this.
- **DB:** async SQLAlchemy + Alembic; migration `0001_initial` is the head.
Existing tables: `users`, `sessions`, `worlds`, `assets`.
- **Redis:** configured (`REDIS_URL`) but unused so far; adopted here for
idempotency short-caching and rate limiting.
- **Models:** `User` (owner_id), `Asset` (owner_id, is_public, asset_type).
Marketplace items reference existing `assets` rows and/or avatar items.
- **Content moderation:** `docs/architecture/content_moderation.md` defines
flag vocabulary, age gate, and moderation actions. Marketplace items and
listings must inherit and enforce these rules (see §7).
- **Client:** Godot 4.6.2, minimal `main_menu.gd`; no economy UI yet.
## 4. Key design decisions
| Concern | Decision |
|---|---|
| Currency model | Single virtual currency **"Credits" (CR)**, integer units. One wallet row per user; no fractional amounts. |
| Ledger | Append-only `transactions` table: signed `amount`, `balance_after`, `reference_type`/`reference_id`, unique `idempotency_key`. |
| Concurrency | `SELECT ... FOR UPDATE` on the wallet row inside the mutation transaction; never read-modify-write from stale data. |
| Payments | Pluggable `PaymentProvider` interface with a **no-op default** that simulates a confirmed payment for dev/testing (clearly flagged). Stripe/PayPal later. |
| Marketplace items | `catalog_items` (what is sold) + `marketplace_listings` (a seller's offer with a price/status). One item can have multiple listings over time. |
| Purchase flow | Atomic transaction: debit buyer → hold funds in escrow (`pending_escrow`) → grant item. Platform fee computed server-side. |
| Escrow | Configurable hold window (default 72h) before `pending_escrow` becomes available seller earnings. Refunds during the window reverse the sale. |
| Fees | Single configurable `MARKETPLACE_FEE_RATE` (default `0.30`). `seller_net = floor(price * (1 - fee_rate))`. |
| Creator balance | `creator_balances` table: `available` (earned, payable), `pending` (inside an open payout request), `lifetime_earned`. Updated in the same transaction as the sale completion. |
| Payouts | Pluggable `PayoutProvider.pay()` with a no-op default (simulated external ref). Minimum payout threshold, admin review for large payouts. |
| Idempotency | Every money-moving request carries a client-generated `Idempotency-Key`; duplicate keys return the original result. |
| Rate limiting | Redis token-bucket dependency reused from the social feature for purchase/listing/payout endpoints. |
| Moderation | Listings enforce content flags + age gate; moderators can `delist`/`remove`; fraud/chargeback reversal is an audited admin action. |
## 5. User stories
### US-1 Currency & wallet
As a player, I can see my Credit balance and transaction history, and the system
credits/debits my wallet only through server-authoritative events.
**Acceptance criteria**
- Exactly one wallet per user, created lazily on first use.
- Balance is always a non-negative integer; no operation can make it negative.
- Every mutation appends an immutable ledger row with amount, resulting balance,
reference, and idempotency key.
- Retrying a request with the same idempotency key does not double-apply.
- `GET` endpoints expose current balance and paged transaction history.
### US-2 Premium purchase (buy Credits)
As a player, I can purchase Credits with real money and have them credited only
after the payment is confirmed.
**Acceptance criteria**
- A purchase session is created through the payment provider (or the no-op
default) and is idempotent.
- Credits are granted exactly once, only after a verified provider confirmation
(webhook/confirm endpoint), never on client claim.
- Purchase amounts are bounded and server-defined; clients cannot pick arbitrary
prices.
- Confirmation is idempotent and audited (provider reference recorded).
### US-3 Marketplace listing (sell)
As a creator, I can list a catalog item for sale at a Credit price and
manage/remove my listings.
**Acceptance criteria**
- A listing references a valid `catalog_item` owned by the seller (or an
authorized seller).
- Price is bounded (min/max) and set server-side; status transitions are
explicit (`active`, `sold`, `delisted`, `removed_by_moderation`).
- Only the seller (or a moderator) can update/remove a listing.
- Listings are searchable/filterable (item type, price range, owner).
### US-4 Marketplace purchase (buy)
As a player, I can buy a listed item. The purchase is atomic, holds funds in
escrow, grants the item, and splits the payment between platform and seller.
**Acceptance criteria**
- A single transaction debits the buyer, holds funds in escrow, and grants the
item — no partial states on failure.
- Platform fee and seller net are computed server-side from the configured rate.
- After the escrow hold window, funds move from escrow to the seller's
available balance.
- Refunds during the hold window reverse the sale and return funds to the buyer.
- Concurrent purchases of a one-of-a-kind listing cannot both succeed.
### US-5 Creator payouts
As a creator, I can view my earned balance, request a payout, and track its
status; admins can review and approve/reject payouts.
**Acceptance criteria**
- Available balance is derived only from completed (escrow-released) sales and
completed payouts; it never goes negative.
- Payout requests are bounded (min threshold, max per period) and rate-limited.
- Large payouts require admin review; all payout requests carry an audit trail
(requested by, reviewed by, provider reference, status).
- The payout provider is called exactly once; idempotent on retry.
### US-6 Godot client
As a player, I can manage Credits, browse/buy/sell marketplace items, and manage
payouts from the game UI.
**Acceptance criteria**
- An API client handles JWT auth, HTTP, and idempotency keys for all economy
calls.
- Dedicated screens: wallet + transaction history, marketplace browse/buy,
sell-listing, and creator payout.
- The client never computes balances, fees, or seller nets — it only displays
server responses.
## 6. Task breakdown
Right-sized to 1–3 files / one coherent piece of functionality each. Task IDs
are sequential for dependency chaining.
| # | Task | Story | Depends on |
|---|------|-------|-----------|
| 1 | Wallet & ledger models + Alembic migration (`wallets`, `transactions`) | US-1 | — |
| 2 | Wallet service (atomic credit/debit, idempotency, locking) + wallet/transactions API | US-1 | 1 |
| 3 | Premium purchase: `PaymentProvider` interface + purchase/confirm endpoints | US-2 | 1 |
| 4 | Catalog item & listing models + migration (`catalog_items`, `marketplace_listings`, `creator_balances`) | US-3 | 1 |
| 5 | Marketplace listing API (create/list/search/update/remove) | US-3 | 4 |
| 6 | Marketplace purchase service + API (atomic escrow, fee split, item grant) | US-4 | 2, 4, 5 |
| 7 | Creator payout request models + API (`payout_requests`) | US-5 | 2, 6 |
| 8 | Payout admin review + `PayoutProvider` + audit | US-5 | 7 |
| 9 | Marketplace moderation + content-flag/age-gate enforcement | cross-cutting | 5, 6 |
| 10 | Godot economy API client (auth + HTTP + idempotency) | US-6 | 2 |
| 11 | Godot wallet & transaction history UI | US-6 | 10 |
| 12 | Godot marketplace browse/buy UI | US-6 | 6, 10 |
| 13 | Godot selling & payout UI | US-6 | 5, 7, 10 |
## 7. Phasing
- **Phase 1 (currency + payments):** tasks 1–3. The foundation — ledger and
money movement must be rock-solid before anything spends Credits.
- **Phase 2 (marketplace):** tasks 4–6. Listing and atomic purchase with escrow.
- **Phase 3 (payouts):** tasks 7–8. Turn released escrow into creator payouts.
- **Phase 4 (moderation):** task 9. Lock down the marketplace against mature
content and abuse.
- **Phase 5 (client):** tasks 10–13, each landing after its backend dependency.
## 8. Data model sketch
```text
wallets
id bigint PK
user_id int UNIQUE FK -> users.id
balance bigint NOT NULL DEFAULT 0 CHECK (balance >= 0)
created_at timestamptz
updated_at timestamptz
transactions (append-only ledger)
id bigint PK
wallet_id bigint FK -> wallets.id
type enum(credit_purchase, marketplace_purchase,
marketplace_sale, escrow_release, payout,
refund, admin_adjustment)
amount bigint NOT NULL (signed)
balance_after bigint NOT NULL
reference_type text (e.g. "purchase", "sale", "payout_request")
reference_id text
idempotency_key text UNIQUE
metadata_json jsonb
created_at timestamptz
catalog_items
id bigint PK
owner_id int FK -> users.id
asset_id int FK -> assets.id (nullable; null for non-asset virtual items)
item_type enum(asset, avatar_item, experience_pass)
name text
description text
metadata_json jsonb
created_at timestamptz
marketplace_listings
id bigint PK
catalog_item_id bigint FK -> catalog_items.id
seller_id int FK -> users.id
price bigint NOT NULL CHECK (price > 0)
status enum(active, sold, delisted, removed_by_moderation)
created_at timestamptz
updated_at timestamptz
purchases
id bigint PK
listing_id bigint FK -> marketplace_listings.id
buyer_id int FK -> users.id
seller_id int FK -> users.id
price bigint NOT NULL
platform_fee bigint NOT NULL
seller_net bigint NOT NULL
status enum(pending_escrow, completed, refunded, reversed)
escrow_hold_until timestamptz
created_at timestamptz
creator_balances
id bigint PK
user_id int UNIQUE FK -> users.id
available bigint NOT NULL DEFAULT 0
pending bigint NOT NULL DEFAULT 0
lifetime_earned bigint NOT NULL DEFAULT 0
updated_at timestamptz
payout_requests
id bigint PK
creator_id int FK -> users.id
amount bigint NOT NULL CHECK (amount > 0)
status enum(pending, approved, paid, rejected)
external_ref text
provider_ref text
requested_by int FK -> users.id
reviewed_by int FK -> users.id (nullable)
reviewed_at timestamptz
created_at timestamptz
```
## 9. Security, anti-fraud & audit
- **Server-authoritative balances/fees:** clients never compute totals; amounts
are validated server-side against bounds and the current listing.
- **Row locking:** wallet and creator-balance mutations use `SELECT ... FOR
UPDATE` within one transaction.
- **Idempotency:** every money-moving endpoint requires an `Idempotency-Key`;
duplicate keys return the stored original outcome.
- **Rate limiting:** purchase, listing, and payout endpoints are token-bucket
rate-limited per user.
- **Escrow & reversibility:** funds are not payable until the hold window
elapses; chargebacks/refunds reverse before release.
- **Moderation:** listings carry the item's effective content flags; mature
items require age verification (per `content_moderation.md`); moderators can
`delist`/`remove` and trigger audited reversals.
- **Audit:** the ledger plus `payout_requests` review fields form an append-only
audit trail for every movement of value.
## 10. Out of scope for this feature
- Real Stripe/PayPal/KYC integrations (provider interfaces leave the hooks; the
defaults are deterministic no-ops).
- Subscriptions, recurring billing, or premium memberships.
- Gift cards, promotional codes, or user-to-user Credit transfers.
- Partial refunds, chargeback disputes, and seller tax/VAT handling.
- In-world vending machines / scripted economy triggers (future, built on this
ledger via the scripting sandbox).
+170
View File
@@ -0,0 +1,170 @@
# Social — Friends, Chat & Groups — Plan
> **Status:** Planned
> **Owner:** Platform team
> **Feature ID:** `social`
> **Godot version:** 4.6.2 stable
> **Backend:** FastAPI + SQLAlchemy (async) + PostgreSQL + Redis
## 1. Overview
This document breaks the **Social** feature into user stories and right-sized
implementation tasks. The feature delivers:
- **Friends & following** — a friend request/accept model plus a one-way follow
relationship, with block lists.
- **Presence & status** — online/offline, custom status message, last-seen.
- **Chat** — direct messages (DM) and in-experience (world/room) chat with
**first-class safety**: server-side text filtering, report-message, and rate
limiting.
- **Groups** — named groups with roles (owner/admin/member) and a simple group
page.
- **Public profiles** — avatar, bio, favorite experiences, and a friends list
that respects privacy settings.
## 2. Why chat safety is the differentiator
The platform is explicitly meant to pull adults away from a youth-oriented
competitor. Chat is the highest-risk surface: it is real-time, user-generated,
and where abuse happens fastest. Safety is therefore treated as a **blocking
requirement**, not an add-on:
- Every message passes a **server-side text filter** before it is stored or
delivered. The client filter is cosmetic only; the server is authoritative.
- Every message can be **reported**; reports flow into a moderation queue with
actor + reason + audit trail (mirroring `content_moderation.md`).
- Sending is **rate limited** per user (Redis token bucket) to stop flooding and
harassment-by-volume.
- Report and block are always reachable from a message context in the UI.
## 3. Existing constraints (do not re-build)
- **Auth:** JWT bearer tokens; `get_current_user` / `CurrentUser` dependency in
`app/dependencies.py`. All social endpoints reuse this.
- **DB:** async SQLAlchemy + Alembic; migration `0001_initial` is the head.
- **Redis:** configured (`REDIS_URL`) but unused so far; adopted here for
presence and rate limiting.
- **Models:** `User`, `World` (owner_id), `Asset` (owner_id, is_public). Avatars
and favorite-experience references point at existing rows.
- **Client:** Godot 4.6.2, minimal `main_menu.gd`; no social UI yet.
## 4. Key design decisions
| Concern | Decision |
|---|---|
| Message transport | REST for send/history; **WebSocket** gateway for real-time delivery + presence (matches `networking.md`'s "WebSocket as secondary transport" for chat) |
| Presence store | Redis (TTL heartbeat) + persisted `last_seen_at` on the user record |
| Text filtering | Pluggable `TextFilter` interface with a deterministic default (profanity list + normalization) so it runs without external services; third-party hook later |
| Rate limiting | Redis token-bucket dependency, reusable across all social endpoints |
| Reports | `message_reports` table + moderation endpoints (list/review/resolve) with audit fields |
| Friends vs. follow | Friends = mutual (request→accept). Follow = one-way (no approval). Block = hard deny that also hides content and prevents DMs |
| Privacy | Per-user settings JSON (`profile_visibility`, `friends_list_visibility`, `favorites_visibility`) enforced server-side on every read |
| Groups | `groups` + `group_members` (role: owner/admin/member) + group page aggregate endpoint |
| Real-time | One WebSocket endpoint multiplexing presence, DM, and world-chat messages via Redis pub/sub |
## 5. User stories
### US-1 Friends & following
As a player, I can send a friend request, accept/decline incoming requests,
unfriend, follow/unfollow other players, and block users so they can't interact
with me.
**Acceptance criteria**
- Friend requests are one-directional until accepted; accepted friendships are mutual.
- Follow is one-way and requires no approval.
- Block prevents DMs, hides the blocked user's profile/friends interactions, and
removes any existing friendship/follow between the two.
- Duplicate/self requests are rejected idempotently.
### US-2 Presence & status
As a player, I can see whether my friends are online, their custom status, and
when they were last seen.
**Acceptance criteria**
- Online/offline is derived from a server-side heartbeat (Redis TTL), not client claims.
- `last_seen_at` persists and is visible (subject to privacy settings).
- Custom status text is optional and length-limited.
### US-3 Public profiles
As a player, I can view a public profile page showing avatar, bio, favorite
experiences, and a friends list that respects the owner's privacy settings.
**Acceptance criteria**
- Avatar and bio are editable by the owner only.
- Favorite experiences reference existing `World` rows.
- A viewer sees only the sections the owner's privacy settings allow; blocked
users see a restricted view.
### US-4 Chat (DM + in-experience) with safety
As a player, I can send direct messages and chat inside a world/room. Messages
are filtered, rate limited, and reportable.
**Acceptance criteria**
- Server filters message text before storing/delivering; filtered content is either
rejected (blocking) or redacted (configurable per filter policy).
- Send rate is limited per user (token bucket); excess sends get `429`.
- Every message has a stable id, author, timestamp, and conversation/world scope.
- Any message can be reported with a reason; reports are stored and reviewable.
- Real-time delivery over WebSocket with REST fallback for history.
### US-5 Groups
As a player, I can create a group, join/leave groups, and (as owner/admin)
manage roles and view a simple group page.
**Acceptance criteria**
- Each group has exactly one owner and any number of admins/members.
- Role transitions are authorized (only owner can transfer ownership or promote/demote admins).
- Group page lists name, description, member list with roles, and member count.
### US-6 Godot client
As a player, I can access friends, presence, chat, profiles, and groups from the
game UI.
**Acceptance criteria**
- An API client handles JWT auth, HTTP, and WebSocket for all social calls.
- Dedicated screens/panels: friends+presence, DM & world chat (with report),
profile page, group page.
- Client-side filtering is cosmetic; server responses remain authoritative.
## 6. Task breakdown
Right-sized to 1–3 files / one coherent piece of functionality each. Task IDs
are sequential for dependency chaining.
| # | Task | Story | Depends on |
|---|------|-------|-----------|
| 1 | Friends/follow/block models + Alembic migration | US-1 | — |
| 2 | Friends & follow API router | US-1 | 1 |
| 3 | Presence & status model + endpoints (Redis heartbeat) | US-2 | — |
| 4 | Profile model + migration + own-profile endpoints | US-3 | 1 |
| 5 | Public profile view + favorites + privacy enforcement | US-3 | 2, 4 |
| 6 | Chat data models + migration (conversations, messages, reports) | US-4 | — |
| 7 | Text filtering service + profanity config | US-4 | — |
| 8 | Redis rate-limiting dependency | US-4 | — |
| 9 | DM chat send/list endpoints (filter + rate limit) | US-4 | 6, 7, 8 |
| 10 | Report-message endpoint + moderation queue | US-4 | 6, 9 |
| 11 | In-experience (world/room) chat endpoints | US-4 | 9 |
| 12 | Real-time WebSocket gateway (presence + DM + world chat) | US-4 | 3, 9, 11 |
| 13 | Group + membership models + migration | US-5 | — |
| 14 | Groups API (CRUD, join/leave, roles, group page) | US-5 | 13 |
| 15 | Godot API client (auth + HTTP + WebSocket) | US-6 | 2 |
| 16 | Godot friends & presence UI | US-6 | 3, 15 |
| 17 | Godot chat UI (DM + world chat + report) | US-6 | 12, 15 |
| 18 | Godot profile page UI | US-6 | 5, 15 |
| 19 | Godot groups UI + group page | US-6 | 14, 15 |
## 7. Phasing
- **Phase 1 (data + core APIs):** tasks 1–5 (friends, presence, profiles).
- **Phase 2 (chat safety — the differentiator):** tasks 6–12 (chat + safety +
realtime). This is the critical path and highest priority.
- **Phase 3 (groups):** tasks 13–14.
- **Phase 4 (client):** tasks 15–19, each landing after its backend dependency.
## 8. Out of scope for this feature
- Age verification / maturity gating (covered by `content_moderation.md`).
- Voice chat, media/image messages, message editing/deletion history.
- Third-party profanity/toxicity vendors (the `TextFilter` interface leaves a
hook, but the default is deterministic and dependency-free).
- Mobile push notifications.
+50
View File
@@ -0,0 +1,50 @@
# Testing at Package Checkpoints
A Package Checkpoint (PC1, PC2, ...) is a periodic review point — every few
real milestones, the client and server builds get packaged together and
independently reviewed before anyone calls anything actually done. See
`project_agp` in Claude's memory, or ask, for the full definition; this
doc is just the mechanical "how do I run the tests" part of that review.
## Running the suite
```
python scripts/run_checkpoint_tests.py
```
Brings up a throwaway Postgres, runs Alembic migrations against it, runs the
backend pytest suite, starts the backend server, and runs every Godot
headless smoke test (`scripts/smoke_test_*.gd`) against it. Tears the
Postgres container down afterward (`--keep-postgres` to skip that if you
want to poke at the DB after a failure).
If port 5432 is already taken on the machine, pass `--postgres-port` with
something else.
Exit code is 0 only if everything that actually ran passed — suites with
nothing to run yet (no Godot binary found, no smoke test scripts yet) are
reported SKIPPED, not treated as a failure.
## What this does and doesn't prove
Passing this script means: migrations apply cleanly to a genuinely empty
database, the backend's unit-level logic is correct, and whatever Godot
smoke tests exist actually round-trip against a live backend. It does
**not** mean full endpoint/integration coverage exists — see
`test_inventory.md` for the current, honest list of what's tested and what
isn't. A checkpoint review should still involve reading the actual diffs
for anything this script's current coverage doesn't reach, not just running
green and calling it done.
## Growing this over time
As real integration tests get added (endpoint tests hitting the FastAPI app,
more Godot smoke coverage beyond `health_check`), they should land under
`backend/tests/` / `scripts/smoke_test_*.gd` respectively — this script
picks up new files automatically, nothing to register by hand. Update
`test_inventory.md`'s tables when you do, so the inventory stays a source
of truth instead of quietly going stale.
Eventual goal (not implemented yet): gate merges to `main` on this script
passing, the same way a real CI pipeline would. For now it's a manual step
run at checkpoint review time.
+106
View File
@@ -0,0 +1,106 @@
# Test Inventory
What automated coverage actually exists, what it actually verifies, and what
doesn't have coverage yet. Keep this current as tests are added — it's the
thing a Package Checkpoint review (see `docs/testing/README.md`) reads first
to know what "run the tests" actually checked.
Run everything below in one shot with `scripts/run_checkpoint_tests.py`.
Last verified against `main`: 2026-08-20 — full PC1 checkpoint run,
migrations + pytest + all 7 Godot smoke tests + both platform exports, all
clean. Real bugs found and fixed via real playtesting this same day (not
caught by the checkpoint suite as it stood before today): the main menu
had no way to actually open the login/register form, the world list's
cards rendered at ~1px wide once real data existed, and navigating Play →
Back logged the user out despite a still-valid session. All three are now
covered by the smoke tests below so they can't silently regress.
## Backend — `backend/tests/` (pytest)
Run: `cd backend && python -m pytest -v` (needs `DATABASE_URL` pointed at a
reachable Postgres — see the runner script).
**Current total: 61 tests, all passing on a clean `main` checkout.**
| File | Tests | What it covers |
|---|---|---|
| `test_auth_age_gate.py` | 11 | `calculate_age()` date-math correctness (exact boundary, leap years, birthday-not-yet-this-year — kept as a tested utility for a future real-DOB pass even though `/register` no longer calls it) + `RegisterRequest.over_18` field tests (true accepted, false accepted, missing rejected). 2026-08-20: the register flow was simplified from collecting/validating a date of birth to a plain client-side "I am 18 or older" confirmation, so the old date-validation parametrized cases were replaced. Backs the 18+ registration gate (PR #23, revised PC1). |
| `test_gltf_metadata.py` | 21 | glTF/GLB binary + JSON parsing: magic-byte detection, chunk-padding handling, mesh/animation/texture-reference metadata extraction, malformed-input rejection. Backs the asset-import pipeline (PR #27). |
| `test_auth_endpoints.py` | 10 | **Integration tests** for `POST /api/auth/register` and `POST /api/auth/login` against the real FastAPI app + real Postgres. Covers: successful register (201), duplicate username (409), duplicate email (409), `over_18=false` rejection (400), missing `over_18` field (422), successful login with correct credentials (200), wrong password (401), nonexistent user (401), form-encoded login (OAuth2), and JWT validity of returned tokens. |
| `test_refresh_endpoint.py` | 8 | **Integration tests** for `POST /api/auth/refresh`. Covers: successful refresh (200, new access token, same refresh token returned), refresh produces different access tokens on repeated calls, invalid refresh token (401), empty body (422), missing field (422), refresh token from login also works, returned access token is valid JWT decodable with public key, `access_token_expires_minutes` in response. |
| `test_refresh_token_jti.py` | 6 | Unit tests for refresh token uniqueness (jti claim) and model column length. Covers: two calls with same sub produce different tokens (jti uniqueness), jti is valid UUID, rapid repeated calls all produce unique jtis, jti survives decode, `RefreshToken.token` column is `String(512)`, column type is `String`. |
| `test_auth_expired_token.py` | 5 | **Integration tests** for expired-token rejection across auth endpoints. Covers: expired access JWT rejected by `GET /api/auth/me` (401), garbage access token rejected by `/me` (401), missing bearer token returns 401, expired refresh token (DB `expires_at` in past) rejected by `/refresh` (401), expired refresh JWT (past `exp` claim) rejected before DB lookup (401). |
**What these are, honestly:** a mix of pure unit tests (`test_auth_age_gate.py`,
`test_gltf_metadata.py`, `test_refresh_token_jti.py`) and real integration tests
(`test_auth_endpoints.py`, `test_refresh_endpoint.py`, `test_auth_expired_token.py`)
that hit the FastAPI app against a real Postgres database via `TestClient`.
The integration tests exercise the full stack: Pydantic validation → DB →
JWT → response. The suite is also re-runnable without manual cleanup —
`backend/tests/conftest.py` has a session-scoped autouse fixture that
truncates all app tables once before the suite runs.
**Known coverage gaps** (real gaps, not placeholders — nothing enforces filling
these yet):
- No endpoint tests for `GET /api/worlds`, `GET /api/assets/*`, or any
world/asset CRUD routes — only auth endpoints have integration coverage
(`GET /api/worlds` is covered indirectly via the Godot smoke tests below).
- No tests for the moderation schema/migration beyond "the migration runs."
- No tests for the `GET /api/auth/me` endpoint's response body (only its
auth-header validation is covered via expired-token tests).
- No tests for session management (create/validate/revoke sessions).
## Godot client — `scripts/smoke_test_*.gd`
**7 smoke tests merged into `main`.** Each needs a live backend —
`AGP_API_BASE_URL` env var, falling back to the
`application/agp/api_base_url` project setting (default
`http://127.0.0.1:8000`) if unset.
| File | What it covers |
|---|---|
| `smoke_test_api.gd` | Exercises `ApiClient.health_check()` against a live backend. Verifies the real HTTP request path (`HTTPRequest` node lifecycle, JSON parse, result-dictionary contract). Exits 1 on failure. |
| `smoke_test_auth.gd` | Tests the full auth flow via `ApiClient` autoload: register unique user (201), duplicate username (409), `over_18=false` registration (400), correct login (200), wrong password (401), nonexistent user (401). |
| `smoke_test_e2e.gd` | End-to-end flow: health check → register → login → worlds list. Verifies the full request chain with a fresh user, including token storage and `get_worlds()` with auth header. |
| `smoke_test_world_list.gd` | Registers a user, **seeds a real world via `POST /api/worlds/`** (without this, `run_checkpoint_tests.py`'s always-fresh DB means the populated-card render path never actually runs in CI, only the empty-list path would), instantiates `world_list.tscn` as a real scene, waits for `_refresh_worlds()` to complete, confirms it reached the populated (not empty) state, and asserts **each rendered card's real computed width is >= 200px**. That last check is the one that matters: 2026-08-20, a real bug shipped where every card existed in the tree with a non-zero child count and no crash, but rendered at 1.0px wide (`ScrollContainer`'s `size_flags_horizontal`/`vertical` used the raw integer `4` = `Control.SIZE_SHRINK_CENTER` instead of `3` = `SIZE_EXPAND_FILL`) — invisible on screen, and a child-count-only check would never have caught it. Confirmed by deliberately reintroducing the bug: card width check correctly fails at 25px. |
| `smoke_test_main_menu_ui.gd` | Instantiates `main_menu.tscn`, confirms the auth panel starts hidden, and confirms a **real signal emission** on LoginButton/RegisterButton (not calling the handler method directly) actually opens it. This is the check that would have caught 2026-08-20's real bug: the auth panel and its logic were fully correct, but nothing in the scene was ever wired to open it, so Login/Register were completely unreachable from the UI. |
| `smoke_test_main_menu_full_flow.gd` | Full click-through: open the register form, fill it exactly as a user would, submit, confirm Play actually becomes enabled, the welcome message shows, and the auth buttons hide themselves. Calls the submit handler directly (awaited) rather than via `auth_button.pressed.emit()` — see the note at the top of the file: a synthetic `.emit()` on an `await`-using handler doesn't get driven forward reliably in `--script`/SceneTree headless mode the way real GUI input event dispatch does, confirmed by comparing direct-call vs `.emit()` behavior. Not a bug in the actual click-driven flow, just a headless-test-harness quirk to route around when writing tests. |
| `smoke_test_main_menu_back.gd` | Regression test for Play → Back logging the user out even with a valid token still held. Registers a user, loads `main_menu.tscn` (asserts it restores the authenticated state via a real `auth_state_ready` signal wait, not a frame-count guess), clicks Play → `world_list.tscn`, clicks Back → `main_menu.tscn`, and asserts the menu is still authenticated. Checks `WelcomeLabel.is_visible_in_tree()`, not just `.visible` — the label lives inside AuthPanel, so `.visible` alone is a false positive if the panel itself is hidden (a real bug briefly present: `_show_authenticated()` hid AuthPanel while claiming the welcome label was visible, which rendered nothing). |
Run (once, to build the class cache): `godot --headless --path . --import`,
then for each smoke test: `godot --headless --path . --script
res://scripts/smoke_test_<name>.gd` — **no `--quit`**, it kills these before
their awaited HTTP calls finish.
**Known gaps:**
- No smoke tests for chunk loading/streaming (`chunk.gd`, `chunk_scene.gd`).
- No smoke tests for dynamic/static object nodes.
- All smoke tests require a live backend; no offline/unit smoke tests exist.
- `smoke_test_world_list.gd` and `smoke_test_main_menu_*.gd` are the only
ones that exercise a real `.tscn` scene instantiation — others only test
the `ApiClient` autoload directly.
- `get_me()` (`GET /auth/me`) is implemented in `scripts/api_client.gd` but
has no smoke coverage yet.
- The Login path (as opposed to Register) through `main_menu.gd` isn't
separately smoke-tested — it shares the same `_on_auth_button_pressed()`/
`_on_auth_complete()` code path already exercised for Register, but that's
inference, not a direct check.
## Client export
`scripts/build.sh` produces real Linux (`agp-game.x86_64`) and Windows
(`agp-game.exe`) binaries via `--export-release`. Verified 2026-08-20: both
exports complete clean from a fresh checkout and produce valid binaries
(confirmed via `file` — real ELF64/PE32+, not empty or truncated). Neither
export is exercised by an automated GUI test (headless Godot can't drive a
window) — actually launching and clicking through the exported binary is a
manual step, see `docs/checkpoints/PC1.md` for playtest instructions.
## What "full coverage" should eventually mean here
Not defined precisely yet — the honest target as of 2026-08-20 is
"integration tests for every endpoint + smoke tests for every major scene."
As real integration/endpoint tests get added, update the tables above
rather than letting this doc drift stale; a wrong inventory is worse than
no inventory.