Compare commits
14 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 4d690a25a5 | |||
| a632f2d8a6 | |||
| 0b973b73ab | |||
| f0d10fc4ae | |||
| 1295b6a389 | |||
| 2ddcd58902 | |||
| 1ba334e983 | |||
| 80bc27426e | |||
| a82ab52919 | |||
| 092ba57977 | |||
| af847ddf78 | |||
| 2b068cce15 | |||
| 41fcbe746e | |||
| 24d2029955 |
@@ -0,0 +1,426 @@
|
|||||||
|
# Content Moderation & Age Verification Architecture
|
||||||
|
|
||||||
|
> **Status:** Proposed
|
||||||
|
> **Owner:** Platform team
|
||||||
|
> **Related:** Server authority + client prediction, avatar component system, backend auth (JWT)
|
||||||
|
|
||||||
|
## 1. Overview
|
||||||
|
|
||||||
|
This document specifies the **content moderation** and **age verification**
|
||||||
|
architecture for **agp-game**, a Godot-based multiplayer game built by
|
||||||
|
autonomous agents. It defines:
|
||||||
|
|
||||||
|
- How **content flags** propagate through the system across the three primary
|
||||||
|
content types: **assets**, **rooms**, and **scripts**.
|
||||||
|
- How flags are **evaluated, aggregated, and enforced** at load time, at
|
||||||
|
runtime, and at the network boundary.
|
||||||
|
- The **age verification flow**, combining a lightweight **self-declaration**
|
||||||
|
with an optional **ID check hook** for higher-assurance verification.
|
||||||
|
|
||||||
|
The goal is a **single, consistent moderation pipeline** that applies the same
|
||||||
|
rules to user-generated content (UGC) regardless of whether it is a 3D asset, a
|
||||||
|
room definition, or a script, and that gates access to mature content behind a
|
||||||
|
verifiable age gate.
|
||||||
|
|
||||||
|
### 1.1 Design Goals
|
||||||
|
|
||||||
|
- **Uniformity:** One flag vocabulary and one propagation mechanism across
|
||||||
|
assets, rooms, and scripts.
|
||||||
|
- **Composability:** Flags on a room aggregate the flags of the assets and
|
||||||
|
scripts it contains; flags on a world aggregate the flags of its rooms.
|
||||||
|
- **Enforcement at the boundary:** The server is authoritative for what content
|
||||||
|
a client may load or receive, mirroring the server-authoritative networking
|
||||||
|
model.
|
||||||
|
- **Privacy-preserving age checks:** Self-declaration requires no PII; the ID
|
||||||
|
check hook is optional, pluggable, and never stores raw identity documents.
|
||||||
|
- **Auditability:** Every flag decision is logged with a reason, an actor, and
|
||||||
|
a timestamp so moderation actions are reviewable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Content Flag Vocabulary
|
||||||
|
|
||||||
|
All content carries a set of **content flags**. A flag is a typed, immutable
|
||||||
|
label with a **severity** and a **scope**. Flags are the atomic unit of
|
||||||
|
moderation; everything else (aggregation, enforcement, gating) operates on
|
||||||
|
flags.
|
||||||
|
|
||||||
|
### 2.1 Flag Definition
|
||||||
|
|
||||||
|
A flag is a structured record:
|
||||||
|
|
||||||
|
| Field | Type | Description |
|
||||||
|
|------------|-----------|--------------------------------------------------------------------------|
|
||||||
|
| `id` | `string` | Stable machine-readable identifier, e.g. `violence.gore`. |
|
||||||
|
| `label` | `string` | Human-readable name, e.g. "Graphic Gore". |
|
||||||
|
| `category` | `enum` | One of `violence`, `sexual`, `language`, `substance`, `gambling`, `horror`, `hate`, `custom`. |
|
||||||
|
| `severity` | `enum` | One of `none`, `mild`, `moderate`, `severe`. |
|
||||||
|
| `scope` | `enum` | One of `visual`, `audio`, `text`, `behavior`, `interaction`. |
|
||||||
|
| `mature` | `bool` | Whether this flag triggers the age gate (see §5). |
|
||||||
|
| `reason` | `string` | Free-text justification supplied by the flagger. |
|
||||||
|
| `source` | `enum` | One of `author`, `moderator`, `automated`, `community`. |
|
||||||
|
|
||||||
|
### 2.2 Canonical Flag Set
|
||||||
|
|
||||||
|
The platform defines a **canonical registry** of flags so that different
|
||||||
|
content types and different flaggers use the same vocabulary. Examples:
|
||||||
|
|
||||||
|
| id | category | severity | scope | mature |
|
||||||
|
|-----------------------------|------------|-----------|--------------|--------|
|
||||||
|
| `violence.none` | violence | none | visual | false |
|
||||||
|
| `violence.mild` | violence | mild | visual | false |
|
||||||
|
| `violence.gore` | violence | severe | visual | true |
|
||||||
|
| `sexual.suggestive` | sexual | mild | visual | true |
|
||||||
|
| `sexual.explicit` | sexual | severe | visual | true |
|
||||||
|
| `language.mild` | language | mild | text | false |
|
||||||
|
| `language.profanity` | language | moderate | text | false |
|
||||||
|
| `substance.references` | substance | mild | text | false |
|
||||||
|
| `gambling.simulated` | gambling | moderate | behavior | true |
|
||||||
|
| `horror.jumpscare` | horror | mild | audio/visual | false |
|
||||||
|
| `hate.hateful` | hate | severe | text | true |
|
||||||
|
|
||||||
|
The registry is **data-driven** (a JSON/YAML manifest shipped with the server)
|
||||||
|
so new flags can be added without code changes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Flag Propagation Model
|
||||||
|
|
||||||
|
Flags propagate through the system along the **content dependency graph**. The
|
||||||
|
three content types form a hierarchy:
|
||||||
|
|
||||||
|
```
|
||||||
|
World
|
||||||
|
└── Room
|
||||||
|
├── Asset (mesh, texture, audio, animation, material)
|
||||||
|
└── Script (behavior, dialogue, event logic)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.1 Propagation Rules
|
||||||
|
|
||||||
|
1. **Explicit flags** are set directly on a content item by its author or a
|
||||||
|
moderator.
|
||||||
|
2. **Inherited flags** flow upward: a room's effective flag set is the **union**
|
||||||
|
of its own explicit flags and the flags of every asset and script it
|
||||||
|
references.
|
||||||
|
3. **Aggregation** uses the **maximum severity** per category across all
|
||||||
|
contributing items. A room containing one `violence.gore` asset and one
|
||||||
|
`violence.none` asset is `violence.severe`.
|
||||||
|
4. **Maturity is monotonic:** if any contributing item is `mature`, the
|
||||||
|
containing room and world are `mature`.
|
||||||
|
5. **Cycles are impossible** by construction: content references form a DAG
|
||||||
|
(an asset cannot contain a room; a room cannot contain itself).
|
||||||
|
|
||||||
|
### 3.2 Propagation Through the Pipeline
|
||||||
|
|
||||||
|
Flags are attached at **authoring time** and carried through every stage:
|
||||||
|
|
||||||
|
```
|
||||||
|
Authoring ──► Upload ──► Validation ──► Storage ──► Distribution ──► Runtime
|
||||||
|
│ │ │ │ │ │
|
||||||
|
│ flag │ flag │ automated │ flag │ flag │ flag
|
||||||
|
│ declared │ preserved │ scan adds │ indexed │ re-checked │ enforced
|
||||||
|
│ │ │ flags │ │ │
|
||||||
|
```
|
||||||
|
|
||||||
|
| Stage | Responsibility |
|
||||||
|
|--------------|-------------------------------------------------------------------------|
|
||||||
|
| **Authoring**| Author declares flags in the editor/upload UI. |
|
||||||
|
| **Upload** | Flags are validated against the canonical registry; unknown flags rejected. |
|
||||||
|
| **Validation**| Automated scanner (hash-based + heuristic) adds flags for detected content. |
|
||||||
|
| **Storage** | Flags are stored alongside the content record and indexed for queries. |
|
||||||
|
| **Distribution**| Flags are re-checked against the current registry before serving. |
|
||||||
|
| **Runtime** | The client enforces flags before loading content (see §4). |
|
||||||
|
|
||||||
|
### 3.3 Data Model
|
||||||
|
|
||||||
|
Flags are stored as a **normalized join table** so they can be queried and
|
||||||
|
aggregated efficiently:
|
||||||
|
|
||||||
|
```
|
||||||
|
content_item (id, type, owner_id, ...)
|
||||||
|
content_flag (content_id, flag_id, source, reason, created_at, actor_id)
|
||||||
|
flag_registry (id, category, severity, scope, mature, ...)
|
||||||
|
```
|
||||||
|
|
||||||
|
Aggregation is computed by a SQL view / query that joins a content item's
|
||||||
|
transitive dependencies. Because the graph is a DAG, this is a bounded
|
||||||
|
recursive query (PostgreSQL `WITH RECURSIVE`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Enforcement
|
||||||
|
|
||||||
|
Enforcement happens at three layers, in order:
|
||||||
|
|
||||||
|
### 4.1 Load-Time Enforcement (Server)
|
||||||
|
|
||||||
|
Before a client is allowed to **request** content, the server evaluates the
|
||||||
|
content's effective flag set against the requesting account's **content
|
||||||
|
preferences** and **age verification status**:
|
||||||
|
|
||||||
|
- If the content is `mature` and the account is **not age-verified**, the
|
||||||
|
request is **rejected** with a `403 MATURE_CONTENT_REQUIRES_VERIFICATION`.
|
||||||
|
- If the content's flags exceed the account's **content filter** (a per-account
|
||||||
|
severity threshold per category), the request is rejected or the offending
|
||||||
|
content is **stripped** (e.g., audio muted, mesh hidden, script not loaded).
|
||||||
|
|
||||||
|
### 4.2 Runtime Enforcement (Client)
|
||||||
|
|
||||||
|
The client receives a **manifest** of allowed content from the server. It never
|
||||||
|
loads content that is not in the manifest. This prevents a malicious or buggy
|
||||||
|
client from bypassing the server's filter by loading content directly.
|
||||||
|
|
||||||
|
- Assets not in the manifest are **not instantiated**.
|
||||||
|
- Scripts not in the manifest are **not executed**.
|
||||||
|
- Rooms not in the manifest are **not entered**.
|
||||||
|
|
||||||
|
### 4.3 Network Boundary Enforcement
|
||||||
|
|
||||||
|
The server **filters replication** based on flags:
|
||||||
|
|
||||||
|
- Mature or filtered content is **not replicated** to clients that cannot
|
||||||
|
receive it.
|
||||||
|
- A room's flag set is broadcast in the room metadata so clients can show a
|
||||||
|
warning before entry.
|
||||||
|
|
||||||
|
### 4.4 Enforcement Decision Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
Client requests content C for account A
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Compute effective flags F(C) [union + max-severity aggregation]
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Is A age-verified? ──no──► Is F(C).mature? ──yes──► REJECT (403)
|
||||||
|
│ yes │ no
|
||||||
|
▼ ▼
|
||||||
|
Does F(C) exceed A's filter? ──► yes ──► REJECT or STRIP
|
||||||
|
│ no
|
||||||
|
▼
|
||||||
|
Serve content + manifest
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Age Verification Flow
|
||||||
|
|
||||||
|
Age verification is a **two-tier** system:
|
||||||
|
|
||||||
|
- **Tier 1 — Self-declaration:** The user asserts their date of birth. This is
|
||||||
|
fast, requires no PII beyond a birthdate, and is sufficient for most content.
|
||||||
|
- **Tier 2 — ID check (optional hook):** A pluggable identity-verification
|
||||||
|
provider confirms the user is an adult. This raises the account's assurance
|
||||||
|
level and unlocks the highest maturity tier.
|
||||||
|
|
||||||
|
### 5.1 Verification States
|
||||||
|
|
||||||
|
An account has an **age verification status**:
|
||||||
|
|
||||||
|
| Status | Meaning |
|
||||||
|
|-------------------|----------------------------------------------------------------------|
|
||||||
|
| `unverified` | No declaration on file. |
|
||||||
|
| `self_declared` | User declared a birthdate; **not** independently confirmed. |
|
||||||
|
| `id_verified` | User passed the ID check hook; independently confirmed adult. |
|
||||||
|
| `rejected` | ID check failed or declaration was inconsistent. |
|
||||||
|
|
||||||
|
### 5.2 Tier 1 — Self-Declaration Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
User submits { date_of_birth }
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Server validates format & plausibility (age 13–120, not in future)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Compute age from DOB
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
age >= 18 ──► status = self_declared (adult)
|
||||||
|
age >= 13 ──► status = self_declared (minor, restricted)
|
||||||
|
age < 13 ──► status = rejected (below minimum platform age)
|
||||||
|
```
|
||||||
|
|
||||||
|
- The DOB is stored **encrypted at rest** and is **never exposed** to other
|
||||||
|
players.
|
||||||
|
- A user may **upgrade** from `self_declared` to `id_verified` at any time via
|
||||||
|
Tier 2.
|
||||||
|
|
||||||
|
### 5.3 Tier 2 — ID Check Hook
|
||||||
|
|
||||||
|
The ID check is an **optional, pluggable hook**. The platform defines an
|
||||||
|
interface; concrete providers (e.g., a third-party KYC service) implement it.
|
||||||
|
|
||||||
|
```
|
||||||
|
User requests ID verification
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Server creates a verification session (idempotent, expiring)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Server calls provider hook: verify(identity_token) -> { adult: bool, ref: string }
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
adult == true ──► status = id_verified, store provider ref (not raw doc)
|
||||||
|
adult == false ─► status = rejected
|
||||||
|
provider error ──► status unchanged, retry allowed
|
||||||
|
```
|
||||||
|
|
||||||
|
**Privacy guarantees:**
|
||||||
|
|
||||||
|
- The platform **never stores** the raw identity document or image.
|
||||||
|
- Only the provider's **opaque reference** and the boolean `adult` result are
|
||||||
|
persisted.
|
||||||
|
- The hook is **asynchronous and idempotent**; a session can be retried without
|
||||||
|
double-charging or double-verifying.
|
||||||
|
|
||||||
|
### 5.4 Hook Interface
|
||||||
|
|
||||||
|
The ID check hook is defined as a backend interface (FastAPI dependency /
|
||||||
|
abstract service):
|
||||||
|
|
||||||
|
```python
|
||||||
|
class IdentityVerificationProvider(Protocol):
|
||||||
|
def create_session(self, user_id: str) -> VerificationSession: ...
|
||||||
|
def verify(self, session_id: str, identity_token: str) -> VerificationResult: ...
|
||||||
|
|
||||||
|
class VerificationResult:
|
||||||
|
adult: bool
|
||||||
|
provider_ref: str | None
|
||||||
|
error: str | None
|
||||||
|
```
|
||||||
|
|
||||||
|
A default **no-op provider** returns `adult=False` with a clear error, so the
|
||||||
|
system runs without any third-party integration until one is configured.
|
||||||
|
|
||||||
|
### 5.5 Age Gate Enforcement
|
||||||
|
|
||||||
|
The age gate is enforced at **load time** (see §4.1):
|
||||||
|
|
||||||
|
| Verification status | Can access `mature` content? |
|
||||||
|
|---------------------|------------------------------|
|
||||||
|
| `unverified` | No |
|
||||||
|
| `self_declared` | Yes (adult) / No (minor) |
|
||||||
|
| `id_verified` | Yes |
|
||||||
|
| `rejected` | No |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Moderation Workflow
|
||||||
|
|
||||||
|
### 6.1 Flagging Sources
|
||||||
|
|
||||||
|
Content can be flagged by:
|
||||||
|
|
||||||
|
- **Author** (self-declared at upload).
|
||||||
|
- **Automated scanner** (hash-based detection, heuristic analysis).
|
||||||
|
- **Community** (players report content; reports become flags after review).
|
||||||
|
- **Moderator** (manual review and override).
|
||||||
|
|
||||||
|
### 6.2 Review Queue
|
||||||
|
|
||||||
|
Community and automated flags enter a **review queue**. A moderator either:
|
||||||
|
|
||||||
|
- **Approves** the flag (it becomes authoritative), or
|
||||||
|
- **Rejects** the flag (it is discarded), or
|
||||||
|
- **Escalates** to a senior moderator for severe/hate flags.
|
||||||
|
|
||||||
|
### 6.3 Actions
|
||||||
|
|
||||||
|
Once a flag is authoritative, a moderator can take actions:
|
||||||
|
|
||||||
|
| Action | Effect |
|
||||||
|
|-----------------|---------------------------------------------------------------|
|
||||||
|
| `warn` | Notify the author; content remains. |
|
||||||
|
| `restrict` | Content hidden from users below a filter threshold. |
|
||||||
|
| `age_gate` | Content marked `mature`; requires age verification. |
|
||||||
|
| `remove` | Content taken down; not served. |
|
||||||
|
| `ban_author` | Author's upload privileges suspended. |
|
||||||
|
|
||||||
|
### 6.4 Audit Log
|
||||||
|
|
||||||
|
Every flag and action is written to an **audit log**:
|
||||||
|
|
||||||
|
```
|
||||||
|
audit_event (id, content_id, actor_id, action, flag_id, reason, created_at)
|
||||||
|
```
|
||||||
|
|
||||||
|
This log is append-only and used for transparency and dispute resolution.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Client-Side Integration (Godot)
|
||||||
|
|
||||||
|
### 7.1 Content Manifest
|
||||||
|
|
||||||
|
The server sends each client a **content manifest** — the set of content the
|
||||||
|
client is permitted to load, derived from the account's verification status and
|
||||||
|
filter. The Godot client:
|
||||||
|
|
||||||
|
- Loads only manifest-listed assets.
|
||||||
|
- Executes only manifest-listed scripts.
|
||||||
|
- Shows a **maturity warning** dialog before entering a room whose effective
|
||||||
|
flags include `mature` content.
|
||||||
|
|
||||||
|
### 7.2 Flag Metadata on Content
|
||||||
|
|
||||||
|
Each content resource carries a `content_flags` metadata field (a Godot
|
||||||
|
`Dictionary` or custom `Resource`). The client uses this for local UI decisions
|
||||||
|
(e.g., showing a warning icon) but **never** as the authority for whether to
|
||||||
|
load — the server manifest is authoritative.
|
||||||
|
|
||||||
|
### 7.3 Age Verification UI
|
||||||
|
|
||||||
|
The client exposes an **age verification screen**:
|
||||||
|
|
||||||
|
- Tier 1: a date-of-birth picker.
|
||||||
|
- Tier 2: a button that opens the ID check provider flow (deep link / webview).
|
||||||
|
|
||||||
|
The client calls the backend endpoints (§8) and reflects the resulting status.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Backend API Surface
|
||||||
|
|
||||||
|
| Method | Path | Description |
|
||||||
|
|--------|-----------------------------------|------------------------------------------------|
|
||||||
|
| `POST` | `/v1/age/declare` | Submit self-declared DOB (Tier 1). |
|
||||||
|
| `POST` | `/v1/age/verify/session` | Create an ID-check session (Tier 2). |
|
||||||
|
| `POST` | `/v1/age/verify/confirm` | Confirm an ID-check session with a token. |
|
||||||
|
| `GET` | `/v1/age/status` | Return the account's verification status. |
|
||||||
|
| `GET` | `/v1/content/{id}/flags` | Return a content item's effective flags. |
|
||||||
|
| `POST` | `/v1/content/{id}/flags` | Add a flag (author/community). |
|
||||||
|
| `POST` | `/v1/moderation/review/{flag_id}` | Moderator approve/reject/escalate. |
|
||||||
|
| `POST` | `/v1/moderation/action` | Apply a moderation action. |
|
||||||
|
| `GET` | `/v1/manifest` | Return the client's permitted content manifest.|
|
||||||
|
|
||||||
|
All endpoints require a valid **JWT** (matching the existing auth stack). The
|
||||||
|
age endpoints additionally require the account to be in a non-`rejected` state.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Security & Privacy Considerations
|
||||||
|
|
||||||
|
- **DOB encryption:** Self-declared birthdates are encrypted at rest; only the
|
||||||
|
derived age and verification status are used in authorization decisions.
|
||||||
|
- **No raw identity storage:** The ID check hook persists only a provider
|
||||||
|
reference and a boolean result.
|
||||||
|
- **Server authority:** The client manifest prevents bypass; the client never
|
||||||
|
trusts its own flag metadata for access decisions.
|
||||||
|
- **Rate limiting:** Age-declaration and ID-check endpoints are rate-limited to
|
||||||
|
prevent abuse and enumeration.
|
||||||
|
- **Auditability:** All flag and verification events are logged append-only.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Open Questions / Future Work
|
||||||
|
|
||||||
|
- **Regional age thresholds:** Different jurisdictions may require different
|
||||||
|
minimum ages; the threshold should be configurable per region.
|
||||||
|
- **Parental controls:** A future feature may let a parent set a content filter
|
||||||
|
on a minor's account.
|
||||||
|
- **Automated scanner accuracy:** The heuristic scanner may produce false
|
||||||
|
positives; a confidence score should be attached to automated flags.
|
||||||
|
- **Offline / local content:** How flags apply to locally-authored content that
|
||||||
|
is never uploaded remains to be defined.
|
||||||
@@ -0,0 +1,369 @@
|
|||||||
|
# Editor Architecture: Godot Editor Plugin vs. In-Game Editor
|
||||||
|
|
||||||
|
> **Status:** Proposed
|
||||||
|
> **Owner:** Platform team
|
||||||
|
> **Related:** World chunk data format & persistence, server authority + client
|
||||||
|
> prediction, content moderation & age verification, avatar component system
|
||||||
|
> **Godot version:** 4.6.2 stable
|
||||||
|
|
||||||
|
## 1. Overview
|
||||||
|
|
||||||
|
This document evaluates the two primary ways to build the **world/level
|
||||||
|
authoring tool** for **agp-game**:
|
||||||
|
|
||||||
|
- **(a) Godot editor plugin** — extend the Godot editor itself with custom
|
||||||
|
panels, dock widgets, and tools that operate on the project's scenes and
|
||||||
|
resources.
|
||||||
|
- **(b) In-game editor** — a fully custom editor built as game UI (Godot
|
||||||
|
`Control` nodes) that runs inside the running game, editing the game's own
|
||||||
|
world data.
|
||||||
|
|
||||||
|
The goal is to pick the approach that best fits the platform's constraints:
|
||||||
|
a **server-authoritative** multiplayer game where the **world is stored as
|
||||||
|
versioned chunks** (see `world.md`), content is **moderation-flagged** (see
|
||||||
|
`content_moderation.md`), and the client is a **shipped binary** that players
|
||||||
|
run (not a dev tool).
|
||||||
|
|
||||||
|
This document is a **decision record**: it lays out the trade-offs, gives a
|
||||||
|
clear recommendation, and defines the resulting architecture boundaries.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Context & Constraints
|
||||||
|
|
||||||
|
Before comparing the two approaches, we must fix the constraints that drive the
|
||||||
|
decision. These come from the existing architecture docs.
|
||||||
|
|
||||||
|
### 2.1 The World Is Data, Not Scenes
|
||||||
|
|
||||||
|
`world.md` defines the world as **chunk data** (JSON for authoring, binary for
|
||||||
|
runtime/network), persisted in SQLite on the server. A chunk is the atomic unit
|
||||||
|
of load, save, and stream. The **server is the only writer** to the store;
|
||||||
|
clients submit **deltas** against a `base_revision` and the server validates and
|
||||||
|
applies them.
|
||||||
|
|
||||||
|
**Implication:** The editor's output is **chunk payloads**, not `.tscn` scene
|
||||||
|
files. Whatever editor we build must serialize to the chunk contract, not to
|
||||||
|
Godot scene resources.
|
||||||
|
|
||||||
|
### 2.2 Server-Authoritative & Moderation-Aware
|
||||||
|
|
||||||
|
- The server validates every edit (schema, bounds, moderation) before
|
||||||
|
persisting (`world.md` §5.4).
|
||||||
|
- Content carries **content flags** aggregated from referenced assets/scripts
|
||||||
|
(`content_moderation.md` §3). Mature content is gated behind age verification.
|
||||||
|
- The client receives a **content manifest** and never loads content not in it.
|
||||||
|
|
||||||
|
**Implication:** An editor that runs on a **player's machine** cannot be the
|
||||||
|
authority on what gets published. Editing must go through the server's
|
||||||
|
validation pipeline, and the editor must surface moderation flags.
|
||||||
|
|
||||||
|
### 2.3 The Client Is a Shipped Binary
|
||||||
|
|
||||||
|
The game is a multiplayer product. Players run a **release build**. The Godot
|
||||||
|
editor is a **development tool** that players do not have.
|
||||||
|
|
||||||
|
**Implication:** If authoring is a first-class *player* feature (user-generated
|
||||||
|
content), the editor must ship inside the game binary. If authoring is a
|
||||||
|
*developer/platform* feature, it can live in the editor.
|
||||||
|
|
||||||
|
### 2.4 Two Distinct Audiences
|
||||||
|
|
||||||
|
The evaluation must separate two audiences, because they have opposite needs:
|
||||||
|
|
||||||
|
| Audience | Who | Where they run | What they build |
|
||||||
|
|----------|-----|----------------|-----------------|
|
||||||
|
| **Platform/dev** | The team building the game | Godot editor (dev machines) | Base worlds, rooms, templates, test levels |
|
||||||
|
| **Players (UGC)** | End users | The shipped game | Their own rooms/worlds, published to the server |
|
||||||
|
|
||||||
|
The two approaches map cleanly onto these audiences, and the right answer is
|
||||||
|
likely **both** — but with a clear primary recommendation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Approach (a): Godot Editor Plugin
|
||||||
|
|
||||||
|
### 3.1 What It Is
|
||||||
|
|
||||||
|
A Godot editor plugin is a set of scripts (and optionally scenes) that extend
|
||||||
|
the editor via the `EditorPlugin` API. It can add:
|
||||||
|
|
||||||
|
- **Dock panels** (`add_control_to_dock`) — e.g., a chunk/tile palette panel.
|
||||||
|
- **Toolbar buttons** (`add_control_to_container`).
|
||||||
|
- **Inspector plugins** (`EditorInspectorPlugin`) — custom property editors.
|
||||||
|
- **Import plugins** (`EditorImportPlugin`) — custom asset importers.
|
||||||
|
- **Scene/Node plugins** (`EditorNode3DGizmoPlugin`) — custom gizmos.
|
||||||
|
- **Menu entries** (`add_tool_menu_item`).
|
||||||
|
|
||||||
|
Plugins are enabled per-project via `project.godot` (`[editor_plugins]`).
|
||||||
|
|
||||||
|
### 3.2 Pros
|
||||||
|
|
||||||
|
- **Full editor power for free:** You get the scene tree, inspector, viewport
|
||||||
|
navigation, undo/redo, gizmos, and asset pipeline — all battle-tested.
|
||||||
|
- **Native Godot editing:** Editing `.tscn` scenes, materials, and nodes is
|
||||||
|
exactly what the editor is built for. Great for authoring **base content**
|
||||||
|
(rooms, templates, avatar base bodies).
|
||||||
|
- **Fast iteration for devs:** The platform team can build and test levels
|
||||||
|
directly in the tool they already use.
|
||||||
|
- **Undo/redo, selection, snapping** come from the editor's `EditorUndoRedoManager`.
|
||||||
|
- **No runtime footprint:** Editor-only code is stripped from export builds
|
||||||
|
(`@tool` scripts and editor plugins are not shipped to players).
|
||||||
|
|
||||||
|
### 3.3 Cons
|
||||||
|
|
||||||
|
- **Not available to players.** Players don't run the Godot editor. Any
|
||||||
|
player-facing UGC authoring cannot use this approach.
|
||||||
|
- **Tied to the editor's UX.** You inherit the editor's look and interaction
|
||||||
|
model; deep customization is limited by what the editor API exposes.
|
||||||
|
- **Editor API churn.** `EditorPlugin` APIs change between Godot versions;
|
||||||
|
plugins are more brittle across upgrades than game code.
|
||||||
|
- **Scene-centric, not data-centric.** The editor is optimized for scenes and
|
||||||
|
resources. Our world is **chunk data** (`world.md`). We'd have to build a
|
||||||
|
translation layer (scene ↔ chunk) and keep it in sync — a real source of
|
||||||
|
drift.
|
||||||
|
- **Single-user, local.** The editor edits local files. It does not natively
|
||||||
|
talk to our server-authoritative chunk store or the moderation pipeline.
|
||||||
|
- **Headless/CI friction.** Validating plugin behavior in CI requires running
|
||||||
|
the editor headless, which is heavier than testing game code.
|
||||||
|
|
||||||
|
### 3.4 Fit for Our Constraints
|
||||||
|
|
||||||
|
- **Platform/dev authoring:** Excellent. This is the natural home for building
|
||||||
|
base worlds, rooms, and templates.
|
||||||
|
- **Player UGC:** Not applicable. Players cannot run the editor.
|
||||||
|
- **Server-authoritative chunk store:** Poor fit natively. The editor edits
|
||||||
|
local scenes; we'd bolt on a chunk export/import and a server sync layer.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Approach (b): In-Game Editor
|
||||||
|
|
||||||
|
### 4.1 What It Is
|
||||||
|
|
||||||
|
A fully custom editor built as **game UI** — Godot `Control` nodes (panels,
|
||||||
|
buttons, toolbars) that run inside the running game. It edits the game's own
|
||||||
|
world data model (chunks) and submits edits through the normal server pipeline.
|
||||||
|
|
||||||
|
### 4.2 Pros
|
||||||
|
|
||||||
|
- **Ships to players.** The editor is part of the game binary, so **player UGC
|
||||||
|
authoring is possible** — the core requirement for a user-generated-content
|
||||||
|
platform.
|
||||||
|
- **Data-native.** The editor operates directly on the **chunk model**
|
||||||
|
(`world.md`), so there is no scene↔chunk translation layer and no drift.
|
||||||
|
- **Server-authoritative by construction.** Edits go through the same
|
||||||
|
`request_chunk_edit` RPCs as gameplay, so validation, revisioning, and
|
||||||
|
moderation apply uniformly. The editor cannot bypass the server.
|
||||||
|
- **Moderation-aware.** Because edits flow through the server, content flags are
|
||||||
|
computed and enforced at the boundary (`content_moderation.md` §4). The editor
|
||||||
|
can surface flags and gate mature content.
|
||||||
|
- **One codebase, one UX.** The editor uses the same UI toolkit and art style as
|
||||||
|
the game; no separate editor theme to maintain.
|
||||||
|
- **Testable in CI.** The editor is ordinary game code; it can be exercised
|
||||||
|
headlessly and unit-tested like any other system.
|
||||||
|
|
||||||
|
### 4.3 Cons
|
||||||
|
|
||||||
|
- **We build everything.** No free scene tree, inspector, gizmos, or undo/redo.
|
||||||
|
We must implement selection, snapping, undo/redo, viewport navigation, and
|
||||||
|
property editing ourselves.
|
||||||
|
- **Runtime footprint.** Editor UI ships in the binary (though it can be gated
|
||||||
|
behind a feature flag / separate scene that is only instantiated on demand).
|
||||||
|
- **More upfront work.** A usable in-game editor is a large feature; it needs
|
||||||
|
its own architecture (tool modes, gizmos, palette, inspector).
|
||||||
|
- **Not ideal for deep asset authoring.** Modeling a mesh or authoring a
|
||||||
|
material is better done in the Godot editor / DCC tools. The in-game editor
|
||||||
|
should focus on **world composition** (placing tiles and objects), not asset
|
||||||
|
creation.
|
||||||
|
|
||||||
|
### 4.4 Fit for Our Constraints
|
||||||
|
|
||||||
|
- **Platform/dev authoring:** Workable but not ideal for deep asset work.
|
||||||
|
- **Player UGC:** The **only** viable approach. Players can only author through
|
||||||
|
the shipped game.
|
||||||
|
- **Server-authoritative chunk store:** Excellent fit. The editor is a client
|
||||||
|
that submits validated deltas.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Comparison Matrix
|
||||||
|
|
||||||
|
| Criterion | (a) Editor plugin | (b) In-game editor |
|
||||||
|
|-----------|-------------------|--------------------|
|
||||||
|
| Ships to players | ❌ No | ✅ Yes |
|
||||||
|
| Player UGC authoring | ❌ Impossible | ✅ Core capability |
|
||||||
|
| World data model fit | ⚠️ Scene-centric; needs translation layer | ✅ Chunk-native |
|
||||||
|
| Server-authoritative edits | ⚠️ Bolted on | ✅ Native (same RPCs) |
|
||||||
|
| Moderation enforcement | ⚠️ Manual | ✅ At the boundary |
|
||||||
|
| Free editor features (undo, gizmos, inspector) | ✅ Rich | ❌ Build ourselves |
|
||||||
|
| Runtime footprint | ✅ None (stripped from export) | ⚠️ Ships in binary (gated) |
|
||||||
|
| Dev iteration speed | ✅ Fast | ⚠️ Slower to build |
|
||||||
|
| Deep asset authoring (mesh/material) | ✅ Native | ❌ Out of scope |
|
||||||
|
| CI testability | ⚠️ Heavier (headless editor) | ✅ Ordinary game code |
|
||||||
|
| Editor API churn risk | ⚠️ Higher | ✅ None (game code) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Recommendation
|
||||||
|
|
||||||
|
**Primary recommendation: build the in-game editor (b) as the platform's
|
||||||
|
authoring surface, and use a lightweight Godot editor plugin (a) only as an
|
||||||
|
internal dev convenience.**
|
||||||
|
|
||||||
|
### 6.1 Rationale
|
||||||
|
|
||||||
|
1. **Player UGC is the deciding factor.** The platform's value proposition is
|
||||||
|
user-generated content (rooms, worlds, avatars). Players cannot run the Godot
|
||||||
|
editor, so **only an in-game editor can deliver player authoring**. This is a
|
||||||
|
hard requirement that the editor-plugin approach cannot meet.
|
||||||
|
|
||||||
|
2. **The world is data, and the in-game editor is data-native.** `world.md`
|
||||||
|
defines chunks as the atomic unit. An in-game editor edits chunks directly
|
||||||
|
and submits them through the server pipeline. An editor plugin would force a
|
||||||
|
scene↔chunk translation layer that is a constant source of drift and bugs.
|
||||||
|
|
||||||
|
3. **Server authority and moderation fall out naturally.** Because in-game
|
||||||
|
edits use the same `request_chunk_edit` RPCs as gameplay, validation,
|
||||||
|
revisioning, optimistic concurrency, and content-flag enforcement
|
||||||
|
(`content_moderation.md`) apply uniformly. The editor cannot bypass the
|
||||||
|
server — which is exactly what a moderation-aware UGC platform needs.
|
||||||
|
|
||||||
|
4. **The editor-plugin's strengths are not what we need.** Its free features
|
||||||
|
(scene tree, inspector, gizmos) are for editing Godot scenes and resources.
|
||||||
|
Our authoring is world composition against a chunk contract, not scene
|
||||||
|
editing. The plugin's main win — deep asset authoring — is better served by
|
||||||
|
DCC tools and the Godot editor for the *platform team*, not by a plugin we
|
||||||
|
must maintain.
|
||||||
|
|
||||||
|
### 6.2 The Role of the Editor Plugin (Dev Convenience Only)
|
||||||
|
|
||||||
|
We still recommend a **small, optional** editor plugin for the **platform team**
|
||||||
|
to author **base content** (base rooms, templates, avatar base bodies, test
|
||||||
|
levels) directly in the Godot editor. This is a dev-time convenience, not a
|
||||||
|
product feature:
|
||||||
|
|
||||||
|
- It can export a scene/resource as a **chunk** (reusing the same encoder as the
|
||||||
|
in-game editor) so dev-authored content flows into the same pipeline.
|
||||||
|
- It is **not shipped** to players and is **not** the authoring surface for UGC.
|
||||||
|
- It should be kept thin to minimize editor-API churn risk.
|
||||||
|
|
||||||
|
### 6.3 Scope Boundary
|
||||||
|
|
||||||
|
To keep the in-game editor tractable, define its scope clearly:
|
||||||
|
|
||||||
|
- **In scope (world composition):** placing tiles, placing static/dynamic
|
||||||
|
objects, setting transforms, editing object properties, saving/loading chunks,
|
||||||
|
submitting edits to the server, showing moderation flags.
|
||||||
|
- **Out of scope (asset creation):** modeling meshes, authoring materials,
|
||||||
|
rigging avatars. These are done in DCC tools / the Godot editor and imported
|
||||||
|
as catalog assets (see `avatar.md` §7). The in-game editor **references**
|
||||||
|
catalog assets by id; it does not create them.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Resulting Architecture
|
||||||
|
|
||||||
|
### 7.1 Shared Editor Core
|
||||||
|
|
||||||
|
Both the in-game editor and the (optional) dev plugin should share a single
|
||||||
|
**editor core** that operates on the chunk model. This avoids two divergent
|
||||||
|
implementations.
|
||||||
|
|
||||||
|
```
|
||||||
|
EditorCore (shared, data-native)
|
||||||
|
├── ChunkEditorModel # in-memory chunk being edited (tiles, objects, flags)
|
||||||
|
├── ChunkEncoder # JSON/binary encode/decode (world.md §3)
|
||||||
|
├── EditSession # base_revision tracking, delta generation
|
||||||
|
├── EditSubmitter # request_chunk_edit RPC wrapper + conflict handling
|
||||||
|
├── UndoRedoStack # command pattern over chunk mutations
|
||||||
|
├── SelectionManager # selected tiles/objects
|
||||||
|
├── ToolRegistry # brush, object-place, select, erase, etc.
|
||||||
|
└── ModerationView # surfaces content flags; gates mature content
|
||||||
|
```
|
||||||
|
|
||||||
|
### 7.2 In-Game Editor (Primary)
|
||||||
|
|
||||||
|
```
|
||||||
|
InGameEditor (Control-based UI, shipped in binary, gated by feature flag)
|
||||||
|
├── EditorViewport # custom 2D/3D viewport with camera navigation
|
||||||
|
├── Toolbar # tool mode switcher
|
||||||
|
├── PalettePanel # tile/object palette (from catalog)
|
||||||
|
├── InspectorPanel # property editor for selected object
|
||||||
|
├── ChunkBrowser # load/save chunks, list rooms
|
||||||
|
└── EditorCore # shared core (above)
|
||||||
|
```
|
||||||
|
|
||||||
|
- Instantiated only when the player enters "build mode" (a separate scene /
|
||||||
|
feature flag), so the runtime footprint is negligible during normal play.
|
||||||
|
- Submits edits via `request_chunk_edit`; handles `chunk_edit_result` conflicts
|
||||||
|
by re-fetching and re-applying (`world.md` §5.4).
|
||||||
|
|
||||||
|
### 7.3 Dev Editor Plugin (Optional, Thin)
|
||||||
|
|
||||||
|
```
|
||||||
|
DevChunkPlugin (EditorPlugin, dev-only, not shipped)
|
||||||
|
├── Dock panel: "Chunk Export"
|
||||||
|
├── Reads the current scene / selected nodes
|
||||||
|
├── Maps scene content → chunk model (via EditorCore.ChunkEncoder)
|
||||||
|
└── Exports a .chunk.json / submits to a dev server
|
||||||
|
```
|
||||||
|
|
||||||
|
Kept thin to minimize editor-API churn. Its only job is to let the platform team
|
||||||
|
author base content in the editor and push it into the same chunk pipeline.
|
||||||
|
|
||||||
|
### 7.4 Data Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
[In-game editor] --request_chunk_edit(delta, base_revision)--> [Server]
|
||||||
|
^ | validate
|
||||||
|
| | apply
|
||||||
|
+--chunk_edit_result(ok/new_revision | conflict)-----------| persist
|
||||||
|
v
|
||||||
|
SQLite (world.db)
|
||||||
|
|
||||||
|
[Dev plugin] --export chunk JSON--> [EditorCore.ChunkEncoder] --> [Server]
|
||||||
|
```
|
||||||
|
|
||||||
|
Both paths converge on the **same server validation pipeline**, so moderation
|
||||||
|
and revisioning are uniform regardless of authoring surface.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Risks & Mitigations
|
||||||
|
|
||||||
|
| Risk | Mitigation |
|
||||||
|
|------|------------|
|
||||||
|
| In-game editor is a large build | Scope it to world composition (§6.3); reuse EditorCore; ship behind a feature flag |
|
||||||
|
| Custom viewport/gizmos are hard | Start with a 2D tile editor (matches `CHUNK_SIZE_Z=1` default); add 3D later |
|
||||||
|
| Undo/redo complexity | Use a command pattern over chunk mutations; test headlessly |
|
||||||
|
| Editor-plugin API churn | Keep the plugin thin; treat it as dev-only convenience, not a product surface |
|
||||||
|
| Drift between dev plugin and in-game editor | Both use the same `EditorCore`/`ChunkEncoder` |
|
||||||
|
| Players bypass moderation | Impossible by construction: edits go through server validation |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Open Questions / Future Work
|
||||||
|
|
||||||
|
- **3D editing:** The default world is flat (`CHUNK_SIZE_Z=1`). A 3D in-game
|
||||||
|
editor (volumetric chunks) is future work (`world.md` §8).
|
||||||
|
- **Collaborative editing:** Multiple players editing the same chunk concurrently
|
||||||
|
— the optimistic-concurrency model already supports it, but the UX (live
|
||||||
|
cursors, conflict resolution) is unspecified.
|
||||||
|
- **Template library:** A curated set of room templates players can start from.
|
||||||
|
- **Publish/review flow:** How a player's authored room enters the moderation
|
||||||
|
review queue (`content_moderation.md` §6) and becomes publicly joinable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Summary
|
||||||
|
|
||||||
|
| Concern | Decision |
|
||||||
|
|---------|----------|
|
||||||
|
| Primary authoring surface | **In-game editor** (ships to players, enables UGC) |
|
||||||
|
| Dev authoring convenience | Thin, optional **editor plugin** (not shipped) |
|
||||||
|
| World data model | Both operate on the **chunk model** via a shared `EditorCore` |
|
||||||
|
| Server authority | Edits go through `request_chunk_edit`; server validates |
|
||||||
|
| Moderation | Enforced at the server boundary for both paths |
|
||||||
|
| In-game editor scope | World composition only; asset creation is out of scope |
|
||||||
|
| Runtime footprint | Editor gated behind a feature flag / separate scene |
|
||||||
@@ -0,0 +1,286 @@
|
|||||||
|
# Multiplayer Networking Architecture
|
||||||
|
|
||||||
|
> **Status:** Proposed
|
||||||
|
> **Owner:** Platform team
|
||||||
|
> **Related:** Server authority + client prediction
|
||||||
|
|
||||||
|
## 1. Overview
|
||||||
|
|
||||||
|
This document specifies the multiplayer networking architecture for **agp-game**,
|
||||||
|
a Godot-based game built by autonomous agents. The architecture follows a
|
||||||
|
**server-authoritative model with client-side prediction and reconciliation**,
|
||||||
|
which is the industry standard for fast-paced, competitive multiplayer games.
|
||||||
|
|
||||||
|
The goals of this architecture are:
|
||||||
|
|
||||||
|
- **Fairness & anti-cheat:** The server is the single source of truth for all
|
||||||
|
gameplay state. Clients never directly mutate world state.
|
||||||
|
- **Responsiveness:** Client-side prediction hides network latency so local input
|
||||||
|
feels immediate.
|
||||||
|
- **Determinism:** The server runs a fixed-timestep simulation so that state is
|
||||||
|
reproducible and testable.
|
||||||
|
- **Scalability:** A clear separation between transport, simulation, and
|
||||||
|
replication layers allows the server to be tuned independently.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Transport Choice: ENet over WebSocket
|
||||||
|
|
||||||
|
### 2.1 Decision
|
||||||
|
|
||||||
|
We choose **ENet** as the primary transport for gameplay traffic.
|
||||||
|
|
||||||
|
### 2.2 Justification
|
||||||
|
|
||||||
|
| Criterion | ENet | WebSocket (TCP) |
|
||||||
|
|----------------------|----------------------------------------|-------------------------------------|
|
||||||
|
| **Reliability model**| Reliable + unreliable channels | Reliable only (TCP) |
|
||||||
|
| **Latency** | Low; unreliable packets skip retransmit| Head-of-line blocking on loss |
|
||||||
|
| **Packet ordering** | Per-channel ordering | Global strict ordering |
|
||||||
|
| **Congestion control**| Lightweight, game-oriented | TCP-style (slow start, backoff) |
|
||||||
|
| **Godot support** | First-class (`ENetConnection`, `MultiplayerPeerENet`) | Supported but TCP-based |
|
||||||
|
| **Use case fit** | Real-time action game | Chat, lobby, HTTP-style messaging |
|
||||||
|
|
||||||
|
**Rationale:**
|
||||||
|
|
||||||
|
- **Movement & state replication** are latency-sensitive and tolerate occasional
|
||||||
|
packet loss. ENet's *unreliable* channel lets us send position snapshots at high
|
||||||
|
frequency without head-of-line blocking.
|
||||||
|
- **RPCs that must be delivered** (e.g., interactions, spawns) use ENet's
|
||||||
|
*reliable* channel, giving us exactly the semantics we need without TCP's
|
||||||
|
retransmission stalls.
|
||||||
|
- Godot's `MultiplayerPeerENet` is mature and integrates directly with the
|
||||||
|
high-level multiplayer API (`MultiplayerAPI`, `rpc()`).
|
||||||
|
|
||||||
|
### 2.3 WebSocket as a Secondary Transport
|
||||||
|
|
||||||
|
WebSocket remains available as a **secondary transport** for:
|
||||||
|
|
||||||
|
- **Lobby / matchmaking** messaging (low-frequency, must-be-reliable).
|
||||||
|
- **Browser builds** where raw UDP/ENet is unavailable.
|
||||||
|
- **Admin / telemetry** channels.
|
||||||
|
|
||||||
|
The transport layer is abstracted behind an interface so gameplay code does not
|
||||||
|
depend on the underlying protocol.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Server-Side Game Loop
|
||||||
|
|
||||||
|
### 3.1 Tick Rate
|
||||||
|
|
||||||
|
- **Simulation tick rate:** **30 Hz** (33.33 ms per tick).
|
||||||
|
- **Replication / snapshot rate:** **30 Hz**, aligned with the simulation tick.
|
||||||
|
|
||||||
|
**Justification:**
|
||||||
|
|
||||||
|
- 30 Hz is sufficient for smooth movement and interaction in a top-down / 2D
|
||||||
|
action game while keeping CPU and bandwidth costs predictable.
|
||||||
|
- A fixed timestep (rather than delta-time) makes the simulation deterministic
|
||||||
|
and testable.
|
||||||
|
- The tick rate is a **configuration constant** so it can be raised (e.g., to
|
||||||
|
60 Hz) for competitive modes without architectural changes.
|
||||||
|
|
||||||
|
### 3.2 Authority Model
|
||||||
|
|
||||||
|
- **Server is authoritative** for:
|
||||||
|
- Player position, velocity, and physics state.
|
||||||
|
- Health, damage, and death.
|
||||||
|
- Item pickups, inventory, and world-object state.
|
||||||
|
- Spawning and despawning of entities.
|
||||||
|
- Game rules, scoring, and win/lose conditions.
|
||||||
|
- **Client is authoritative** for:
|
||||||
|
- Its own *input* (which keys/buttons are pressed).
|
||||||
|
- Local camera and UI.
|
||||||
|
- Cosmetic / non-gameplay presentation.
|
||||||
|
|
||||||
|
### 3.3 Server Loop Pseudocode
|
||||||
|
|
||||||
|
```
|
||||||
|
while server_running:
|
||||||
|
process_network_inbound() # read client inputs & RPCs
|
||||||
|
for _ in range(ticks_to_advance):
|
||||||
|
simulate_tick() # fixed 30 Hz step
|
||||||
|
broadcast_snapshots() # replicate authoritative state
|
||||||
|
process_network_outbound() # flush reliable/unreliable channels
|
||||||
|
```
|
||||||
|
|
||||||
|
The server accumulates real elapsed time and advances the simulation in fixed
|
||||||
|
`TICK_DURATION` increments, decoupling simulation from frame rate.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. RPC Call Patterns
|
||||||
|
|
||||||
|
RPCs are grouped by direction and reliability. All gameplay RPCs are routed
|
||||||
|
through the high-level `MultiplayerAPI` with explicit `rpc_id` targeting.
|
||||||
|
|
||||||
|
### 4.1 Client → Server (Input & Requests)
|
||||||
|
|
||||||
|
| RPC | Channel | Payload | Notes |
|
||||||
|
|------------------------------|-----------|-------------------------------------------|-------|
|
||||||
|
| `request_move_input` | unreliable| `{ seq, input_state, timestamp }` | Batched per tick; drives prediction |
|
||||||
|
| `request_interact` | reliable | `{ target_id }` | Open door, pick up item, use object |
|
||||||
|
| `request_attack` | reliable | `{ target_id, aim_dir }` | Combat action |
|
||||||
|
| `request_join` | reliable | `{ player_name, profile }` | Session setup |
|
||||||
|
| `request_respawn` | reliable | `{}` | After death |
|
||||||
|
|
||||||
|
**Design notes:**
|
||||||
|
|
||||||
|
- **Input is batched** per client tick and sent with a monotonically increasing
|
||||||
|
`seq` so the server can detect gaps and the client can reconcile.
|
||||||
|
- **Interactions are reliable** because they are discrete, must-happen events
|
||||||
|
(opening a door, picking up an item). Losing them would corrupt world state.
|
||||||
|
|
||||||
|
### 4.2 Server → Client (State & Events)
|
||||||
|
|
||||||
|
| RPC | Channel | Payload | Notes |
|
||||||
|
|------------------------------|-----------|-------------------------------------------|-------|
|
||||||
|
| `snapshot_state` | unreliable| `{ tick, entities[], player_states[] }` | Periodic authoritative state |
|
||||||
|
| `spawn_entity` | reliable | `{ entity_id, type, transform, props }` | New entity enters world |
|
||||||
|
| `despawn_entity` | reliable | `{ entity_id, reason }` | Entity removed |
|
||||||
|
| `interact_result` | reliable | `{ request_seq, success, result }` | Acknowledges interaction |
|
||||||
|
| `damage_event` | reliable | `{ target_id, amount, source_id }` | Combat feedback |
|
||||||
|
| `game_event` | reliable | `{ type, payload }` | Score, win, round start/end |
|
||||||
|
|
||||||
|
### 4.3 Broadcast vs. Targeted
|
||||||
|
|
||||||
|
- **Broadcast (all clients):** `snapshot_state`, `spawn_entity`,
|
||||||
|
`despawn_entity`, `game_event`.
|
||||||
|
- **Targeted (single client):** `interact_result`, `damage_event` (to affected
|
||||||
|
players), and any player-private data.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Client-Side Prediction & Reconciliation
|
||||||
|
|
||||||
|
### 5.1 Prediction
|
||||||
|
|
||||||
|
- The client runs a **local copy of the movement simulation** using the same
|
||||||
|
physics constants as the server.
|
||||||
|
- When the player provides input, the client:
|
||||||
|
1. Applies the input to its local player immediately.
|
||||||
|
2. Renders the predicted position.
|
||||||
|
3. Sends the input to the server (batched, with `seq`).
|
||||||
|
|
||||||
|
### 5.2 Reconciliation
|
||||||
|
|
||||||
|
- The server sends authoritative `snapshot_state` at 30 Hz containing the
|
||||||
|
server's tick number and authoritative player positions.
|
||||||
|
- The client keeps a **ring buffer of predicted states** keyed by input `seq`.
|
||||||
|
- On receiving a snapshot:
|
||||||
|
1. Find the last predicted state that matches the server's tick.
|
||||||
|
2. Compute the **error** between predicted and authoritative position.
|
||||||
|
3. If error exceeds a threshold, **snap or smoothly interpolate** the local
|
||||||
|
player to the authoritative position (error correction).
|
||||||
|
4. Re-simulate any inputs that occurred after the server's tick (rollback).
|
||||||
|
|
||||||
|
### 5.3 Interpolation of Remote Entities
|
||||||
|
|
||||||
|
- **Local player:** predicted + reconciled (see above).
|
||||||
|
- **Remote players / entities:** rendered with **interpolation delay** of
|
||||||
|
`2 × TICK_DURATION` (≈ 66 ms) to smooth out snapshot jitter.
|
||||||
|
- The client buffers the last N snapshots and interpolates between them.
|
||||||
|
|
||||||
|
### 5.4 Prediction Buffer Sizing
|
||||||
|
|
||||||
|
- Buffer size is adaptive based on measured RTT.
|
||||||
|
- Default: hold up to `ceil(RTT / TICK_DURATION) + 2` predicted states.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Entity & World State Replication
|
||||||
|
|
||||||
|
### 6.1 Entity Model
|
||||||
|
|
||||||
|
Every networked entity has:
|
||||||
|
|
||||||
|
- A globally unique `entity_id` (server-assigned).
|
||||||
|
- An `EntityType` (player, enemy, item, projectile, door, ...).
|
||||||
|
- A `Transform` (position, rotation).
|
||||||
|
- A typed `props` dictionary of gameplay state (health, state machine, etc.).
|
||||||
|
|
||||||
|
### 6.2 Snapshot Format
|
||||||
|
|
||||||
|
```
|
||||||
|
snapshot_state:
|
||||||
|
tick: uint32
|
||||||
|
entities:
|
||||||
|
- entity_id: uint32
|
||||||
|
type: uint8
|
||||||
|
transform: { x, y, rot }
|
||||||
|
props: { ... } # delta-encoded where possible
|
||||||
|
player_states:
|
||||||
|
- peer_id: int
|
||||||
|
entity_id: uint32
|
||||||
|
health: int
|
||||||
|
state: uint8
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.3 Delta Compression
|
||||||
|
|
||||||
|
- Only **changed** entity fields are sent per snapshot (dirty-flag per entity).
|
||||||
|
- Static or unchanged entities are omitted from the snapshot.
|
||||||
|
- This keeps bandwidth proportional to *activity*, not world size.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Connection & Session Lifecycle
|
||||||
|
|
||||||
|
1. **Connect:** Client opens an ENet connection to the server.
|
||||||
|
2. **Authenticate / Join:** Client sends `request_join`; server validates and
|
||||||
|
assigns a `peer_id` and spawns the player entity.
|
||||||
|
3. **Spawn:** Server broadcasts `spawn_entity` for the new player to all clients.
|
||||||
|
4. **Play:** Input → prediction → server simulation → snapshot → reconciliation.
|
||||||
|
5. **Disconnect:** Server removes the player entity, broadcasts `despawn_entity`,
|
||||||
|
and cleans up session state.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Security & Anti-Cheat
|
||||||
|
|
||||||
|
- **No client-authoritative gameplay values.** Clients only send *inputs* and
|
||||||
|
*requests*; the server computes all outcomes.
|
||||||
|
- **Input validation:** Server clamps movement speed, rejects out-of-bounds
|
||||||
|
positions, and validates interaction targets.
|
||||||
|
- **Rate limiting:** Per-peer input and RPC rate limits prevent flooding.
|
||||||
|
- **Sequence validation:** Gaps or replays in input `seq` are detected and
|
||||||
|
rejected.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Testing & Determinism
|
||||||
|
|
||||||
|
- The simulation is a **pure function of (state, inputs, tick)** so it can be
|
||||||
|
unit-tested headlessly.
|
||||||
|
- A **replay system** records inputs and can re-simulate to verify server and
|
||||||
|
client produce identical results.
|
||||||
|
- Network code is tested with a **mock transport** that can inject latency,
|
||||||
|
jitter, and packet loss.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Open Questions / Future Work
|
||||||
|
|
||||||
|
- **Interest management / spatial partitioning:** For large worlds, restrict
|
||||||
|
snapshots to entities near each client (AOI culling).
|
||||||
|
- **Server-side physics:** Decide whether to use Godot's physics server on the
|
||||||
|
host or a custom deterministic physics step.
|
||||||
|
- **Dedicated server vs. listen server:** The architecture supports both; a
|
||||||
|
dedicated headless server is the target for production.
|
||||||
|
- **WebSocket fallback:** Implement the transport abstraction for browser builds.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Summary
|
||||||
|
|
||||||
|
| Concern | Decision |
|
||||||
|
|------------------------|-------------------------------------------------|
|
||||||
|
| Transport | ENet (primary), WebSocket (secondary) |
|
||||||
|
| Tick rate | 30 Hz fixed timestep |
|
||||||
|
| Authority | Server-authoritative; client sends input only |
|
||||||
|
| Prediction | Client-side prediction + reconciliation |
|
||||||
|
| Remote interpolation | 2-tick interpolation delay |
|
||||||
|
| Replication | Delta-compressed snapshots at 30 Hz |
|
||||||
|
| Interaction RPCs | Reliable channel |
|
||||||
|
| Movement input | Unreliable channel, batched with sequence |
|
||||||
@@ -0,0 +1,737 @@
|
|||||||
|
# World Chunk Data Format & Persistence Architecture
|
||||||
|
|
||||||
|
> **Status:** Proposed
|
||||||
|
> **Owner:** Platform team
|
||||||
|
> **Related:** Server authority + client prediction, avatar component system, content moderation & age verification, backend auth (JWT)
|
||||||
|
|
||||||
|
## 1. Overview
|
||||||
|
|
||||||
|
This document specifies the **world chunk data format** and **persistence
|
||||||
|
schema** for **agp-game**, a Godot-based multiplayer game built by autonomous
|
||||||
|
agents. It defines:
|
||||||
|
|
||||||
|
- The **chunk data format** — both a human-readable **JSON** form (for
|
||||||
|
authoring, tooling, and debugging) and a compact **binary** form (for
|
||||||
|
runtime loading and network transfer).
|
||||||
|
- The **SQLite schema** that persists world state on the server, including
|
||||||
|
chunk metadata, chunk payloads, and the world/room hierarchy.
|
||||||
|
- The **read/write API** that the server and client use to load, save, and
|
||||||
|
stream chunks.
|
||||||
|
|
||||||
|
The goal is a **single, versioned chunk contract** that is shared across the
|
||||||
|
Godot client, the Python backend, and the persistence layer. A chunk is the
|
||||||
|
atomic unit of world geometry, gameplay objects, and persistence — the server
|
||||||
|
loads and saves whole chunks, and clients stream the chunks around them.
|
||||||
|
|
||||||
|
### 1.1 Design Goals
|
||||||
|
|
||||||
|
- **Versioned & forward-compatible:** Every chunk carries a format version so
|
||||||
|
the loader can migrate old data and reject unknown future data gracefully.
|
||||||
|
- **Dual representation:** JSON for authoring/debugging, binary for
|
||||||
|
runtime/network. Both encode the *same* logical model so there is exactly one
|
||||||
|
source of truth for the schema.
|
||||||
|
- **Server-authoritative persistence:** The server is the only writer to the
|
||||||
|
SQLite store. Clients submit *edits* (deltas) and the server applies and
|
||||||
|
persists them, mirroring the server-authoritative networking model.
|
||||||
|
- **Chunk-granular persistence:** Persistence happens at chunk granularity so
|
||||||
|
the server can save a dirty chunk without rewriting the whole world, and can
|
||||||
|
stream only the chunks a client needs.
|
||||||
|
- **Deterministic & testable:** The binary format is byte-stable for a given
|
||||||
|
logical payload, enabling checksums, caching, and reproducible tests.
|
||||||
|
- **Moderation-aware:** Chunks reference content (assets, scripts) that carry
|
||||||
|
content flags (see `content_moderation.md`); the persistence layer stores the
|
||||||
|
moderation state alongside chunk data so enforcement can happen at load time.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. World Model & Chunking
|
||||||
|
|
||||||
|
### 2.1 World Hierarchy
|
||||||
|
|
||||||
|
The world is organized as a hierarchy that matches the moderation content graph:
|
||||||
|
|
||||||
|
```
|
||||||
|
World
|
||||||
|
└── Room
|
||||||
|
└── Chunk (the atomic persistence & streaming unit)
|
||||||
|
├── Tiles / Terrain
|
||||||
|
├── Static objects (props, geometry)
|
||||||
|
├── Dynamic objects (spawnable entities, interactables)
|
||||||
|
└── Metadata (flags, version, timestamps)
|
||||||
|
```
|
||||||
|
|
||||||
|
- A **World** is the top-level container (a named, versioned space).
|
||||||
|
- A **Room** is a bounded region within a world (a level, a building, a zone).
|
||||||
|
- A **Chunk** is a fixed-size, axis-aligned region of a room. It is the unit of
|
||||||
|
load, save, and stream.
|
||||||
|
|
||||||
|
### 2.2 Chunk Dimensions
|
||||||
|
|
||||||
|
Chunks are **fixed-size, axis-aligned boxes** in world space. The default
|
||||||
|
dimensions are a configuration constant so they can be tuned per world:
|
||||||
|
|
||||||
|
| Constant | Default | Description |
|
||||||
|
|---------------------|---------|----------------------------------------------------|
|
||||||
|
| `CHUNK_SIZE_X` | 32 | Chunk width in world units (tiles). |
|
||||||
|
| `CHUNK_SIZE_Y` | 32 | Chunk height in world units (tiles). |
|
||||||
|
| `CHUNK_SIZE_Z` | 1 | Chunk depth (1 = flat 2D world; >1 = volumetric). |
|
||||||
|
| `CHUNK_ORIGIN` | `(0,0)` | World-space origin of chunk coordinate `(0,0)`. |
|
||||||
|
|
||||||
|
A chunk is identified by its **integer chunk coordinate** `(cx, cy, cz)`,
|
||||||
|
computed from a world-space position `p` as:
|
||||||
|
|
||||||
|
```
|
||||||
|
cx = floor((p.x - CHUNK_ORIGIN.x) / CHUNK_SIZE_X)
|
||||||
|
cy = floor((p.y - CHUNK_ORIGIN.y) / CHUNK_SIZE_Y)
|
||||||
|
cz = floor((p.z - CHUNK_ORIGIN.z) / CHUNK_SIZE_Z)
|
||||||
|
```
|
||||||
|
|
||||||
|
The chunk's world-space bounding box is therefore:
|
||||||
|
|
||||||
|
```
|
||||||
|
min = (CHUNK_ORIGIN.x + cx*CHUNK_SIZE_X, ...)
|
||||||
|
max = min + (CHUNK_SIZE_X, CHUNK_SIZE_Y, CHUNK_SIZE_Z)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2.3 Chunk Coordinate Key
|
||||||
|
|
||||||
|
For storage and lookup, a chunk is addressed by a **canonical string key**:
|
||||||
|
|
||||||
|
```
|
||||||
|
"{world_id}:{room_id}:{cx}:{cy}:{cz}"
|
||||||
|
```
|
||||||
|
|
||||||
|
Example: `"w_0001:r_0001:12:-4:0"`. This key is the primary lookup key in the
|
||||||
|
SQLite `chunks` table and is used in the read/write API.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Chunk Data Format
|
||||||
|
|
||||||
|
### 3.1 Logical Model
|
||||||
|
|
||||||
|
Both the JSON and binary encodings serialize the same logical chunk model. The
|
||||||
|
logical model is defined here once; the two encodings are projections of it.
|
||||||
|
|
||||||
|
```
|
||||||
|
Chunk
|
||||||
|
├── version : uint (format version, currently 1)
|
||||||
|
├── key : string (canonical chunk key, see §2.3)
|
||||||
|
├── world_id : string
|
||||||
|
├── room_id : string
|
||||||
|
├── coord : { cx, cy, cz } (int)
|
||||||
|
├── bounds : { min_x, min_y, min_z, max_x, max_y, max_z } (float)
|
||||||
|
├── tiles : TileLayer[] (terrain / tile data)
|
||||||
|
├── objects : WorldObject[] (static + dynamic objects)
|
||||||
|
├── flags : ContentFlags (moderation flags, see §3.5)
|
||||||
|
├── meta : { created_at, updated_at, revision, author }
|
||||||
|
└── checksum : string (SHA-256 of canonical payload)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.2 Tile Layer
|
||||||
|
|
||||||
|
A chunk's terrain is stored as one or more **tile layers**. Each layer is a
|
||||||
|
dense grid of tile IDs of size `CHUNK_SIZE_X × CHUNK_SIZE_Y` (× `CHUNK_SIZE_Z`
|
||||||
|
for volumetric worlds).
|
||||||
|
|
||||||
|
```
|
||||||
|
TileLayer
|
||||||
|
├── name : string (e.g. "ground", "walls", "decor")
|
||||||
|
├── layer_id : uint
|
||||||
|
├── width : uint (= CHUNK_SIZE_X)
|
||||||
|
├── height : uint (= CHUNK_SIZE_Y)
|
||||||
|
├── depth : uint (= CHUNK_SIZE_Z)
|
||||||
|
├── tiles : uint[] (flattened tile IDs, row-major, length = w*h*d)
|
||||||
|
└── palette : TileDef[] (tile ID → tile definition, shared per layer)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Tile encoding:** A tile ID is a `uint16` index into the layer's `palette`.
|
||||||
|
`0` is reserved for "empty/air". The palette maps each tile ID to a `TileDef`:
|
||||||
|
|
||||||
|
```
|
||||||
|
TileDef
|
||||||
|
├── id : uint16
|
||||||
|
├── asset_id : string (reference to a tile/terrain asset)
|
||||||
|
├── solid : bool (collision)
|
||||||
|
├── flags : uint32 (bitmask: walkable, climbable, liquid, ...)
|
||||||
|
└── variants : uint8 (number of visual variants for this tile)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Rationale for palette + indices:** Storing dense `uint16` indices rather than
|
||||||
|
full asset strings per cell keeps the binary payload small (2 bytes/cell) while
|
||||||
|
the palette (shared across the layer) carries the rich per-tile metadata.
|
||||||
|
|
||||||
|
### 3.3 World Objects
|
||||||
|
|
||||||
|
Objects are the non-tile entities in a chunk: static props, interactables,
|
||||||
|
spawn points, and dynamic-object templates.
|
||||||
|
|
||||||
|
```
|
||||||
|
WorldObject
|
||||||
|
├── id : string (stable object id, unique within the chunk)
|
||||||
|
├── type : string (e.g. "prop", "interactable", "spawn_point", "door")
|
||||||
|
├── asset_id : string (reference to the object's mesh/glTF asset)
|
||||||
|
├── transform : { pos: [x,y,z], rot: [x,y,z,w], scale: [x,y,z] }
|
||||||
|
├── properties : object (type-specific key/value map, JSON-typed)
|
||||||
|
├── dynamic : bool (true if this is a spawnable dynamic entity)
|
||||||
|
└── flags : ContentFlags (per-object moderation flags)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Dynamic objects** are stored as *templates* in the chunk. Their *runtime
|
||||||
|
state* (position, health, inventory) is **not** stored in the chunk; it lives
|
||||||
|
in the runtime entity store and is persisted separately (see §4.4). This keeps
|
||||||
|
chunks stable and diffable — a chunk only changes when its *static* content
|
||||||
|
changes.
|
||||||
|
|
||||||
|
### 3.4 Content Flags
|
||||||
|
|
||||||
|
Each chunk (and each object) carries a **content flag set** that is the
|
||||||
|
aggregate of the moderation flags of the assets and scripts it references. This
|
||||||
|
mirrors the aggregation rules in `content_moderation.md`:
|
||||||
|
|
||||||
|
```
|
||||||
|
ContentFlags
|
||||||
|
├── categories : { violence, sexual, language, substance, gambling, horror, hate, custom }
|
||||||
|
│ (each → severity: none|mild|moderate|severe)
|
||||||
|
├── mature : bool (true if any contributing item is mature)
|
||||||
|
└── sources : string[] (flag ids that contributed, for audit)
|
||||||
|
```
|
||||||
|
|
||||||
|
The server computes and persists these flags at write time so that enforcement
|
||||||
|
(age gating, content filtering) can happen at load time without re-scanning
|
||||||
|
assets.
|
||||||
|
|
||||||
|
### 3.5 JSON Encoding
|
||||||
|
|
||||||
|
The JSON encoding is the **authoring and debugging** representation. It is
|
||||||
|
human-readable, diffable, and used by tools and tests. It is the canonical
|
||||||
|
logical model serialized directly.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"key": "w_0001:r_0001:12:-4:0",
|
||||||
|
"world_id": "w_0001",
|
||||||
|
"room_id": "r_0001",
|
||||||
|
"coord": { "cx": 12, "cy": -4, "cz": 0 },
|
||||||
|
"bounds": { "min_x": 384.0, "min_y": -128.0, "min_z": 0.0,
|
||||||
|
"max_x": 416.0, "max_y": -96.0, "max_z": 1.0 },
|
||||||
|
"tiles": [
|
||||||
|
{
|
||||||
|
"name": "ground",
|
||||||
|
"layer_id": 0,
|
||||||
|
"width": 32,
|
||||||
|
"height": 32,
|
||||||
|
"depth": 1,
|
||||||
|
"palette": [
|
||||||
|
{ "id": 0, "asset_id": "", "solid": false, "flags": 0, "variants": 1 },
|
||||||
|
{ "id": 1, "asset_id": "tiles/grass", "solid": false, "flags": 1, "variants": 4 }
|
||||||
|
],
|
||||||
|
"tiles": [0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"objects": [
|
||||||
|
{
|
||||||
|
"id": "obj_0001",
|
||||||
|
"type": "interactable",
|
||||||
|
"asset_id": "props/chest",
|
||||||
|
"transform": { "pos": [400.0, -110.0, 0.0],
|
||||||
|
"rot": [0.0, 0.0, 0.0, 1.0],
|
||||||
|
"scale": [1.0, 1.0, 1.0] },
|
||||||
|
"properties": { "loot_table": "chest_basic", "locked": false },
|
||||||
|
"dynamic": false,
|
||||||
|
"flags": { "categories": { "violence": "none" }, "mature": false, "sources": [] }
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"flags": { "categories": { "violence": "none" }, "mature": false, "sources": [] },
|
||||||
|
"meta": { "created_at": "2026-08-17T00:00:00Z",
|
||||||
|
"updated_at": "2026-08-17T00:00:00Z",
|
||||||
|
"revision": 3,
|
||||||
|
"author": "agent-bot" },
|
||||||
|
"checksum": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Note on the example checksum:** The value shown above is illustrative only.
|
||||||
|
> In practice the `checksum` field is the SHA-256 of the canonical binary
|
||||||
|
> payload body (see §3.6.3), not of the JSON text. The value here is a
|
||||||
|
> placeholder for documentation purposes.
|
||||||
|
|
||||||
|
### 3.6 Binary Encoding
|
||||||
|
|
||||||
|
The binary encoding is the **runtime and network** representation. It is compact
|
||||||
|
and byte-stable. It uses a simple, explicitly-specified layout (no external
|
||||||
|
serialization library) so it can be produced and consumed identically by the
|
||||||
|
Godot client (GDScript) and the Python backend.
|
||||||
|
|
||||||
|
#### 3.6.1 Header
|
||||||
|
|
||||||
|
| Offset | Size | Field | Type / Encoding |
|
||||||
|
|--------|------|--------------|------------------------------------------|
|
||||||
|
| 0 | 4 | magic | ASCII `"AGPC"` (0x41 0x47 0x50 0x43) |
|
||||||
|
| 4 | 2 | version | `uint16` LE (currently `1`) |
|
||||||
|
| 6 | 2 | header_len | `uint16` LE (bytes of header after this) |
|
||||||
|
| 8 | 4 | flags | `uint32` LE (bitmask, reserved) |
|
||||||
|
| 12 | 4 | payload_len | `uint32` LE (bytes of payload body) |
|
||||||
|
| 16 | 32 | checksum | SHA-256 of payload body (hex, 32 bytes) |
|
||||||
|
|
||||||
|
Total header size: **48 bytes**.
|
||||||
|
|
||||||
|
#### 3.6.2 Payload Body
|
||||||
|
|
||||||
|
The payload body is a sequence of typed sections. Each section has a small
|
||||||
|
header:
|
||||||
|
|
||||||
|
| Offset | Size | Field | Type / Encoding |
|
||||||
|
|--------|------|------------|----------------------------|
|
||||||
|
| 0 | 1 | section_id | `uint8` (see table below) |
|
||||||
|
| 1 | 4 | section_len| `uint32` LE (bytes of section data) |
|
||||||
|
| 5 | ... | data | section-specific |
|
||||||
|
|
||||||
|
Section IDs:
|
||||||
|
|
||||||
|
| section_id | Section | Data layout |
|
||||||
|
|------------|----------------|-------------------------------------------------------------------|
|
||||||
|
| `0x01` | metadata | world_id, room_id (length-prefixed strings), coord (3×int32), bounds (6×float32) |
|
||||||
|
| `0x02` | tile layer | name, layer_id, dims (3×uint16), palette, dense tile indices |
|
||||||
|
| `0x03` | object | one `WorldObject` (repeated; one section per object) |
|
||||||
|
| `0x04` | content flags | category severities (8×uint8), mature (uint8), sources (string list) |
|
||||||
|
| `0x05` | meta | created_at, updated_at (int64 unix), revision (uint32), author (string) |
|
||||||
|
|
||||||
|
**String encoding:** length-prefixed UTF-8 — `uint16` LE byte length followed by
|
||||||
|
the UTF-8 bytes.
|
||||||
|
|
||||||
|
**Tile layer data layout (section `0x02`):**
|
||||||
|
|
||||||
|
```
|
||||||
|
name : string
|
||||||
|
layer_id : uint16 LE
|
||||||
|
width : uint16 LE
|
||||||
|
height : uint16 LE
|
||||||
|
depth : uint16 LE
|
||||||
|
palette_count : uint16 LE
|
||||||
|
palette : palette_count × TileDef
|
||||||
|
tile_count : uint32 LE
|
||||||
|
tiles : tile_count × uint16 LE (dense, row-major)
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
TileDef:
|
||||||
|
id : uint16 LE
|
||||||
|
asset_id : string
|
||||||
|
solid : uint8
|
||||||
|
flags : uint32 LE
|
||||||
|
variants : uint8
|
||||||
|
```
|
||||||
|
|
||||||
|
**Object data layout (section `0x03`):**
|
||||||
|
|
||||||
|
```
|
||||||
|
id : string
|
||||||
|
type : string
|
||||||
|
asset_id : string
|
||||||
|
pos : 3 × float32
|
||||||
|
rot : 4 × float32
|
||||||
|
scale : 3 × float32
|
||||||
|
dynamic : uint8
|
||||||
|
properties : JSON string (length-prefixed)
|
||||||
|
flags : ContentFlags (see 0x04 layout)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Content flags layout (section `0x04`):**
|
||||||
|
|
||||||
|
```
|
||||||
|
violence, sexual, language, substance, gambling, horror, hate, custom
|
||||||
|
: 8 × uint8 (severity enum: 0=none,1=mild,2=moderate,3=severe)
|
||||||
|
mature : uint8
|
||||||
|
source_count: uint16 LE
|
||||||
|
sources : source_count × string
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 3.6.3 Byte-Stability
|
||||||
|
|
||||||
|
For a given logical payload, the binary encoding is **deterministic**:
|
||||||
|
|
||||||
|
- All integers are little-endian.
|
||||||
|
- All strings are UTF-8, length-prefixed, and never null-terminated.
|
||||||
|
- Maps (e.g. `properties`) are serialized as a **canonical JSON string** with
|
||||||
|
keys sorted lexicographically.
|
||||||
|
- The `checksum` is the SHA-256 of the payload body bytes.
|
||||||
|
|
||||||
|
This guarantees that two encoders produce identical bytes, enabling content
|
||||||
|
addressing, caching, and reproducible tests.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. SQLite Persistence Schema
|
||||||
|
|
||||||
|
The server persists world state in a **SQLite** database. SQLite is chosen for
|
||||||
|
its zero-configuration, single-file, transactional, and embedded nature — ideal
|
||||||
|
for a server that owns the world store and needs ACID guarantees without a
|
||||||
|
separate database process.
|
||||||
|
|
||||||
|
### 4.1 Database Layout
|
||||||
|
|
||||||
|
The database file is `world.db` (path configurable). It contains the following
|
||||||
|
tables:
|
||||||
|
|
||||||
|
```
|
||||||
|
worlds
|
||||||
|
rooms
|
||||||
|
chunks
|
||||||
|
chunk_objects
|
||||||
|
content_flags
|
||||||
|
revisions
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.2 `worlds`
|
||||||
|
|
||||||
|
Top-level world containers.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE worlds (
|
||||||
|
id TEXT PRIMARY KEY, -- e.g. 'w_0001'
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
description TEXT,
|
||||||
|
chunk_size_x INTEGER NOT NULL DEFAULT 32,
|
||||||
|
chunk_size_y INTEGER NOT NULL DEFAULT 32,
|
||||||
|
chunk_size_z INTEGER NOT NULL DEFAULT 1,
|
||||||
|
origin_x REAL NOT NULL DEFAULT 0,
|
||||||
|
origin_y REAL NOT NULL DEFAULT 0,
|
||||||
|
origin_z REAL NOT NULL DEFAULT 0,
|
||||||
|
created_at TEXT NOT NULL, -- ISO-8601 UTC
|
||||||
|
updated_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.3 `rooms`
|
||||||
|
|
||||||
|
Rooms within a world.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE rooms (
|
||||||
|
id TEXT PRIMARY KEY, -- e.g. 'r_0001'
|
||||||
|
world_id TEXT NOT NULL REFERENCES worlds(id) ON DELETE CASCADE,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
bounds_min_x REAL NOT NULL,
|
||||||
|
bounds_min_y REAL NOT NULL,
|
||||||
|
bounds_min_z REAL NOT NULL DEFAULT 0,
|
||||||
|
bounds_max_x REAL NOT NULL,
|
||||||
|
bounds_max_y REAL NOT NULL,
|
||||||
|
bounds_max_z REAL NOT NULL DEFAULT 1,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL,
|
||||||
|
UNIQUE (world_id, name)
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_rooms_world ON rooms(world_id);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.4 `chunks`
|
||||||
|
|
||||||
|
The core persistence table. One row per chunk. The **payload** is stored as a
|
||||||
|
BLOB in the **binary format** (§3.6); the JSON form is derived on demand for
|
||||||
|
authoring/debugging and is not stored (to avoid dual-write drift).
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE chunks (
|
||||||
|
key TEXT PRIMARY KEY, -- canonical key 'world:room:cx:cy:cz'
|
||||||
|
world_id TEXT NOT NULL REFERENCES worlds(id) ON DELETE CASCADE,
|
||||||
|
room_id TEXT NOT NULL REFERENCES rooms(id) ON DELETE CASCADE,
|
||||||
|
cx INTEGER NOT NULL,
|
||||||
|
cy INTEGER NOT NULL,
|
||||||
|
cz INTEGER NOT NULL DEFAULT 0,
|
||||||
|
format_version INTEGER NOT NULL DEFAULT 1,
|
||||||
|
payload BLOB NOT NULL, -- binary chunk encoding (§3.6)
|
||||||
|
payload_len INTEGER NOT NULL,
|
||||||
|
checksum TEXT NOT NULL, -- SHA-256 of payload
|
||||||
|
revision INTEGER NOT NULL DEFAULT 0,
|
||||||
|
dirty INTEGER NOT NULL DEFAULT 0, -- 1 = needs flush to disk
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
updated_at TEXT NOT NULL,
|
||||||
|
UNIQUE (world_id, room_id, cx, cy, cz)
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_chunks_world_room ON chunks(world_id, room_id);
|
||||||
|
CREATE INDEX idx_chunks_coord ON chunks(cx, cy, cz);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Design notes:**
|
||||||
|
|
||||||
|
- The `payload` BLOB is the single source of truth for chunk content. The
|
||||||
|
`checksum` guards against corruption and enables content addressing.
|
||||||
|
- `revision` increments on every write; it is used for optimistic concurrency
|
||||||
|
and change detection (see §5.4).
|
||||||
|
- `dirty` marks chunks that have been modified in memory but not yet flushed,
|
||||||
|
enabling a write-behind persistence strategy (see §5.5).
|
||||||
|
|
||||||
|
### 4.5 `chunk_objects`
|
||||||
|
|
||||||
|
A denormalized, queryable index of the objects inside each chunk. This is
|
||||||
|
**derived** from the chunk payload and kept in sync by the write path. It exists
|
||||||
|
so the server can answer queries like "which chunks contain a door?" without
|
||||||
|
parsing every payload BLOB.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE chunk_objects (
|
||||||
|
id TEXT NOT NULL, -- object id (unique within chunk)
|
||||||
|
chunk_key TEXT NOT NULL REFERENCES chunks(key) ON DELETE CASCADE,
|
||||||
|
type TEXT NOT NULL, -- 'prop', 'interactable', ...
|
||||||
|
asset_id TEXT,
|
||||||
|
dynamic INTEGER NOT NULL DEFAULT 0,
|
||||||
|
pos_x REAL NOT NULL,
|
||||||
|
pos_y REAL NOT NULL,
|
||||||
|
pos_z REAL NOT NULL DEFAULT 0,
|
||||||
|
PRIMARY KEY (chunk_key, id)
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_chunk_objects_type ON chunk_objects(type);
|
||||||
|
CREATE INDEX idx_chunk_objects_asset ON chunk_objects(asset_id);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Sync mechanism:** `chunk_objects` is maintained **explicitly by the write
|
||||||
|
path** — it is not a SQL trigger. Whenever `put_chunk` (or `delete_chunk`)
|
||||||
|
commits a chunk, the service layer performs a **delete-and-reinsert** of that
|
||||||
|
chunk's object rows in the same transaction: it deletes all rows for
|
||||||
|
`chunk_key`, then inserts fresh rows parsed from the new payload. This keeps the
|
||||||
|
index transactionally consistent with the payload and avoids the complexity and
|
||||||
|
fragility of SQL triggers over a binary BLOB. Because the index is derived, it
|
||||||
|
can always be rebuilt from `chunks.payload` if it ever drifts (a `rebuild_index`
|
||||||
|
utility is provided for this).
|
||||||
|
|
||||||
|
### 4.6 `content_flags`
|
||||||
|
|
||||||
|
Persists the aggregated moderation flags per chunk (and optionally per object).
|
||||||
|
This lets the server enforce age/content gates at load time without parsing
|
||||||
|
payloads.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE content_flags (
|
||||||
|
chunk_key TEXT NOT NULL REFERENCES chunks(key) ON DELETE CASCADE,
|
||||||
|
object_id TEXT NOT NULL DEFAULT '', -- '' = chunk-level flag (sentinel)
|
||||||
|
violence INTEGER NOT NULL DEFAULT 0,
|
||||||
|
sexual INTEGER NOT NULL DEFAULT 0,
|
||||||
|
language INTEGER NOT NULL DEFAULT 0,
|
||||||
|
substance INTEGER NOT NULL DEFAULT 0,
|
||||||
|
gambling INTEGER NOT NULL DEFAULT 0,
|
||||||
|
horror INTEGER NOT NULL DEFAULT 0,
|
||||||
|
hate INTEGER NOT NULL DEFAULT 0,
|
||||||
|
custom INTEGER NOT NULL DEFAULT 0,
|
||||||
|
mature INTEGER NOT NULL DEFAULT 0,
|
||||||
|
sources TEXT NOT NULL DEFAULT '[]', -- JSON array of flag ids
|
||||||
|
PRIMARY KEY (chunk_key, object_id)
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_content_flags_mature ON content_flags(mature);
|
||||||
|
```
|
||||||
|
|
||||||
|
Severity is stored as `0=none, 1=mild, 2=moderate, 3=severe`.
|
||||||
|
|
||||||
|
**Chunk-level vs. object-level flags:** Chunk-level flags use the **sentinel
|
||||||
|
value `''` (empty string)** in `object_id` rather than `NULL`. This is
|
||||||
|
deliberate: in SQLite, `NULL` values in a composite primary key are **not**
|
||||||
|
unique-constrained, so two rows with the same `chunk_key` and `object_id = NULL`
|
||||||
|
could be inserted accidentally. Using `''` as the sentinel means the composite
|
||||||
|
`PRIMARY KEY (chunk_key, object_id)` enforces that there is **exactly one**
|
||||||
|
chunk-level flag row per chunk. Object-level flags use the object's actual `id`
|
||||||
|
(which is never empty). The application layer treats `object_id = ''` as
|
||||||
|
"chunk-level" and any non-empty value as "object-level".
|
||||||
|
|
||||||
|
### 4.7 `revisions`
|
||||||
|
|
||||||
|
An append-only audit log of chunk writes. Enables rollback, auditing, and
|
||||||
|
replication of changes.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE revisions (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
chunk_key TEXT NOT NULL, -- NOT a FK: survives chunk deletion
|
||||||
|
revision INTEGER NOT NULL,
|
||||||
|
actor TEXT NOT NULL, -- user / agent id
|
||||||
|
action TEXT NOT NULL, -- 'create' | 'update' | 'delete'
|
||||||
|
checksum TEXT NOT NULL, -- SHA-256 of payload
|
||||||
|
payload BLOB, -- full binary payload at this revision
|
||||||
|
created_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_revisions_chunk ON revisions(chunk_key);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Rollback support:** The `payload` BLOB stores the **full binary chunk payload**
|
||||||
|
at each revision. This is what makes rollback possible: to restore a chunk to a
|
||||||
|
previous revision, the server reads the `payload` of the target revision and
|
||||||
|
writes it back to `chunks.payload` (incrementing `revision` and appending a new
|
||||||
|
`revisions` row). Without the payload, only the checksum would be available,
|
||||||
|
which is insufficient to reconstruct the chunk.
|
||||||
|
|
||||||
|
**Audit trail preservation:** `chunk_key` is **not** a foreign key to `chunks`.
|
||||||
|
This means deleting a chunk does **not** cascade-delete its revision history —
|
||||||
|
the audit trail is preserved even after a chunk is removed. (The `chunk_key`
|
||||||
|
string is still stored so the history can be correlated to a chunk that existed
|
||||||
|
at that time.) If a chunk is later recreated with the same key, its history is
|
||||||
|
still intact.
|
||||||
|
|
||||||
|
### 4.8 Schema Versioning & Migrations
|
||||||
|
|
||||||
|
The database carries a schema version in a `meta` table:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE TABLE meta (
|
||||||
|
key TEXT PRIMARY KEY,
|
||||||
|
value TEXT NOT NULL
|
||||||
|
);
|
||||||
|
-- seed: INSERT INTO meta(key, value) VALUES ('schema_version', '1');
|
||||||
|
```
|
||||||
|
|
||||||
|
Migrations are applied in order at startup. Each migration is a numbered SQL
|
||||||
|
script; the server tracks the applied version in `meta` and applies any pending
|
||||||
|
migrations transactionally. This keeps the schema forward-compatible as the
|
||||||
|
chunk format evolves.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Read/Write API
|
||||||
|
|
||||||
|
The API is exposed as a **Python service layer** on the backend (the same
|
||||||
|
process that owns the SQLite store) and consumed by the Godot client over the
|
||||||
|
network transport (see `networking.md`). The service layer is the **only**
|
||||||
|
writer to the database; clients never touch SQLite directly.
|
||||||
|
|
||||||
|
### 5.1 Service Interface (Python)
|
||||||
|
|
||||||
|
```python
|
||||||
|
class WorldStore:
|
||||||
|
"""Owns the SQLite connection and chunk read/write operations."""
|
||||||
|
|
||||||
|
def get_chunk(self, key: str) -> Chunk | None: ...
|
||||||
|
def get_chunk_binary(self, key: str) -> bytes | None: ...
|
||||||
|
def put_chunk(self, chunk: Chunk, actor: str) -> Chunk: ...
|
||||||
|
def delete_chunk(self, key: str, actor: str) -> None: ...
|
||||||
|
def list_chunks(self, world_id: str, room_id: str | None = None,
|
||||||
|
bounds: Bounds | None = None) -> list[str]: ...
|
||||||
|
def get_chunk_flags(self, key: str) -> ContentFlags: ...
|
||||||
|
def flush_dirty(self) -> int: ...
|
||||||
|
def rollback_chunk(self, key: str, revision: int, actor: str) -> Chunk: ...
|
||||||
|
def rebuild_index(self, chunk_key: str | None = None) -> int: ...
|
||||||
|
def migrate(self) -> None: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.2 Encoding Helpers
|
||||||
|
|
||||||
|
The service layer provides symmetric encode/decode functions shared by the
|
||||||
|
client and server:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def encode_binary(chunk: Chunk) -> bytes: ...
|
||||||
|
def decode_binary(data: bytes) -> Chunk: ...
|
||||||
|
def encode_json(chunk: Chunk) -> str: ...
|
||||||
|
def decode_json(text: str) -> Chunk: ...
|
||||||
|
def checksum(payload: bytes) -> str: ...
|
||||||
|
```
|
||||||
|
|
||||||
|
These are the single implementation of the format spec (§3), so the Godot
|
||||||
|
client and Python backend never drift.
|
||||||
|
|
||||||
|
### 5.3 Network RPCs
|
||||||
|
|
||||||
|
The chunk API is exposed to the client via RPCs (matching the networking
|
||||||
|
architecture). All chunk traffic is **reliable** (chunks are discrete,
|
||||||
|
must-arrive data).
|
||||||
|
|
||||||
|
| RPC | Direction | Payload | Notes |
|
||||||
|
|----------------------------|-----------|-------------------------------------------|-------|
|
||||||
|
| `request_chunk` | C → S | `{ key }` | Client asks for a chunk |
|
||||||
|
| `chunk_data` | S → C | `{ key, binary_payload, checksum }` | Server streams a chunk |
|
||||||
|
| `request_chunk_edit` | C → S | `{ key, base_revision, delta }` | Client submits an edit |
|
||||||
|
| `chunk_edit_result` | S → C | `{ key, success, new_revision, error? }` | Acknowledges an edit |
|
||||||
|
| `chunk_invalidated` | S → C | `{ key, new_revision }` | Server pushes a change |
|
||||||
|
| `request_chunk_flags` | C → S | `{ key }` | Age/content gate check |
|
||||||
|
|
||||||
|
### 5.4 Edit & Concurrency Model
|
||||||
|
|
||||||
|
- **Read:** Client requests a chunk; server returns the binary payload and its
|
||||||
|
`revision`.
|
||||||
|
- **Edit:** Client submits a **delta** against a known `base_revision`. The
|
||||||
|
server applies the delta, increments `revision`, persists, and returns the new
|
||||||
|
revision.
|
||||||
|
- **Optimistic concurrency:** If the chunk's current revision differs from the
|
||||||
|
client's `base_revision`, the server rejects the edit with a conflict error
|
||||||
|
and the client re-fetches and re-applies. This prevents lost updates when two
|
||||||
|
clients edit the same chunk.
|
||||||
|
- **Deltas:** A delta is a JSON patch (RFC 6902 style) applied to the chunk's
|
||||||
|
logical model. The server validates the patched chunk (schema, bounds,
|
||||||
|
moderation) before persisting.
|
||||||
|
|
||||||
|
### 5.5 Write-Behind Persistence
|
||||||
|
|
||||||
|
To amortize disk I/O, the server keeps a **dirty set** of chunks modified in
|
||||||
|
memory and flushes them to SQLite in batches:
|
||||||
|
|
||||||
|
1. `put_chunk` updates the in-memory chunk and marks it `dirty`.
|
||||||
|
2. A background flusher periodically calls `flush_dirty()`, which writes all
|
||||||
|
dirty chunks in a single transaction and clears the `dirty` flag.
|
||||||
|
3. On graceful shutdown, `flush_dirty()` is called to ensure durability.
|
||||||
|
|
||||||
|
This keeps the hot path (in-memory chunk access) fast while guaranteeing
|
||||||
|
durability at shutdown and on a configurable interval.
|
||||||
|
|
||||||
|
**Recovery after unclean shutdown:** The `dirty` column in the `chunks` table
|
||||||
|
exists specifically to support recovery if the process crashes before a flush
|
||||||
|
completes. On startup, the server scans for rows where `dirty = 1` and treats
|
||||||
|
them as **unflushed in-memory state that was lost** — it re-reads the last
|
||||||
|
persisted payload from `chunks.payload` (the last durable state) and clears the
|
||||||
|
`dirty` flag. In other words, `dirty = 1` at startup means "the last write was
|
||||||
|
not durable; discard the in-memory delta and fall back to the persisted
|
||||||
|
payload." This guarantees that the on-disk state is always a consistent,
|
||||||
|
previously-committed snapshot, never a partially-written chunk. (The in-memory
|
||||||
|
dirty set is authoritative during normal operation; the `dirty` column is only a
|
||||||
|
recovery marker.)
|
||||||
|
|
||||||
|
### 5.6 Streaming & Caching
|
||||||
|
|
||||||
|
- The server streams only the chunks a client needs (those within the client's
|
||||||
|
view radius), using `list_chunks` with a bounds query.
|
||||||
|
- Chunks are cached in memory (LRU) keyed by `key`; the cache is invalidated on
|
||||||
|
write.
|
||||||
|
- The binary payload is content-addressed by `checksum`, so unchanged chunks can
|
||||||
|
be served from cache without re-reading SQLite.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Format Versioning & Migration
|
||||||
|
|
||||||
|
- The chunk format `version` field (currently `1`) is checked on decode. A
|
||||||
|
decoder that encounters a higher version than it supports **rejects** the
|
||||||
|
chunk with a clear error rather than misreading it.
|
||||||
|
- The SQLite `schema_version` in `meta` drives database migrations (§4.8).
|
||||||
|
- Chunk payloads are migrated **lazily**: when a chunk is read and its
|
||||||
|
`format_version` is older than the current one, the server migrates it to the
|
||||||
|
current version in memory and marks it dirty so it is rewritten on the next
|
||||||
|
flush. This avoids a costly full-world migration at startup.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Security & Integrity
|
||||||
|
|
||||||
|
- **Server-authoritative writes:** Only the `WorldStore` writes to SQLite;
|
||||||
|
clients submit deltas that are validated before application.
|
||||||
|
- **Checksum verification:** Every chunk payload is checksummed (SHA-256) on
|
||||||
|
write and verified on read. Corruption is detected and the chunk is re-fetched
|
||||||
|
or flagged.
|
||||||
|
- **Bounds validation:** Chunk coordinates and object positions are validated
|
||||||
|
against the world's chunk dimensions and room bounds before persistence.
|
||||||
|
- **Moderation enforcement:** `content_flags` are checked at load time; mature
|
||||||
|
chunks are gated behind the age-verification flow (see `content_moderation.md`).
|
||||||
|
- **Input validation:** All string fields are length-limited and validated
|
||||||
|
against allowed character sets to prevent injection and oversized payloads.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Open Questions / Future Work
|
||||||
|
|
||||||
|
- **Volumetric worlds:** The schema and format already support `CHUNK_SIZE_Z >
|
||||||
|
1`, but the tooling and streaming for 3D chunks are not yet specified.
|
||||||
|
- **Chunk compression:** For very large worlds, the binary payload could be
|
||||||
|
compressed (e.g. zstd) before storage/transfer. The header reserves a `flags`
|
||||||
|
bitmask for a compression flag.
|
||||||
|
- **Multi-server sharding:** If a single SQLite file becomes a bottleneck, the
|
||||||
|
`chunks` table could be sharded by `world_id` across multiple databases. The
|
||||||
|
key design already supports this (world_id is a leading key component).
|
||||||
|
- **Delta compression for streaming:** Instead of sending full chunks, the
|
||||||
|
server could send only the delta between revisions to reduce bandwidth for
|
||||||
|
frequently-updated chunks.
|
||||||
Reference in New Issue
Block a user