# Content Moderation & Age Verification Architecture > **Status:** Proposed > **Owner:** Platform team > **Related:** Server authority + client prediction, avatar component system, backend auth (JWT) ## 1. Overview This document specifies the **content moderation** and **age verification** architecture for **agp-game**, a Godot-based multiplayer game built by autonomous agents. It defines: - How **content flags** propagate through the system across the three primary content types: **assets**, **rooms**, and **scripts**. - How flags are **evaluated, aggregated, and enforced** at load time, at runtime, and at the network boundary. - The **age verification flow**, combining a lightweight **self-declaration** with an optional **ID check hook** for higher-assurance verification. The goal is a **single, consistent moderation pipeline** that applies the same rules to user-generated content (UGC) regardless of whether it is a 3D asset, a room definition, or a script, and that gates access to mature content behind a verifiable age gate. ### 1.1 Design Goals - **Uniformity:** One flag vocabulary and one propagation mechanism across assets, rooms, and scripts. - **Composability:** Flags on a room aggregate the flags of the assets and scripts it contains; flags on a world aggregate the flags of its rooms. - **Enforcement at the boundary:** The server is authoritative for what content a client may load or receive, mirroring the server-authoritative networking model. - **Privacy-preserving age checks:** Self-declaration requires no PII; the ID check hook is optional, pluggable, and never stores raw identity documents. - **Auditability:** Every flag decision is logged with a reason, an actor, and a timestamp so moderation actions are reviewable. --- ## 2. Content Flag Vocabulary All content carries a set of **content flags**. A flag is a typed, immutable label with a **severity** and a **scope**. Flags are the atomic unit of moderation; everything else (aggregation, enforcement, gating) operates on flags. ### 2.1 Flag Definition A flag is a structured record: | Field | Type | Description | |------------|-----------|--------------------------------------------------------------------------| | `id` | `string` | Stable machine-readable identifier, e.g. `violence.gore`. | | `label` | `string` | Human-readable name, e.g. "Graphic Gore". | | `category` | `enum` | One of `violence`, `sexual`, `language`, `substance`, `gambling`, `horror`, `hate`, `custom`. | | `severity` | `enum` | One of `none`, `mild`, `moderate`, `severe`. | | `scope` | `enum` | One of `visual`, `audio`, `text`, `behavior`, `interaction`. | | `mature` | `bool` | Whether this flag triggers the age gate (see §5). | | `reason` | `string` | Free-text justification supplied by the flagger. | | `source` | `enum` | One of `author`, `moderator`, `automated`, `community`. | ### 2.2 Canonical Flag Set The platform defines a **canonical registry** of flags so that different content types and different flaggers use the same vocabulary. Examples: | id | category | severity | scope | mature | |-----------------------------|------------|-----------|--------------|--------| | `violence.none` | violence | none | visual | false | | `violence.mild` | violence | mild | visual | false | | `violence.gore` | violence | severe | visual | true | | `sexual.suggestive` | sexual | mild | visual | true | | `sexual.explicit` | sexual | severe | visual | true | | `language.mild` | language | mild | text | false | | `language.profanity` | language | moderate | text | false | | `substance.references` | substance | mild | text | false | | `gambling.simulated` | gambling | moderate | behavior | true | | `horror.jumpscare` | horror | mild | audio/visual | false | | `hate.hateful` | hate | severe | text | true | The registry is **data-driven** (a JSON/YAML manifest shipped with the server) so new flags can be added without code changes. --- ## 3. Flag Propagation Model Flags propagate through the system along the **content dependency graph**. The three content types form a hierarchy: ``` World └── Room ├── Asset (mesh, texture, audio, animation, material) └── Script (behavior, dialogue, event logic) ``` ### 3.1 Propagation Rules 1. **Explicit flags** are set directly on a content item by its author or a moderator. 2. **Inherited flags** flow upward: a room's effective flag set is the **union** of its own explicit flags and the flags of every asset and script it references. 3. **Aggregation** uses the **maximum severity** per category across all contributing items. A room containing one `violence.gore` asset and one `violence.none` asset is `violence.severe`. 4. **Maturity is monotonic:** if any contributing item is `mature`, the containing room and world are `mature`. 5. **Cycles are impossible** by construction: content references form a DAG (an asset cannot contain a room; a room cannot contain itself). ### 3.2 Propagation Through the Pipeline Flags are attached at **authoring time** and carried through every stage: ``` Authoring ──► Upload ──► Validation ──► Storage ──► Distribution ──► Runtime │ │ │ │ │ │ │ flag │ flag │ automated │ flag │ flag │ flag │ declared │ preserved │ scan adds │ indexed │ re-checked │ enforced │ │ │ flags │ │ │ ``` | Stage | Responsibility | |--------------|-------------------------------------------------------------------------| | **Authoring**| Author declares flags in the editor/upload UI. | | **Upload** | Flags are validated against the canonical registry; unknown flags rejected. | | **Validation**| Automated scanner (hash-based + heuristic) adds flags for detected content. | | **Storage** | Flags are stored alongside the content record and indexed for queries. | | **Distribution**| Flags are re-checked against the current registry before serving. | | **Runtime** | The client enforces flags before loading content (see §4). | ### 3.3 Data Model Flags are stored as a **normalized join table** so they can be queried and aggregated efficiently: ``` content_item (id, type, owner_id, ...) content_flag (content_id, flag_id, source, reason, created_at, actor_id) flag_registry (id, category, severity, scope, mature, ...) ``` Aggregation is computed by a SQL view / query that joins a content item's transitive dependencies. Because the graph is a DAG, this is a bounded recursive query (PostgreSQL `WITH RECURSIVE`). --- ## 4. Enforcement Enforcement happens at three layers, in order: ### 4.1 Load-Time Enforcement (Server) Before a client is allowed to **request** content, the server evaluates the content's effective flag set against the requesting account's **content preferences** and **age verification status**: - If the content is `mature` and the account is **not age-verified**, the request is **rejected** with a `403 MATURE_CONTENT_REQUIRES_VERIFICATION`. - If the content's flags exceed the account's **content filter** (a per-account severity threshold per category), the request is rejected or the offending content is **stripped** (e.g., audio muted, mesh hidden, script not loaded). ### 4.2 Runtime Enforcement (Client) The client receives a **manifest** of allowed content from the server. It never loads content that is not in the manifest. This prevents a malicious or buggy client from bypassing the server's filter by loading content directly. - Assets not in the manifest are **not instantiated**. - Scripts not in the manifest are **not executed**. - Rooms not in the manifest are **not entered**. ### 4.3 Network Boundary Enforcement The server **filters replication** based on flags: - Mature or filtered content is **not replicated** to clients that cannot receive it. - A room's flag set is broadcast in the room metadata so clients can show a warning before entry. ### 4.4 Enforcement Decision Flow ``` Client requests content C for account A │ ▼ Compute effective flags F(C) [union + max-severity aggregation] │ ▼ Is A age-verified? ──no──► Is F(C).mature? ──yes──► REJECT (403) │ yes │ no ▼ ▼ Does F(C) exceed A's filter? ──► yes ──► REJECT or STRIP │ no ▼ Serve content + manifest ``` --- ## 5. Age Verification Flow Age verification is a **two-tier** system: - **Tier 1 — Self-declaration:** The user asserts their date of birth. This is fast, requires no PII beyond a birthdate, and is sufficient for most content. - **Tier 2 — ID check (optional hook):** A pluggable identity-verification provider confirms the user is an adult. This raises the account's assurance level and unlocks the highest maturity tier. ### 5.1 Verification States An account has an **age verification status**: | Status | Meaning | |-------------------|----------------------------------------------------------------------| | `unverified` | No declaration on file. | | `self_declared` | User declared a birthdate; **not** independently confirmed. | | `id_verified` | User passed the ID check hook; independently confirmed adult. | | `rejected` | ID check failed or declaration was inconsistent. | ### 5.2 Tier 1 — Self-Declaration Flow > **Implemented, PC3 Phase 3.** `POST /api/auth/register` now takes a real > `date_of_birth` (ISO `YYYY-MM-DD`), not the throwaway `over_18` boolean it > had before. The server computes age (`auth.calculate_age`) and rejects > anyone under `auth.MIN_AGE` (18) with a **403** — this platform is > adults-only, so there is no "minor, restricted" tier here (the 13+ rows > below are the general design, not the current gate). A future-in-the-past > or nonsensical date is a **400**. The DOB is stored on the user row and > surfaced only back to that user (`/api/auth/me`), never to other players. > > **Limitation, stated plainly:** this is still self-attested — a determined > user can enter a false DOB. It is a concrete, stored, legally-meaningful > data point (matching how most platforms gate before adding hard checks) > and the seam for Tier 2. Encryption-at-rest for the DOB column and the > Tier 2 ID-check hook below are **not yet implemented**. ``` User submits { date_of_birth } │ ▼ Server validates format & plausibility (real date, not in future) │ ▼ Compute age from DOB │ ▼ age >= 18 ──► status = self_declared (adult) ← the current gate age >= 13 ──► status = self_declared (minor, restricted) ← design only age < 13 ──► status = rejected (below minimum platform age) ``` - The DOB is stored **encrypted at rest** and is **never exposed** to other players. - A user may **upgrade** from `self_declared` to `id_verified` at any time via Tier 2. ### 5.3 Tier 2 — ID Check Hook The ID check is an **optional, pluggable hook**. The platform defines an interface; concrete providers (e.g., a third-party KYC service) implement it. ``` User requests ID verification │ ▼ Server creates a verification session (idempotent, expiring) │ ▼ Server calls provider hook: verify(identity_token) -> { adult: bool, ref: string } │ ▼ adult == true ──► status = id_verified, store provider ref (not raw doc) adult == false ─► status = rejected provider error ──► status unchanged, retry allowed ``` **Privacy guarantees:** - The platform **never stores** the raw identity document or image. - Only the provider's **opaque reference** and the boolean `adult` result are persisted. - The hook is **asynchronous and idempotent**; a session can be retried without double-charging or double-verifying. ### 5.4 Hook Interface The ID check hook is defined as a backend interface (FastAPI dependency / abstract service): ```python class IdentityVerificationProvider(Protocol): def create_session(self, user_id: str) -> VerificationSession: ... def verify(self, session_id: str, identity_token: str) -> VerificationResult: ... class VerificationResult: adult: bool provider_ref: str | None error: str | None ``` A default **no-op provider** returns `adult=False` with a clear error, so the system runs without any third-party integration until one is configured. ### 5.5 Age Gate Enforcement The age gate is enforced at **load time** (see §4.1): | Verification status | Can access `mature` content? | |---------------------|------------------------------| | `unverified` | No | | `self_declared` | Yes (adult) / No (minor) | | `id_verified` | Yes | | `rejected` | No | --- ## 6. Moderation Workflow ### 6.1 Flagging Sources Content can be flagged by: - **Author** (self-declared at upload). - **Automated scanner** (hash-based detection, heuristic analysis). - **Community** (players report content; reports become flags after review). - **Moderator** (manual review and override). ### 6.2 Review Queue Community and automated flags enter a **review queue**. A moderator either: - **Approves** the flag (it becomes authoritative), or - **Rejects** the flag (it is discarded), or - **Escalates** to a senior moderator for severe/hate flags. ### 6.3 Actions Once a flag is authoritative, a moderator can take actions: | Action | Effect | |-----------------|---------------------------------------------------------------| | `warn` | Notify the author; content remains. | | `restrict` | Content hidden from users below a filter threshold. | | `age_gate` | Content marked `mature`; requires age verification. | | `remove` | Content taken down; not served. | | `ban_author` | Author's upload privileges suspended. | ### 6.4 Audit Log Every flag and action is written to an **audit log**: ``` audit_event (id, content_id, actor_id, action, flag_id, reason, created_at) ``` This log is append-only and used for transparency and dispute resolution. --- ## 7. Client-Side Integration (Godot) ### 7.1 Content Manifest The server sends each client a **content manifest** — the set of content the client is permitted to load, derived from the account's verification status and filter. The Godot client: - Loads only manifest-listed assets. - Executes only manifest-listed scripts. - Shows a **maturity warning** dialog before entering a room whose effective flags include `mature` content. ### 7.2 Flag Metadata on Content Each content resource carries a `content_flags` metadata field (a Godot `Dictionary` or custom `Resource`). The client uses this for local UI decisions (e.g., showing a warning icon) but **never** as the authority for whether to load — the server manifest is authoritative. ### 7.3 Age Verification UI The client exposes an **age verification screen**: - Tier 1: a date-of-birth picker. - Tier 2: a button that opens the ID check provider flow (deep link / webview). The client calls the backend endpoints (§8) and reflects the resulting status. --- ## 8. Backend API Surface | Method | Path | Description | |--------|-----------------------------------|------------------------------------------------| | `POST` | `/v1/age/declare` | Submit self-declared DOB (Tier 1). | | `POST` | `/v1/age/verify/session` | Create an ID-check session (Tier 2). | | `POST` | `/v1/age/verify/confirm` | Confirm an ID-check session with a token. | | `GET` | `/v1/age/status` | Return the account's verification status. | | `GET` | `/v1/content/{id}/flags` | Return a content item's effective flags. | | `POST` | `/v1/content/{id}/flags` | Add a flag (author/community). | | `POST` | `/v1/moderation/review/{flag_id}` | Moderator approve/reject/escalate. | | `POST` | `/v1/moderation/action` | Apply a moderation action. | | `GET` | `/v1/manifest` | Return the client's permitted content manifest.| All endpoints require a valid **JWT** (matching the existing auth stack). The age endpoints additionally require the account to be in a non-`rejected` state. --- ## 9. Security & Privacy Considerations - **DOB encryption:** Self-declared birthdates are encrypted at rest; only the derived age and verification status are used in authorization decisions. - **No raw identity storage:** The ID check hook persists only a provider reference and a boolean result. - **Server authority:** The client manifest prevents bypass; the client never trusts its own flag metadata for access decisions. - **Rate limiting:** Age-declaration and ID-check endpoints are rate-limited to prevent abuse and enumeration. - **Auditability:** All flag and verification events are logged append-only. --- ## 10. Open Questions / Future Work - **Regional age thresholds:** Different jurisdictions may require different minimum ages; the threshold should be configurable per region. - **Parental controls:** A future feature may let a parent set a content filter on a minor's account. - **Automated scanner accuracy:** The heuristic scanner may produce false positives; a confidence score should be attached to automated flags. - **Offline / local content:** How flags apply to locally-authored content that is never uploaded remains to be defined.