Files
modbison/docs/architecture/world_persistence_impl.md
daddy b562e9f7b3 Design: server-side world persistence architecture
Decides where chunk/object content lives at rest and who may write it, now
that there are two server-side processes per world (the always-on FastAPI
backend and the per-join, reaper-killed Godot dedicated server).

Decisions: chunk data lives in the existing Postgres DB (not world.md's
per-world SQLite file); the FastAPI backend is the sole writer and the editor
saves to it over HTTP; the dedicated server pulls content read-only at startup
over a loopback /api/internal route with a per-spawn token; full-chunk PUT with
base_revision instead of JSON-Patch deltas; the client streams chunks over HTTP
through the existing ChunkStreamManager.set_server_rpc() seam.

Adds docs/architecture/world_persistence_impl.md and marks the superseded
sections of world.md (SS4, SS5) and editor.md (the request_chunk_edit path).
Design only -- no implementation code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y2FESCaMXeJpC9gSctiEeD
2026-09-14 19:19:10 +00:00

34 KiB
Raw Permalink Blame History

World Persistence: Server-Side Implementation Architecture

Status: Accepted (design). Implementation is split across two follow-on tasks (backend endpoints; editor wiring). Owner: Platform team Supersedes: the storage engine and write ownership parts of world.md §4 and §5. Everything else in world.md — the chunk logical model, the JSON and binary encodings, checksums, revisions, content flags — stands unchanged and is normative. Related: world.md (chunk format), editor.md (authoring surface), networking.md (ENet sim transport), content_moderation.md (flags & gating).

1. Why this document exists

world.md was written during the client-side chunk-streaming work (M2). It assumes one server that owns one embedded SQLite file and is both the authoritative simulation and the authoritative store. That is not the system we built. What actually exists today is:

Process Lifetime Owns Code
FastAPI backend always on, single worker Postgres (users, worlds, assets, avatar loadouts), game-server orchestration backend/app/
Godot dedicated server, one per world spawned on join, reaped ~5 min after the last join when empty nothing durable; a 30 Hz in-memory player sim scripts/server.gd, spawned by backend/app/services/game_servers.py
Godot client player session local chunk streaming, the in-game editor (planned) scripts/chunk_stream_manager.gd, scripts/chunk_data_decoder.gd

Nothing server-side has ever stored or served a real chunk. server.gd preload()s res://scenes/worlds/test_level.tscn for every world and uses world_id only for logging and the status file. ChunkStreamManager has a set_server_rpc() seam that no code calls. The backend's World model is metadata only (id, name, description, owner_id, visibility).

This document decides where chunk content lives, who may write it, how the dedicated server gets it, and the exact API surface — so the two implementation tasks that follow have no architectural decisions left to make.


2. Decisions at a glance

# Question Decision
D1 Where does chunk data live at rest? Postgres, in the existing backend database. Not a per-world SQLite file.
D2 Who writes it? The FastAPI backend, exclusively. The editor writes through it over HTTP with optimistic concurrency. The dedicated Godot server never writes world content.
D3 How does the dedicated server get content? HTTP pull from the backend at startup, over a loopback-only /api/internal route authenticated by a per-spawn token the supervisor mints. Lazy fetch as players roam.
D4 Edit while a live session exists? Out of scope this milestone, with a defined seam. The editor is a non-joined client mode; writes always succeed and the response flags live_session so the UI can say "players see this when the session restarts".
D5 Edit granularity on the wire Full-chunk PUT with base_revision, not RFC-6902 deltas.
D6 Client chunk transport HTTP from the backend, not ENet from the dedicated server. The set_server_rpc() seam is satisfied by an HTTP-backed adapter.

3. D1 — Chunk data lives in Postgres

3.1 Decision

World content (rooms, chunk payloads, chunk revisions) is stored in the existing Postgres database, alongside worlds, in new tables created by an Alembic migration. world.md §4's world.db SQLite file is not built.

The chunk payload stays exactly what world.md §3.6 specifies: the 48-byte header + sectioned binary body, checksummed with SHA-256. It moves from a SQLite BLOB to a Postgres BYTEA. Nothing about the format changes, so scripts/chunk_data_decoder.gd remains correct and the format spec stays the single contract.

3.2 Reasoning

  1. SQLite's premise does not hold here. world.md §4 justifies SQLite as "ideal for a server that owns the world store". No process in this system owns a world store: the only process that could plausibly hold the file (the dedicated Godot server) is ephemeral — reaped when idle, SIGTERM'd on backend shutdown, SIGKILL'd 10 s later if it doesn't exit, and orphaned entirely if the backend dies. Meanwhile the editor must write when no such process exists. So the file would have to be shared by two processes with independent lifetimes, which is precisely the workload SQLite's single-writer/file-lock model handles worst.
  2. Postgres is already a hard dependency; SQLite would be a second one. The backend cannot start without Postgres. Adding a per-world SQLite file adds a second storage engine, a second migration system (world.md §4.8 specifies a hand-rolled meta.schema_version runner next to the Alembic we already have), a second backup story, and a second consistency story.
  3. Referential integrity with metadata we already have. worlds rows carry owner_id and visibility, and every authorization rule for content derives from them. In one database this is a join and a foreign key with ON DELETE CASCADE; across two engines, deleting a world silently orphans a file on disk and every ownership check becomes a cross-store read.
  4. Concurrency is the normal case, not the edge case. Two creators editing one world, or an editor writing while the backend serves chunks to a running dedicated server, are ordinary. Postgres gives row-level MVCC and real transactions for the read-modify-write in §5.4; SQLite gives a database-level write lock and SQLITE_BUSY.
  5. The repo already has the precedent and the plumbing. avatar_loadouts (migration 0004) stores a client-shaped document in Postgres JSONB rather than inventing a side store, and backend/app/database.py already provides the async session factory this code will use.

3.3 What we give up, and why it's acceptable

  • Zero-config embedding. Irrelevant — the backend already requires a configured Postgres.
  • world.md §5.5 write-behind + the dirty column. Deliberately dropped (§4.4). It existed to amortize disk I/O for a process that owned the file. Our writes are human-rate editor saves, not sim-rate; each one is a single transaction and is durable when the HTTP response returns. Dropping it also deletes the entire "unclean shutdown recovery" scan in §5.5 — there is no window in which committed content can be lost.
  • Sharding by world_id (world.md §8). Postgres partitioning by world_id remains available on the same table shape if a single table ever becomes a bottleneck. At ~2–8 KB per chunk, 10 000 chunks is under 100 MB.

4. Schema (Alembic migration 0005_add_world_content_tables)

New model module: backend/app/models/world_content.py, registered in backend/app/models/__init__.py. backend/app/models/world.py is left alone except for one added relationship, so nothing PC1/PC2 depends on moves.

Conventions follow the existing models: Column(...) declarative style, Integer surrogate PKs, DateTime(timezone=True) with server_default=func.now(), JSONB for document-shaped fields (as avatar_loadout.py does).

4.1 world_rooms

class WorldRoom(Base):
    __tablename__ = "world_rooms"

    id         = Column(Integer, primary_key=True, index=True)
    world_id   = Column(Integer, ForeignKey("worlds.id", ondelete="CASCADE"),
                        nullable=False, index=True)
    slug       = Column(String(64), nullable=False)   # url-safe; "main" by default
    name       = Column(String(255), nullable=False)
    # Chunk grid config. world.md §2.2 puts this on the world; it lives on the
    # room so a world can later hold rooms with different grids, and so this
    # migration is purely additive (the existing `worlds` table is untouched).
    grid       = Column(JSONB, nullable=False,
                        server_default='{"size": [32, 32, 1], "origin": [0, 0, 0]}')
    bounds     = Column(JSONB, nullable=True)   # {"min":[x,y,z], "max":[x,y,z]}; null = unbounded
    spawn      = Column(JSONB, nullable=True)   # {"pos":[x,y,z], "yaw": 0.0}
    created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
    updated_at = Column(DateTime(timezone=True), server_default=func.now(),
                        onupdate=func.now(), nullable=False)

    __table_args__ = (UniqueConstraint("world_id", "slug", name="uq_world_rooms_world_slug"),)

World gains rooms = relationship("WorldRoom", back_populates="world", cascade="all, delete-orphan").

Every world gets a main room: created in create_world, and lazily created on first room read/write for worlds that predate the migration. grid defaults match Chunk.CHUNK_SIZE_* in scripts/chunk.gd, which is the client's source of truth for the same constants.

4.2 world_chunks

class WorldChunk(Base):
    __tablename__ = "world_chunks"

    id             = Column(Integer, primary_key=True, index=True)
    world_id       = Column(Integer, ForeignKey("worlds.id", ondelete="CASCADE"),
                            nullable=False, index=True)   # denormalized: world-scoped queries
    room_id        = Column(Integer, ForeignKey("world_rooms.id", ondelete="CASCADE"),
                            nullable=False, index=True)
    chunk_key      = Column(String(160), nullable=False, unique=True)   # see §4.5
    cx             = Column(Integer, nullable=False)
    cy             = Column(Integer, nullable=False)
    cz             = Column(Integer, nullable=False, server_default="0")

    format_version = Column(Integer, nullable=False, server_default="1")
    payload        = Column(LargeBinary, nullable=False)   # world.md §3.6 bytes, BYTEA
    payload_len    = Column(Integer, nullable=False)
    checksum       = Column(String(64), nullable=False)    # SHA-256 hex of payload body
    revision       = Column(Integer, nullable=False, server_default="1")

    content_flags  = Column(JSONB, nullable=False, server_default="{}")  # world.md §3.4
    is_mature      = Column(Boolean, nullable=False, server_default="false")

    updated_by_id  = Column(Integer, ForeignKey("users.id", ondelete="SET NULL"), nullable=True)
    created_at     = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
    updated_at     = Column(DateTime(timezone=True), server_default=func.now(),
                            onupdate=func.now(), nullable=False)

    __table_args__ = (
        UniqueConstraint("room_id", "cx", "cy", "cz", name="uq_world_chunks_coord"),
        Index("ix_world_chunks_room_coord", "room_id", "cx", "cy", "cz"),
        Index("ix_world_chunks_mature", "world_id", postgresql_where=text("is_mature")),
    )

Deltas from world.md §4.4:

  • Surrogate id PK + unique coordinate constraint, with chunk_key kept as a unique denormalized column. The key string is the client-facing contract (Chunk.build_key) and is embedded in the payload, so storing it gives a single-index lookup by key; the coordinate constraint is what actually enforces one chunk per cell.
  • dirty removed — see §3.3.
  • Content flags folded into the chunk row as JSONB + an indexed is_mature boolean, instead of world.md §4.6's separate 11-column content_flags table. The flags are a document the client already produces in exactly this shape (ChunkDataDecoder.ContentFlags.to_dict()), the only query we anticipate is "is this mature", and folding them in removes a second write per save and the object_id = '' sentinel hack that §4.6 needed to work around SQLite's NULL-in-primary-key behaviour (Postgres has the same NULL semantics, so the hack would have carried over).
  • world.md §4.5 chunk_objects is deferred, not dropped. It is a derived index that exists to answer queries ("which chunks contain a door?") that no code asks yet, and maintaining it means a delete-and-reinsert of every object row on every save. It is fully rebuildable from payload, so it can be added in its own migration the day a query needs it, with a backfill. Do not build it in 0005.

4.3 world_chunk_revisions

class WorldChunkRevision(Base):
    __tablename__ = "world_chunk_revisions"

    id         = Column(Integer, primary_key=True, index=True)
    chunk_key  = Column(String(160), nullable=False, index=True)  # deliberately NOT a FK
    world_id   = Column(Integer, nullable=False, index=True)      # also not a FK, same reason
    revision   = Column(Integer, nullable=False)
    action     = Column(String(16), nullable=False)               # 'create' | 'update' | 'delete'
    actor_id   = Column(Integer, ForeignKey("users.id", ondelete="SET NULL"), nullable=True)
    checksum   = Column(String(64), nullable=False)
    payload    = Column(LargeBinary, nullable=True)               # full payload at this revision
    created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)

Kept from world.md §4.7 including its two design points: the full payload is stored (rollback needs it) and chunk_key/world_id are not foreign keys, so deleting a chunk or a world preserves the audit trail.

Retention (new, required): this table grows without bound — a creator dragging a brush can produce hundreds of revisions per minute. The implementation must ship a retention policy from day one: keep the newest WORLD_CHUNK_REVISION_KEEP (default 20) payloads per chunk_key and null out payload on older rows (keeping the metadata row for the audit trail), pruned in the same transaction as the write. A created_at index supports a later time-based prune.

4.4 Config additions (backend/app/config.py)

WORLD_CHUNK_MAX_PAYLOAD_BYTES: int = 1_048_576   # 1 MiB; a dense 32×32 uint16 layer is 2 KiB
WORLD_CHUNK_MAX_OBJECTS: int = 512
WORLD_CHUNK_MAX_BATCH: int = 64
WORLD_CHUNK_REVISION_KEEP: int = 20
GAME_SERVER_INTERNAL_API_URL: str = "http://127.0.0.1:8000"

4.5 Chunk key format

world.md §2.3 specifies "{world_id}:{room_id}:{cx}:{cy}:{cz}" and illustrates it with w_0001:r_0001. Those ids were illustrative. The real key uses the backend's identifiers:

"{world.id}:{room.slug}:{cx}:{cy}:{cz}"      e.g.  "12:main:3:-1:0"

The room slug rather than its numeric id, so a key is human-readable in logs and stable if rooms are ever re-imported. Chunk.build_key() in scripts/chunk.gd already produces this shape unmodified.


5. D2 — The FastAPI backend is the only writer

5.1 Decision

The in-game editor (editor.md §7.2) submits edits over HTTP to the FastAPI backend, which validates, versions, and persists them. The dedicated Godot server process never writes world content and does not sit in the edit path.

This preserves world.md's and editor.md's actual invariant — clients never touch storage; they submit edits to a server-authoritative tier that validates them — while identifying that tier as the always-on backend rather than the ephemeral sim process. editor.md §4.2's claim that edits ride the same request_chunk_edit RPC as gameplay is the one part of that doc this design changes; the moderation and validation properties it derives from that claim are unaffected, because the validation boundary moves with the writer.

5.2 Reasoning

  1. Nothing that can vanish in 300 seconds may own the write path. The reaper in game_servers.py stop()s an idle server with SIGTERM and escalates to SIGKILL after 10 s; supervisor.shutdown() does the same to every server on backend shutdown; and if the backend dies uncleanly the servers are orphaned with no one to flush them. Because the sim process holds no unsaved world content in this design, no reap, crash, or kill can ever lose world data — that is the single strongest practical reason for this split.
  2. Editing must work with no dedicated server running. A creator opening their world to build should not have to spin up a 30 Hz sim process, burn a port from the 100-port range, and hold it for 5 minutes past their last save.
  3. server.gd has no database access, no credentials, and no HTTP client. Routing writes through it means either shipping Postgres credentials into a process spawned per join, or building an ENet→HTTP proxy inside the sim loop. Both are strictly worse than the editor calling the backend directly.
  4. Authorization already lives in the backend. CurrentUser, the owner_id check, and WorldVisibility are all there. The dedicated server has no user identities at all — it knows peer ids, not accounts.

5.3 Authorization

  • Read a chunk: the same rule as GET /api/worlds/{id} — the world is PUBLIC, or the caller owns it. Otherwise 404 (not 403), matching the existing routers' deliberate non-disclosure.
  • Write a chunk: caller must be world.owner_id, matching update_world; 403 otherwise. Collaborator/role-based editing is future work and slots in at this one check.
  • Mature gating hook: is_mature chunks are where content_moderation.md's age gate attaches on the read path. Specify the column and the index now; the gate itself lands with that doc's work.

5.4 Concurrency

Optimistic concurrency exactly as world.md §5.4, expressed over HTTP:

  1. Reader gets revision (header X-Chunk-Revision, or the JSON field).
  2. Writer sends base_revision.
  3. The service does SELECT ... FOR UPDATE on the chunk row, compares base_revision to revision, and either commits at revision + 1 or returns 409 with current_revision and current_checksum so the editor can re-fetch and re-apply.
  4. base_revision: 0 means "create; fail with 409 if it already exists".

5.5 Validation (the backend must not trust the client)

The write path decodes the submitted payload server-side before storing it. This requires a Python port of the world.md §3.6 codec — backend/app/services/chunk_codec.py, mirroring scripts/chunk_data_decoder.gd. It checks: magic AGPC, format_version <= 1, declared vs. actual payload_len, recomputed SHA-256 (the header's checksum is advisory), tile layer dims equal the room's grid.size, tile indices in palette range, object count ≤ WORLD_CHUNK_MAX_OBJECTS, object positions inside the chunk's bounds, chunk_key in the payload matching the URL, and string-field length limits (world.md §7). content_flags and is_mature are recomputed from the payload's flag sections, never taken from the request body.

Drift control (required): the two codecs must be pinned by golden files — fixture chunks under tests/fixtures/chunks/ encoded by one implementation and decoded byte-identically by the other, asserted in both the pytest suite and the Godot headless suite. Without this the format has two implementations and no contract test, which is exactly the failure world.md §5.2 was trying to avoid by having one shared implementation.


6. D4 — Editing versus a live session (scope boundary)

In scope for this milestone:

  • The editor is entered as "Edit world" from the world browser: a non-joined client mode. It does not call POST /api/worlds/{id}/join, does not open an ENet connection, and does not spawn a dedicated server. It loads chunks over HTTP, edits them locally, and saves them over HTTP.
  • Writes always succeed regardless of whether a session is live. The write response carries live_session: bool (from supervisor.debug_state()), so the editor can surface "this world has players in it right now — they'll see your changes when the session restarts."
  • A dedicated server started after a write sees the new content, because it loads from the store at startup (§7).

Explicitly out of scope, deferred:

  • Build-in-place while joined to a live session.
  • chunk_invalidated push to connected clients (world.md §5.3) and hot-reload of the dedicated server's collision world.
  • Live collaborative editing / cursors (editor.md §9).

The seam that makes the deferred work cheap: the manifest endpoint (§7.2) returns a monotonic content_revision for the world, bumped on every chunk write. Hot reload is then: the dedicated server polls the manifest on a timer (or the backend pushes to it over the same internal channel), re-fetches the chunks whose revision changed, and rebuilds those collision bodies. Nothing in this design has to change to add it.

Why the editor must not join: if it joined, the sim process would enforce collision against content it loaded at spawn while the creator edits a diverging local copy — players would walk through new walls and bounce off deleted ones, with no error anywhere. Making build mode non-joined turns a silent correctness bug into an explicit product boundary.


7. D3 — What the dedicated server does at startup

7.1 Decision

server.gd pulls its world's content from the backend over HTTP at startup, authenticated by a per-spawn token the supervisor mints, and does not bind the ENet port until content is loaded.

Rejected alternative — supervisor writes a whole-world snapshot file at spawn and passes --world-file: attractive (no new auth surface, no HTTP client in the sim process), but it makes spawn latency proportional to world size, gives the server no way to fetch a chunk lazily as players roam beyond the preloaded set, and still needs a separate channel for the hot-reload work above. Rejected — give the Godot process a Postgres connection: there is no PG driver in GDScript, and it would put database credentials in a process spawned per join.

7.2 Startup sequence

  1. _parse_args() gains --api-url and --server-token (§7.3).
  2. GET /api/internal/worlds/{world_id}/manifest → rooms, per-room grid, bounds, spawn, chunk_count, and the world's content_revision.
  3. GET /api/internal/worlds/{world_id}/rooms/{slug}/chunks? bounds query for the spawn neighbourhood (radius from the spawn point, default 3 chunks), then fetch each payload; decode with the existing ChunkDataDecoder and build static collision from the tile layers and objects.
  4. Then network_peer.create_server(port), then write the status file.
  5. Fallback: if the backend is unreachable, the world has no rooms, or the room has zero chunks, load test_level.tscn as today and record "content": "fallback" in the status file. A content outage must never make a world un-joinable, and this keeps every PC3 flow working before any world has authored content.

During the session the server may lazily fetch chunks as players approach unloaded coordinates, through the same internal route. It never writes. Dynamic entity runtime state (positions, health) stays in memory and out of chunks, exactly as world.md §3.3 already specifies. Shutdown flushes nothing.

7.3 Internal authentication

GameServerSupervisor._spawn() generates token = secrets.token_urlsafe(32), stores it on the _WorldServer dataclass, and passes it on the command line alongside the existing --port/--world/--status-file. A FastAPI dependency require_game_server resolves Authorization: Bearer <token> against supervisor.validate_token(world_id, token) — a dict lookup, since the supervisor is an in-process singleton. Properties this gives us:

  • A token grants access to one world and dies with its process (the reaper's del self._servers[wid] invalidates it).
  • No long-lived shared secret in config or on disk.
  • It reuses the already documented single-worker constraint in game_servers.py; it does not add a new one.

Rejected alternative — let the dedicated server use the public routes with a service user account: the public routes enforce per-user visibility and ownership, which a service identity would have to be exempted from, and it would pollute updated_by_id/audit rows with a fake user. A separate /api/internal prefix keeps the two auth models apart and can be blocked at the reverse proxy.

7.4 Lifecycle changes this requires in game_servers.py

These are real, load-bearing findings from the current implementation:

  • _await_port() is the wrong readiness signal now. It returns as soon as the UDP port is bound, and the port is bound before content exists today. With content loading it must await a "ready": true field that server.gd writes to the status file after content load + create_server. Otherwise the backend hands a client an address for a server that is still fetching chunks.
  • _await_port() fails open. After its 8 s deadline it returns normally, with no error — ensure_server then reports success for a server that may never have started. Content loading makes a slow start more likely, so the readiness wait needs a real timeout error (503 to the joining client) and a longer, configurable deadline.
  • The status file is the only server→backend channel, and _spawn currently passes no credentials at all — hence §7.3.
  • player_count() returns 0 when the status file is missing or unparseable, so a live server whose status write fails can be reaped after the idle timeout. Harmless for data (the sim owns nothing durable — the point of D2) but worth a comment where the fallback lives.
  • No crash restart (documented in the module docstring): if a server dies, the next join respawns it and it reloads content from the store. With D2 this is now genuinely safe rather than merely tolerated.

8. API surface

All paths are under the existing worlds router prefix (/api/worlds) unless noted. New router module: backend/app/routers/world_content.py, mounted at /api/worlds; internal routes in backend/app/routers/internal.py at /api/internal.

8.1 Public (editor & player client; JWT CurrentUser)

Method Path Purpose
GET /api/worlds/{world_id}/rooms List rooms + grid config
POST /api/worlds/{world_id}/rooms Create a room (owner)
GET /api/worlds/{world_id}/rooms/{slug}/chunks Chunk index over a bounds query
GET /api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz} Fetch one chunk
PUT /api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz} Write one chunk
PUT /api/worlds/{world_id}/rooms/{slug}/chunks Write up to WORLD_CHUNK_MAX_BATCH chunks atomically
DELETE /api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz} Delete one chunk
GET /api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz}/revisions Revision history (owner)
POST /api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz}/rollback Restore a revision (owner)

Room list — GET .../rooms → 200

[{ "id": 7, "slug": "main", "name": "Main",
   "grid": { "size": [32, 32, 1], "origin": [0, 0, 0] },
   "bounds": null,
   "spawn": { "pos": [0, 1.5, 0], "yaw": 0.0 },
   "chunk_count": 41 }]

Chunk index — GET .../chunks?cx_min=&cx_max=&cy_min=&cy_max=&cz=0 → 200. Payloads are not included; this is world.md §5.1's list_chunks(bounds) and lets a streamer decide what to fetch and skip anything already cached by checksum.

{ "room": "main", "content_revision": 118,
  "chunks": [{ "key": "12:main:0:0:0", "cx": 0, "cy": 0, "cz": 0,
               "revision": 4, "checksum": "9f86…", "payload_len": 2344,
               "is_mature": false, "updated_at": "2026-09-14T10:02:11Z" }] }

Bounds are capped (default 32×32 chunks per query) and return 422 beyond that.

Chunk fetch — GET .../chunks/{cx}/{cy}/{cz}, content-negotiated:

  • Accept: application/octet-stream (default for the game client) → 200 with the raw world.md §3.6 bytes, ETag: "<checksum>", X-Chunk-Revision: <n>, X-Chunk-Key: <key>.
  • Accept: application/json (tooling, tests, the editor's debug view) → the §3.5 JSON encoding, decoded server-side from the payload. JSON is derived, never stored, so there is no dual-write drift (world.md §4.4).
  • If-None-Match: "<checksum>" → 304. This is what makes the client's user://chunk_cache/ cache cheap and is why the checksum is the ETag.
  • 404 if the chunk does not exist (an unwritten coordinate is not an error to the streamer — it renders empty).

Chunk write — PUT .../chunks/{cx}/{cy}/{cz}

Content-Type: application/octet-stream
X-Base-Revision: 4          # 0 = create
<binary payload>

or, for clients that prefer JSON: {"base_revision": 4, "payload_b64": "…"}.

Responses:

Code Body / meaning
200 {"key", "revision", "checksum", "payload_len", "updated_at", "is_mature", "live_session"}
409 {"detail": "revision conflict", "current_revision": 6, "current_checksum": "…"}
403 not the world owner
404 world or room not found / not visible
413 payload over WORLD_CHUNK_MAX_PAYLOAD_BYTES
422 decode/validation failure ({"detail", "reason"}, e.g. checksum_mismatch, layer_dims, object_out_of_bounds)

Batch write — PUT .../chunks with {"chunks": [{"cx","cy","cz","base_revision","payload_b64"}, …]}. Applied in one transaction, all-or-nothing: if any entry conflicts, nothing is written and the response is 409 naming the offending coordinate. Justification: an editor brush stroke or an object drag routinely straddles a chunk boundary, and without a batch the editor either issues N round trips or leaves half of a user action persisted. Capped at WORLD_CHUNK_MAX_BATCH chunks and 8 MB.

8.2 D5 — Full-chunk PUT, not JSON-Patch deltas

world.md §5.4 specifies RFC-6902 deltas against base_revision. This design sends whole chunks instead.

  • A 32×32×1 chunk is ~2 KB of dense tile indices plus its objects — a few KB. The bandwidth a patch saves is negligible at editor rates.
  • The stored form is an opaque binary blob. Patching it means the backend must decode → apply a JSON-Patch → re-encode → re-checksum, i.e. implement and test a second mutation format on top of the codec, with its own failure modes, and keep it in sync with the client's patch generator. That is the same dual-path drift risk world.md §5.2 exists to prevent.
  • Lost-update protection comes from base_revision, not from the patch, so nothing is given up on correctness.

Delta encoding stays available as a pure bandwidth optimization later — world.md §8 already files it as future work.

8.3 Internal (dedicated server; require_game_server token)

Method Path Purpose
GET /api/internal/worlds/{world_id}/manifest Rooms, grids, spawn, content_revision, chunk counts
GET /api/internal/worlds/{world_id}/rooms/{slug}/chunks Bounds index (same shape as §8.1)
GET /api/internal/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz} Binary payload

Read-only by construction — there is no internal write route, and adding one would violate D2. The token is world-scoped, so a path whose world_id differs from the token's world is 403. These routes skip the per-user visibility check (the sim process has no user) and apply no mature gating (gating is a client/user concern; the server still needs collision).

8.4 D6 — The client streams chunks over HTTP, not over ENet

world.md §5.3 routes request_chunk / chunk_data over the game's ENet connection. This design has the playing client fetch chunks from the backend over HTTP, the same routes the editor uses.

  • The dedicated server has no content the backend doesn't have; proxying bytes through it adds a hop and puts multi-megabyte reliable transfers in contention with the 30 Hz snapshot broadcast on the same ENet peer.
  • HTTP gives caching for free — ETag/If-None-Match on the checksum, and a local user://chunk_cache/<checksum> that survives sessions.
  • ApiClient (scripts/api_client.gd) already carries the JWT and the result contract; no new transport code.

This does not require rewriting ChunkStreamManager. It already calls through a set_server_rpc() seam that only needs an object with request_chunk(key). The editor-wiring task supplies an HttpChunkSource that implements that method against ApiClient and feeds _on_chunk_data_received(key, bytes) — which also finally gives that dormant seam a caller. Note the existing _load_chunk() builds its key with empty world/room ids (Chunk.build_key("", "", coord)); the adapter must supply the real world id and room slug.


9. What each follow-on task owns

Backend task

  • backend/app/models/world_content.py + 0005_add_world_content_tables
  • backend/app/services/chunk_codec.py (Python port of world.md §3.6) and the golden-file fixtures shared with the Godot suite
  • backend/app/services/world_store.py (the world.md §5.1 service interface, minus flush_dirty/rebuild_index, over Postgres)
  • backend/app/routers/world_content.py, backend/app/routers/internal.py
  • game_servers.py: per-spawn token, --api-url/--server-token args, readiness-from-status-file instead of _await_port (§7.4)
  • create_world creates the main room

Client/editor task

  • HttpChunkSource adapter → ChunkStreamManager.set_server_rpc()
  • Editor save path (single + batch PUT), 409 conflict re-fetch-and-reapply, live_session warning UI
  • "Edit world" entry from the world browser that does not join
  • server.gd: arg parsing, manifest + chunk fetch before create_server, collision build from decoded chunks, test_level.tscn fallback, "ready"/"content" fields in the status file

10. Open questions (deliberately not decided here)

  • Hot reload / build-in-place while a session is live (§6) — the content_revision seam is in place for it.
  • Collaborator permissions beyond owner_id — one check in §5.3.
  • Mature-content gating on the read path — column and index exist; the policy belongs to content_moderation.md.
  • world_chunk_objects derived index — spec'd in world.md §4.5, build it when a query needs it (§4.2).
  • Payload compression — the §3.6 header reserves a flags bitmask; Postgres TOAST already compresses BYTEA, so measure before adding a second layer.
  • Chunk streaming for very large worlds — the bounds-query cap in §8.1 is a guardrail, not a scaling plan.