Decides where chunk/object content lives at rest and who may write it, now that there are two server-side processes per world (the always-on FastAPI backend and the per-join, reaper-killed Godot dedicated server). Decisions: chunk data lives in the existing Postgres DB (not world.md's per-world SQLite file); the FastAPI backend is the sole writer and the editor saves to it over HTTP; the dedicated server pulls content read-only at startup over a loopback /api/internal route with a per-spawn token; full-chunk PUT with base_revision instead of JSON-Patch deltas; the client streams chunks over HTTP through the existing ChunkStreamManager.set_server_rpc() seam. Adds docs/architecture/world_persistence_impl.md and marks the superseded sections of world.md (SS4, SS5) and editor.md (the request_chunk_edit path). Design only -- no implementation code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y2FESCaMXeJpC9gSctiEeD
34 KiB
World Persistence: Server-Side Implementation Architecture
Status: Accepted (design). Implementation is split across two follow-on tasks (backend endpoints; editor wiring). Owner: Platform team Supersedes: the storage engine and write ownership parts of
world.md§4 and §5. Everything else inworld.md— the chunk logical model, the JSON and binary encodings, checksums, revisions, content flags — stands unchanged and is normative. Related:world.md(chunk format),editor.md(authoring surface),networking.md(ENet sim transport),content_moderation.md(flags & gating).
1. Why this document exists
world.md was written during the client-side chunk-streaming work (M2). It
assumes one server that owns one embedded SQLite file and is both the
authoritative simulation and the authoritative store. That is not the system we
built. What actually exists today is:
| Process | Lifetime | Owns | Code |
|---|---|---|---|
| FastAPI backend | always on, single worker | Postgres (users, worlds, assets, avatar loadouts), game-server orchestration | backend/app/ |
| Godot dedicated server, one per world | spawned on join, reaped ~5 min after the last join when empty | nothing durable; a 30 Hz in-memory player sim | scripts/server.gd, spawned by backend/app/services/game_servers.py |
| Godot client | player session | local chunk streaming, the in-game editor (planned) | scripts/chunk_stream_manager.gd, scripts/chunk_data_decoder.gd |
Nothing server-side has ever stored or served a real chunk. server.gd
preload()s res://scenes/worlds/test_level.tscn for every world and uses
world_id only for logging and the status file. ChunkStreamManager has a
set_server_rpc() seam that no code calls. The backend's World model is
metadata only (id, name, description, owner_id, visibility).
This document decides where chunk content lives, who may write it, how the dedicated server gets it, and the exact API surface — so the two implementation tasks that follow have no architectural decisions left to make.
2. Decisions at a glance
| # | Question | Decision |
|---|---|---|
| D1 | Where does chunk data live at rest? | Postgres, in the existing backend database. Not a per-world SQLite file. |
| D2 | Who writes it? | The FastAPI backend, exclusively. The editor writes through it over HTTP with optimistic concurrency. The dedicated Godot server never writes world content. |
| D3 | How does the dedicated server get content? | HTTP pull from the backend at startup, over a loopback-only /api/internal route authenticated by a per-spawn token the supervisor mints. Lazy fetch as players roam. |
| D4 | Edit while a live session exists? | Out of scope this milestone, with a defined seam. The editor is a non-joined client mode; writes always succeed and the response flags live_session so the UI can say "players see this when the session restarts". |
| D5 | Edit granularity on the wire | Full-chunk PUT with base_revision, not RFC-6902 deltas. |
| D6 | Client chunk transport | HTTP from the backend, not ENet from the dedicated server. The set_server_rpc() seam is satisfied by an HTTP-backed adapter. |
3. D1 — Chunk data lives in Postgres
3.1 Decision
World content (rooms, chunk payloads, chunk revisions) is stored in the
existing Postgres database, alongside worlds, in new tables created by an
Alembic migration. world.md §4's world.db SQLite file is not built.
The chunk payload stays exactly what world.md §3.6 specifies: the 48-byte
header + sectioned binary body, checksummed with SHA-256. It moves from a
SQLite BLOB to a Postgres BYTEA. Nothing about the format changes, so
scripts/chunk_data_decoder.gd remains correct and the format spec stays the
single contract.
3.2 Reasoning
- SQLite's premise does not hold here.
world.md§4 justifies SQLite as "ideal for a server that owns the world store". No process in this system owns a world store: the only process that could plausibly hold the file (the dedicated Godot server) is ephemeral — reaped when idle,SIGTERM'd on backend shutdown,SIGKILL'd 10 s later if it doesn't exit, and orphaned entirely if the backend dies. Meanwhile the editor must write when no such process exists. So the file would have to be shared by two processes with independent lifetimes, which is precisely the workload SQLite's single-writer/file-lock model handles worst. - Postgres is already a hard dependency; SQLite would be a second one. The
backend cannot start without Postgres. Adding a per-world SQLite file adds a
second storage engine, a second migration system (
world.md§4.8 specifies a hand-rolledmeta.schema_versionrunner next to the Alembic we already have), a second backup story, and a second consistency story. - Referential integrity with metadata we already have.
worldsrows carryowner_idandvisibility, and every authorization rule for content derives from them. In one database this is a join and a foreign key withON DELETE CASCADE; across two engines, deleting a world silently orphans a file on disk and every ownership check becomes a cross-store read. - Concurrency is the normal case, not the edge case. Two creators editing
one world, or an editor writing while the backend serves chunks to a running
dedicated server, are ordinary. Postgres gives row-level MVCC and real
transactions for the read-modify-write in §5.4; SQLite gives a database-level
write lock and
SQLITE_BUSY. - The repo already has the precedent and the plumbing.
avatar_loadouts(migration0004) stores a client-shaped document in Postgres JSONB rather than inventing a side store, andbackend/app/database.pyalready provides the async session factory this code will use.
3.3 What we give up, and why it's acceptable
- Zero-config embedding. Irrelevant — the backend already requires a configured Postgres.
world.md§5.5 write-behind + thedirtycolumn. Deliberately dropped (§4.4). It existed to amortize disk I/O for a process that owned the file. Our writes are human-rate editor saves, not sim-rate; each one is a single transaction and is durable when the HTTP response returns. Dropping it also deletes the entire "unclean shutdown recovery" scan in §5.5 — there is no window in which committed content can be lost.- Sharding by
world_id(world.md§8). Postgres partitioning byworld_idremains available on the same table shape if a single table ever becomes a bottleneck. At ~2–8 KB per chunk, 10 000 chunks is under 100 MB.
4. Schema (Alembic migration 0005_add_world_content_tables)
New model module: backend/app/models/world_content.py, registered in
backend/app/models/__init__.py. backend/app/models/world.py is left alone
except for one added relationship, so nothing PC1/PC2 depends on moves.
Conventions follow the existing models: Column(...) declarative style,
Integer surrogate PKs, DateTime(timezone=True) with server_default=func.now(),
JSONB for document-shaped fields (as avatar_loadout.py does).
4.1 world_rooms
class WorldRoom(Base):
__tablename__ = "world_rooms"
id = Column(Integer, primary_key=True, index=True)
world_id = Column(Integer, ForeignKey("worlds.id", ondelete="CASCADE"),
nullable=False, index=True)
slug = Column(String(64), nullable=False) # url-safe; "main" by default
name = Column(String(255), nullable=False)
# Chunk grid config. world.md §2.2 puts this on the world; it lives on the
# room so a world can later hold rooms with different grids, and so this
# migration is purely additive (the existing `worlds` table is untouched).
grid = Column(JSONB, nullable=False,
server_default='{"size": [32, 32, 1], "origin": [0, 0, 0]}')
bounds = Column(JSONB, nullable=True) # {"min":[x,y,z], "max":[x,y,z]}; null = unbounded
spawn = Column(JSONB, nullable=True) # {"pos":[x,y,z], "yaw": 0.0}
created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
updated_at = Column(DateTime(timezone=True), server_default=func.now(),
onupdate=func.now(), nullable=False)
__table_args__ = (UniqueConstraint("world_id", "slug", name="uq_world_rooms_world_slug"),)
World gains
rooms = relationship("WorldRoom", back_populates="world", cascade="all, delete-orphan").
Every world gets a main room: created in create_world, and lazily created on
first room read/write for worlds that predate the migration. grid defaults
match Chunk.CHUNK_SIZE_* in scripts/chunk.gd, which is the client's source
of truth for the same constants.
4.2 world_chunks
class WorldChunk(Base):
__tablename__ = "world_chunks"
id = Column(Integer, primary_key=True, index=True)
world_id = Column(Integer, ForeignKey("worlds.id", ondelete="CASCADE"),
nullable=False, index=True) # denormalized: world-scoped queries
room_id = Column(Integer, ForeignKey("world_rooms.id", ondelete="CASCADE"),
nullable=False, index=True)
chunk_key = Column(String(160), nullable=False, unique=True) # see §4.5
cx = Column(Integer, nullable=False)
cy = Column(Integer, nullable=False)
cz = Column(Integer, nullable=False, server_default="0")
format_version = Column(Integer, nullable=False, server_default="1")
payload = Column(LargeBinary, nullable=False) # world.md §3.6 bytes, BYTEA
payload_len = Column(Integer, nullable=False)
checksum = Column(String(64), nullable=False) # SHA-256 hex of payload body
revision = Column(Integer, nullable=False, server_default="1")
content_flags = Column(JSONB, nullable=False, server_default="{}") # world.md §3.4
is_mature = Column(Boolean, nullable=False, server_default="false")
updated_by_id = Column(Integer, ForeignKey("users.id", ondelete="SET NULL"), nullable=True)
created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
updated_at = Column(DateTime(timezone=True), server_default=func.now(),
onupdate=func.now(), nullable=False)
__table_args__ = (
UniqueConstraint("room_id", "cx", "cy", "cz", name="uq_world_chunks_coord"),
Index("ix_world_chunks_room_coord", "room_id", "cx", "cy", "cz"),
Index("ix_world_chunks_mature", "world_id", postgresql_where=text("is_mature")),
)
Deltas from world.md §4.4:
- Surrogate
idPK + unique coordinate constraint, withchunk_keykept as a unique denormalized column. The key string is the client-facing contract (Chunk.build_key) and is embedded in the payload, so storing it gives a single-index lookup by key; the coordinate constraint is what actually enforces one chunk per cell. dirtyremoved — see §3.3.- Content flags folded into the chunk row as JSONB + an indexed
is_matureboolean, instead ofworld.md§4.6's separate 11-columncontent_flagstable. The flags are a document the client already produces in exactly this shape (ChunkDataDecoder.ContentFlags.to_dict()), the only query we anticipate is "is this mature", and folding them in removes a second write per save and theobject_id = ''sentinel hack that §4.6 needed to work around SQLite's NULL-in-primary-key behaviour (Postgres has the same NULL semantics, so the hack would have carried over). world.md§4.5chunk_objectsis deferred, not dropped. It is a derived index that exists to answer queries ("which chunks contain a door?") that no code asks yet, and maintaining it means a delete-and-reinsert of every object row on every save. It is fully rebuildable frompayload, so it can be added in its own migration the day a query needs it, with a backfill. Do not build it in0005.
4.3 world_chunk_revisions
class WorldChunkRevision(Base):
__tablename__ = "world_chunk_revisions"
id = Column(Integer, primary_key=True, index=True)
chunk_key = Column(String(160), nullable=False, index=True) # deliberately NOT a FK
world_id = Column(Integer, nullable=False, index=True) # also not a FK, same reason
revision = Column(Integer, nullable=False)
action = Column(String(16), nullable=False) # 'create' | 'update' | 'delete'
actor_id = Column(Integer, ForeignKey("users.id", ondelete="SET NULL"), nullable=True)
checksum = Column(String(64), nullable=False)
payload = Column(LargeBinary, nullable=True) # full payload at this revision
created_at = Column(DateTime(timezone=True), server_default=func.now(), nullable=False)
Kept from world.md §4.7 including its two design points: the full payload is
stored (rollback needs it) and chunk_key/world_id are not foreign keys,
so deleting a chunk or a world preserves the audit trail.
Retention (new, required): this table grows without bound — a creator
dragging a brush can produce hundreds of revisions per minute. The
implementation must ship a retention policy from day one: keep the newest
WORLD_CHUNK_REVISION_KEEP (default 20) payloads per chunk_key and null out
payload on older rows (keeping the metadata row for the audit trail), pruned
in the same transaction as the write. A created_at index supports a later
time-based prune.
4.4 Config additions (backend/app/config.py)
WORLD_CHUNK_MAX_PAYLOAD_BYTES: int = 1_048_576 # 1 MiB; a dense 32×32 uint16 layer is 2 KiB
WORLD_CHUNK_MAX_OBJECTS: int = 512
WORLD_CHUNK_MAX_BATCH: int = 64
WORLD_CHUNK_REVISION_KEEP: int = 20
GAME_SERVER_INTERNAL_API_URL: str = "http://127.0.0.1:8000"
4.5 Chunk key format
world.md §2.3 specifies "{world_id}:{room_id}:{cx}:{cy}:{cz}" and illustrates
it with w_0001:r_0001. Those ids were illustrative. The real key uses the
backend's identifiers:
"{world.id}:{room.slug}:{cx}:{cy}:{cz}" e.g. "12:main:3:-1:0"
The room slug rather than its numeric id, so a key is human-readable in logs
and stable if rooms are ever re-imported. Chunk.build_key() in
scripts/chunk.gd already produces this shape unmodified.
5. D2 — The FastAPI backend is the only writer
5.1 Decision
The in-game editor (editor.md §7.2) submits edits over HTTP to the FastAPI
backend, which validates, versions, and persists them. The dedicated Godot
server process never writes world content and does not sit in the edit path.
This preserves world.md's and editor.md's actual invariant — clients never
touch storage; they submit edits to a server-authoritative tier that validates
them — while identifying that tier as the always-on backend rather than the
ephemeral sim process. editor.md §4.2's claim that edits ride the same
request_chunk_edit RPC as gameplay is the one part of that doc this design
changes; the moderation and validation properties it derives from that claim are
unaffected, because the validation boundary moves with the writer.
5.2 Reasoning
- Nothing that can vanish in 300 seconds may own the write path. The reaper
in
game_servers.pystop()s an idle server withSIGTERMand escalates toSIGKILLafter 10 s;supervisor.shutdown()does the same to every server on backend shutdown; and if the backend dies uncleanly the servers are orphaned with no one to flush them. Because the sim process holds no unsaved world content in this design, no reap, crash, or kill can ever lose world data — that is the single strongest practical reason for this split. - Editing must work with no dedicated server running. A creator opening their world to build should not have to spin up a 30 Hz sim process, burn a port from the 100-port range, and hold it for 5 minutes past their last save.
server.gdhas no database access, no credentials, and no HTTP client. Routing writes through it means either shipping Postgres credentials into a process spawned per join, or building an ENet→HTTP proxy inside the sim loop. Both are strictly worse than the editor calling the backend directly.- Authorization already lives in the backend.
CurrentUser, theowner_idcheck, andWorldVisibilityare all there. The dedicated server has no user identities at all — it knows peer ids, not accounts.
5.3 Authorization
- Read a chunk: the same rule as
GET /api/worlds/{id}— the world isPUBLIC, or the caller owns it. Otherwise 404 (not 403), matching the existing routers' deliberate non-disclosure. - Write a chunk: caller must be
world.owner_id, matchingupdate_world; 403 otherwise. Collaborator/role-based editing is future work and slots in at this one check. - Mature gating hook:
is_maturechunks are wherecontent_moderation.md's age gate attaches on the read path. Specify the column and the index now; the gate itself lands with that doc's work.
5.4 Concurrency
Optimistic concurrency exactly as world.md §5.4, expressed over HTTP:
- Reader gets
revision(headerX-Chunk-Revision, or the JSON field). - Writer sends
base_revision. - The service does
SELECT ... FOR UPDATEon the chunk row, comparesbase_revisiontorevision, and either commits atrevision + 1or returns 409 withcurrent_revisionandcurrent_checksumso the editor can re-fetch and re-apply. base_revision: 0means "create; fail with 409 if it already exists".
5.5 Validation (the backend must not trust the client)
The write path decodes the submitted payload server-side before storing it. This
requires a Python port of the world.md §3.6 codec —
backend/app/services/chunk_codec.py, mirroring scripts/chunk_data_decoder.gd.
It checks: magic AGPC, format_version <= 1, declared vs. actual
payload_len, recomputed SHA-256 (the header's checksum is advisory), tile
layer dims equal the room's grid.size, tile indices in palette range, object
count ≤ WORLD_CHUNK_MAX_OBJECTS, object positions inside the chunk's bounds,
chunk_key in the payload matching the URL, and string-field length limits
(world.md §7). content_flags and is_mature are recomputed from the
payload's flag sections, never taken from the request body.
Drift control (required): the two codecs must be pinned by golden files —
fixture chunks under tests/fixtures/chunks/ encoded by one implementation and
decoded byte-identically by the other, asserted in both the pytest suite and the
Godot headless suite. Without this the format has two implementations and no
contract test, which is exactly the failure world.md §5.2 was trying to avoid
by having one shared implementation.
6. D4 — Editing versus a live session (scope boundary)
In scope for this milestone:
- The editor is entered as "Edit world" from the world browser: a non-joined
client mode. It does not call
POST /api/worlds/{id}/join, does not open an ENet connection, and does not spawn a dedicated server. It loads chunks over HTTP, edits them locally, and saves them over HTTP. - Writes always succeed regardless of whether a session is live. The write
response carries
live_session: bool(fromsupervisor.debug_state()), so the editor can surface "this world has players in it right now — they'll see your changes when the session restarts." - A dedicated server started after a write sees the new content, because it loads from the store at startup (§7).
Explicitly out of scope, deferred:
- Build-in-place while joined to a live session.
chunk_invalidatedpush to connected clients (world.md§5.3) and hot-reload of the dedicated server's collision world.- Live collaborative editing / cursors (
editor.md§9).
The seam that makes the deferred work cheap: the manifest endpoint (§7.2)
returns a monotonic content_revision for the world, bumped on every chunk
write. Hot reload is then: the dedicated server polls the manifest on a timer
(or the backend pushes to it over the same internal channel), re-fetches the
chunks whose revision changed, and rebuilds those collision bodies. Nothing in
this design has to change to add it.
Why the editor must not join: if it joined, the sim process would enforce collision against content it loaded at spawn while the creator edits a diverging local copy — players would walk through new walls and bounce off deleted ones, with no error anywhere. Making build mode non-joined turns a silent correctness bug into an explicit product boundary.
7. D3 — What the dedicated server does at startup
7.1 Decision
server.gd pulls its world's content from the backend over HTTP at startup,
authenticated by a per-spawn token the supervisor mints, and does not bind
the ENet port until content is loaded.
Rejected alternative — supervisor writes a whole-world snapshot file at spawn
and passes --world-file: attractive (no new auth surface, no HTTP client in
the sim process), but it makes spawn latency proportional to world size, gives
the server no way to fetch a chunk lazily as players roam beyond the preloaded
set, and still needs a separate channel for the hot-reload work above. Rejected —
give the Godot process a Postgres connection: there is no PG driver in
GDScript, and it would put database credentials in a process spawned per join.
7.2 Startup sequence
_parse_args()gains--api-urland--server-token(§7.3).GET /api/internal/worlds/{world_id}/manifest→ rooms, per-roomgrid,bounds,spawn,chunk_count, and the world'scontent_revision.GET /api/internal/worlds/{world_id}/rooms/{slug}/chunks?bounds query for the spawn neighbourhood (radius from the spawn point, default 3 chunks), then fetch each payload; decode with the existingChunkDataDecoderand build static collision from the tile layers and objects.- Then
network_peer.create_server(port), then write the status file. - Fallback: if the backend is unreachable, the world has no rooms, or the
room has zero chunks, load
test_level.tscnas today and record"content": "fallback"in the status file. A content outage must never make a world un-joinable, and this keeps every PC3 flow working before any world has authored content.
During the session the server may lazily fetch chunks as players approach
unloaded coordinates, through the same internal route. It never writes.
Dynamic entity runtime state (positions, health) stays in memory and out of
chunks, exactly as world.md §3.3 already specifies. Shutdown flushes nothing.
7.3 Internal authentication
GameServerSupervisor._spawn() generates token = secrets.token_urlsafe(32),
stores it on the _WorldServer dataclass, and passes it on the command line
alongside the existing --port/--world/--status-file. A FastAPI dependency
require_game_server resolves Authorization: Bearer <token> against
supervisor.validate_token(world_id, token) — a dict lookup, since the
supervisor is an in-process singleton. Properties this gives us:
- A token grants access to one world and dies with its process (the
reaper's
del self._servers[wid]invalidates it). - No long-lived shared secret in config or on disk.
- It reuses the already documented single-worker constraint in
game_servers.py; it does not add a new one.
Rejected alternative — let the dedicated server use the public routes with a
service user account: the public routes enforce per-user visibility and
ownership, which a service identity would have to be exempted from, and it would
pollute updated_by_id/audit rows with a fake user. A separate /api/internal
prefix keeps the two auth models apart and can be blocked at the reverse proxy.
7.4 Lifecycle changes this requires in game_servers.py
These are real, load-bearing findings from the current implementation:
_await_port()is the wrong readiness signal now. It returns as soon as the UDP port is bound, and the port is bound before content exists today. With content loading it must await a"ready": truefield thatserver.gdwrites to the status file after content load +create_server. Otherwise the backend hands a client an address for a server that is still fetching chunks._await_port()fails open. After its 8 s deadline it returns normally, with no error —ensure_serverthen reports success for a server that may never have started. Content loading makes a slow start more likely, so the readiness wait needs a real timeout error (503 to the joining client) and a longer, configurable deadline.- The status file is the only server→backend channel, and
_spawncurrently passes no credentials at all — hence §7.3. player_count()returns 0 when the status file is missing or unparseable, so a live server whose status write fails can be reaped after the idle timeout. Harmless for data (the sim owns nothing durable — the point of D2) but worth a comment where the fallback lives.- No crash restart (documented in the module docstring): if a server dies, the next join respawns it and it reloads content from the store. With D2 this is now genuinely safe rather than merely tolerated.
8. API surface
All paths are under the existing worlds router prefix (/api/worlds) unless
noted. New router module: backend/app/routers/world_content.py, mounted at
/api/worlds; internal routes in backend/app/routers/internal.py at
/api/internal.
8.1 Public (editor & player client; JWT CurrentUser)
| Method | Path | Purpose |
|---|---|---|
GET |
/api/worlds/{world_id}/rooms |
List rooms + grid config |
POST |
/api/worlds/{world_id}/rooms |
Create a room (owner) |
GET |
/api/worlds/{world_id}/rooms/{slug}/chunks |
Chunk index over a bounds query |
GET |
/api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz} |
Fetch one chunk |
PUT |
/api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz} |
Write one chunk |
PUT |
/api/worlds/{world_id}/rooms/{slug}/chunks |
Write up to WORLD_CHUNK_MAX_BATCH chunks atomically |
DELETE |
/api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz} |
Delete one chunk |
GET |
/api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz}/revisions |
Revision history (owner) |
POST |
/api/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz}/rollback |
Restore a revision (owner) |
Room list — GET .../rooms → 200
[{ "id": 7, "slug": "main", "name": "Main",
"grid": { "size": [32, 32, 1], "origin": [0, 0, 0] },
"bounds": null,
"spawn": { "pos": [0, 1.5, 0], "yaw": 0.0 },
"chunk_count": 41 }]
Chunk index — GET .../chunks?cx_min=&cx_max=&cy_min=&cy_max=&cz=0 → 200.
Payloads are not included; this is world.md §5.1's list_chunks(bounds) and
lets a streamer decide what to fetch and skip anything already cached by
checksum.
{ "room": "main", "content_revision": 118,
"chunks": [{ "key": "12:main:0:0:0", "cx": 0, "cy": 0, "cz": 0,
"revision": 4, "checksum": "9f86…", "payload_len": 2344,
"is_mature": false, "updated_at": "2026-09-14T10:02:11Z" }] }
Bounds are capped (default 32×32 chunks per query) and return 422 beyond that.
Chunk fetch — GET .../chunks/{cx}/{cy}/{cz}, content-negotiated:
Accept: application/octet-stream(default for the game client) →200with the rawworld.md§3.6 bytes,ETag: "<checksum>",X-Chunk-Revision: <n>,X-Chunk-Key: <key>.Accept: application/json(tooling, tests, the editor's debug view) → the §3.5 JSON encoding, decoded server-side from the payload. JSON is derived, never stored, so there is no dual-write drift (world.md§4.4).If-None-Match: "<checksum>"→304. This is what makes the client'suser://chunk_cache/cache cheap and is why the checksum is the ETag.404if the chunk does not exist (an unwritten coordinate is not an error to the streamer — it renders empty).
Chunk write — PUT .../chunks/{cx}/{cy}/{cz}
Content-Type: application/octet-stream
X-Base-Revision: 4 # 0 = create
<binary payload>
or, for clients that prefer JSON:
{"base_revision": 4, "payload_b64": "…"}.
Responses:
| Code | Body / meaning |
|---|---|
200 |
{"key", "revision", "checksum", "payload_len", "updated_at", "is_mature", "live_session"} |
409 |
{"detail": "revision conflict", "current_revision": 6, "current_checksum": "…"} |
403 |
not the world owner |
404 |
world or room not found / not visible |
413 |
payload over WORLD_CHUNK_MAX_PAYLOAD_BYTES |
422 |
decode/validation failure ({"detail", "reason"}, e.g. checksum_mismatch, layer_dims, object_out_of_bounds) |
Batch write — PUT .../chunks with
{"chunks": [{"cx","cy","cz","base_revision","payload_b64"}, …]}. Applied in
one transaction, all-or-nothing: if any entry conflicts, nothing is written
and the response is 409 naming the offending coordinate. Justification: an
editor brush stroke or an object drag routinely straddles a chunk boundary, and
without a batch the editor either issues N round trips or leaves half of a user
action persisted. Capped at WORLD_CHUNK_MAX_BATCH chunks and 8 MB.
8.2 D5 — Full-chunk PUT, not JSON-Patch deltas
world.md §5.4 specifies RFC-6902 deltas against base_revision. This design
sends whole chunks instead.
- A 32×32×1 chunk is ~2 KB of dense tile indices plus its objects — a few KB. The bandwidth a patch saves is negligible at editor rates.
- The stored form is an opaque binary blob. Patching it means the backend must
decode → apply a JSON-Patch → re-encode → re-checksum, i.e. implement and test
a second mutation format on top of the codec, with its own failure modes, and
keep it in sync with the client's patch generator. That is the same dual-path
drift risk
world.md§5.2 exists to prevent. - Lost-update protection comes from
base_revision, not from the patch, so nothing is given up on correctness.
Delta encoding stays available as a pure bandwidth optimization later —
world.md §8 already files it as future work.
8.3 Internal (dedicated server; require_game_server token)
| Method | Path | Purpose |
|---|---|---|
GET |
/api/internal/worlds/{world_id}/manifest |
Rooms, grids, spawn, content_revision, chunk counts |
GET |
/api/internal/worlds/{world_id}/rooms/{slug}/chunks |
Bounds index (same shape as §8.1) |
GET |
/api/internal/worlds/{world_id}/rooms/{slug}/chunks/{cx}/{cy}/{cz} |
Binary payload |
Read-only by construction — there is no internal write route, and adding one
would violate D2. The token is world-scoped, so a path whose world_id differs
from the token's world is 403. These routes skip the per-user visibility check
(the sim process has no user) and apply no mature gating (gating is a client/user
concern; the server still needs collision).
8.4 D6 — The client streams chunks over HTTP, not over ENet
world.md §5.3 routes request_chunk / chunk_data over the game's ENet
connection. This design has the playing client fetch chunks from the backend
over HTTP, the same routes the editor uses.
- The dedicated server has no content the backend doesn't have; proxying bytes through it adds a hop and puts multi-megabyte reliable transfers in contention with the 30 Hz snapshot broadcast on the same ENet peer.
- HTTP gives caching for free —
ETag/If-None-Matchon the checksum, and a localuser://chunk_cache/<checksum>that survives sessions. ApiClient(scripts/api_client.gd) already carries the JWT and the result contract; no new transport code.
This does not require rewriting ChunkStreamManager. It already calls
through a set_server_rpc() seam that only needs an object with
request_chunk(key). The editor-wiring task supplies an HttpChunkSource that
implements that method against ApiClient and feeds
_on_chunk_data_received(key, bytes) — which also finally gives that dormant
seam a caller. Note the existing _load_chunk() builds its key with empty
world/room ids (Chunk.build_key("", "", coord)); the adapter must supply the
real world id and room slug.
9. What each follow-on task owns
Backend task
backend/app/models/world_content.py+0005_add_world_content_tablesbackend/app/services/chunk_codec.py(Python port ofworld.md§3.6) and the golden-file fixtures shared with the Godot suitebackend/app/services/world_store.py(theworld.md§5.1 service interface, minusflush_dirty/rebuild_index, over Postgres)backend/app/routers/world_content.py,backend/app/routers/internal.pygame_servers.py: per-spawn token,--api-url/--server-tokenargs, readiness-from-status-file instead of_await_port(§7.4)create_worldcreates themainroom
Client/editor task
HttpChunkSourceadapter →ChunkStreamManager.set_server_rpc()- Editor save path (single + batch PUT), 409 conflict re-fetch-and-reapply,
live_sessionwarning UI - "Edit world" entry from the world browser that does not join
server.gd: arg parsing, manifest + chunk fetch beforecreate_server, collision build from decoded chunks,test_level.tscnfallback,"ready"/"content"fields in the status file
10. Open questions (deliberately not decided here)
- Hot reload / build-in-place while a session is live (§6) — the
content_revisionseam is in place for it. - Collaborator permissions beyond
owner_id— one check in §5.3. - Mature-content gating on the read path — column and index exist; the
policy belongs to
content_moderation.md. world_chunk_objectsderived index — spec'd inworld.md§4.5, build it when a query needs it (§4.2).- Payload compression — the §3.6 header reserves a flags bitmask; Postgres
TOAST already compresses
BYTEA, so measure before adding a second layer. - Chunk streaming for very large worlds — the bounds-query cap in §8.1 is a guardrail, not a scaling plan.