Skip to main content

Deployment

Services and ports​

docker-compose.yml runs the whole stack:

serviceportwhat it is
portal8082player accounts, characters, JWT play tokens
admin8081server controls, bans, audit log
docs8083this site (Docusaurus → nginx)
game-server7777/udpGodot headless, ENet
uptime-kuma3001uptime monitoring
postgres5432database

Start everything:

cp .env.example .env      # then edit secrets
docker compose up -d --build

Bring up only the game server and its database:

docker compose --profile game build game-server
docker compose --profile game up -d game-server

Game server hosting​

The game server is built as a container image by docker/Dockerfile.game-server, which compiles the C# project and runs a Godot export in one step. The image does not need Godot or .NET installed on the host — only Docker.

Moving it to another host​

A second game server is normally deployed to measure movement and physics rather than to serve players. Only the game server needs to move; the portal, admin and docs are not involved.

Prerequisites

  • Docker
  • Repo access — godot_client/, Moondog.GameServer/, docker-compose.yml, rpc_contract.json
  • UDP 7777 reachable from the client host

Verify the map data matches. This is the check that matters most when comparing two hosts:

diff -rq godot_client/map_data Moondog.GameServer/map_data

No output means identical. Both sides must load the same heightmap, or the comparison is meaningless.

Build and run

git pull
docker compose --profile game build game-server
docker compose --profile game up -d game-server
docker logs --tail 30 moondog-game-server-1

Look for two lines:

Moondog game server listening on UDP 7777
Terrain: v... collision_mode=3 regions=N ...

collision_mode=3 is FULL_GAME and is mandatory. The default generates collision around a Camera3D, which a headless server does not have, so players would fall straight through the world.

Godot buffers stdout inside a container, so listening and Peer connected may not appear in docker logs for several seconds. Diagnostics that matter are written to stderr with GD.PrintErr and appear immediately. If the container looks hung, wait and re-check rather than assuming failure.

Environment variables​

  • MOONDOG_DB or ConnectionStrings__Default
  • JWT_SECRET — must match the portal's, or tokens are rejected. A mismatch is reported as IDX10517 ... the token's kid is missing, which reads like a key-rotation fault but is not one; see diagnosing a rejected token. Each server prints a JWT_SECRET fingerprint at startup so the two can be compared without revealing either secret.
  • JWT_ISSUER (default Moondog)
  • JWT_AUDIENCE (default MoondogClient)
  • GAMESERVER_PORT (default 7777)
  • GAMESERVER_MAX_CLIENTS (default 100)
  • GAMESERVER_NAME (default main)

Running a movement measurement​

The client ships a headless probe that drives scripted input through the normal code path and reports divergence between the client's prediction and the server's authoritative position.

python3 scripts/mint_token.py --creds /tmp/moondog-creds.txt --out-dir /tmp/moondog-tokens

MOONDOG_HOST=<game-server-host> MOONDOG_PORT=7777 \
godot --headless --path godot_client -- \
  --token-file=/tmp/moondog-tokens/nyx.token \
  --probe=straight --probe-seconds=20

Host and port come from the MOONDOG_HOST and MOONDOG_PORT environment variables — there are no CLI flags for them. The client accepts --token= / --token-file= and --probe= / --probe-seconds= as arguments.

scenariowhat it does
idlestands still — baseline for agreement at rest
straightwalks one direction
turnwalks while sweeping facing
figure8wanders continuously
sprint_jumpsprints, jumping periodically
stopsprints 2s, then releases and stands still

stop exists because the lurch on releasing a key is not covered by the others — every other scenario holds its input for the whole run.

Tokens expire in 60 minutes. An expired one is logged as Lifetime validation failed by the server while the client reports connection_failed, which reads like two unrelated bugs.

Reading the output​

DIVERGENCE scenario=stop count=1211 mean=7.7317 p50=0.0977 p95=0.2310 max=1154.0

These are divergence statistics. Ignore max on short runs — the first couple of packets compare against the default origin position before the prediction buffer fills.

For per-packet detail, run with correction enabled and read:

~/.local/share/godot/app_userdata/"Moondog Client (example)"/correction_<character>_<pid>.log

The filename carries the character's first 8 hex digits and the OS process id, so two clients running at once write separate files rather than appending to one -- which is what makes a two-client comparison possible at all. Each file's first line also records the full character id, pid and start time, so it stays self-describing after several runs have accumulated. Match them with correction_*.log and take the newest.

columnmeaning
h / ht / hfhorizontal gap, threshold, whether it fired
v / vt / vfvertical gap, threshold, whether it fired
spdour horizontal speed, m/s
stmeasured staleness of the snapshot, ms
fpsclient frame rate
yaw_c / yaw_s / dyawour yaw, the server's, and the difference
steerhow differently the server's yaw would steer us, m/s
terrterrain's own contribution to the vertical gap
resvertical gap after terrain is removed — what vf decides on
fl / sflour floor state / the server's
truelike-for-like error — server position against our prediction for the seq it echoed
dtms since the previous packet

true is the column that isolates simulation disagreement. Everything else is measuring the link. h in particular compares the server's position against our current position — two different instants — so it includes transport lag by construction. See Architecture → Movement reconciliation.

Simulating latency on a single host​

To reproduce a specific round trip without a second machine, netem delays packets in-kernel. It must be applied to the Docker bridge the game server container is attached to, not lo — loopback delay will not touch the game's traffic at all.

# which bridge?
docker inspect moondog-game-server-1 \
  --format '{{range $k,$v := .NetworkSettings.Networks}}{{$k}} {{$v.IPAddress}}{{end}}'
ip -br link | grep br-

tc qdisc add dev <bridge> root netem delay 63ms   # ~126ms round trip
ping -c 3 <container-ip>                          # expect ~63ms

tc qdisc del dev <bridge> root                    # always clean up
ping -c 3 <container-ip>                          # expect <1ms

Check tc qdisc show dev <bridge> before adding. A leftover qdisc from an earlier session stacks with a new one and the delays compound, which invalidates measurements without producing any visible error.

Telemetry​

The portal accepts movement captures from the example client when, and only when, a sink is configured:

Telemetry__Sink: "http://<collector>:8080"
Telemetry__RetentionDays: "14"

Empty means no sink, which means the portal tells clients there is nowhere to send anything and the endpoint answers 404. A production deployment should leave this unset. It is a diagnostic for people testing movement, not player telemetry, and the two should not be confused or enabled together — a player build has no upload path, and turning the sink on for real players would be a decision about consent and retention that this is not the place to make quietly.

The sink URL is returned in the /api/gameauth/token response, which is what makes the client side zero-configuration: a tester pointed at a portal that has one gets it without setting anything, and a portal without one hands the client nothing so the code path never runs.

Uploads are authenticated by the play token the client already holds. That works when the client and this portal are the same deployment, and cannot work across two: a tester whose portal has its own JWT secret gets a token this portal rejects, and every capture is refused. Making both secrets identical would couple the environments so that leaking either leaks both.

So a second credential exists, scoped to telemetry and independent of JWT:

Telemetry__Sink: "http://<collector>:8080"
Telemetry__SinkKey: "<shared ingest credential>"

Either one authenticates. The key travels in an X-Moondog-Telemetry-Key header, never in a URL, so it does not land in proxy access logs, and it is compared in constant time. An upload authenticated by the key is stored with no user or character attached — the key carries no identity claims, and guessing at one would be worse than recording that nobody is known.

Set it in .env, not in a shell export. A per-invocation export does not survive a restart, and the failure mode when it is missing is the quiet one: the portal comes up with no key, every key-authenticated capture is refused, and nothing on the tester's side reports an error. The key is also returned to clients in the /api/gameauth/token response, so anyone who can log into a portal can read it — fine for a test diagnostic, but it means the key is only as safe as the portal login and must not be reused anywhere else.

To rotate: change it in both .env files and restart both portals. Between the two restarts, uploads fail 401, which is the safe direction to fail.

Because this endpoint is internet-facing rather than a localhost convenience:

  • Telemetry__RatePerMinute (default 30) is a sliding one-minute window per credential, so one client in a runaway loop cannot spend everyone's budget. Rate-limit keys are SHA-256 fingerprints, never the raw token — otherwise the map would hold every play token the collector ever saw.
  • Telemetry__MaxTotalMB (default 2048) caps everything stored. Per-row age retention is not enough: a client posting 8 MB every few seconds fills a disk long before the age sweep helps. The counter is maintained incrementally rather than summed per request, because SELECT sum(length(payload)) over a table whose purpose is to grow is a scan on every upload.
  • Telemetry__RetentionDays (default 14) expires rows on write rather than from a timer — the table's only failure mode is growth, and growth only happens when uploads arrive.

Retrieval is direct rather than through an endpoint, because there is exactly one reader and a read API would be surface without a user:

SELECT "CreatedAt", "Tester", "Platform", "OsVersion", "RefreshRate",
       "Scenario", "Note", length("Payload")
FROM telemetry_uploads
-- connectivity-check rows are the client's own "is this working" probe and are
-- not movement data; filtering them keeps the list meaningful.
WHERE "Scenario" <> 'connectivity-check'
ORDER BY "CreatedAt" DESC LIMIT 20;

RefreshRate is the column worth having. Client frame rate governs input cadence and prediction against a server that always ticks at 60 Hz, so a capture from a 144 Hz machine and one from a 60 Hz machine are different experiments wearing the same label. It is -1 when unknown — chiefly in headless runs — and -1 rather than 0, so that "not reported" stays distinguishable from a real reading.

Payloads are gzipped.

Production checklist​

  • Use strong passwords and JWT secrets
  • Run the web apps behind HTTPS (Caddy/Traefik/nginx)
  • Back up PostgreSQL — docker compose includes a postgres-backup service
  • Restrict admin panel access (network allowlist or VPN)
  • Monitor server_status heartbeats
  • Rotate credentials periodically
  • Enable Docker restart policies
  • Leave Telemetry__Sink unset. A production deployment must not accept movement captures; the feature exists for people testing movement and is inert without it.

This host is not yet a hardened deployment. It has been a development target: the admin panel is still bound publicly, SSH is open to internet scans, and the database password and JWT_SECRET were reachable before 3be6b01 unpublished Postgres. docs/ALPHA.md in the repository tracks the hardening phase that closes these. Expand the bullets above into a concrete checklist when it opens.

portal.example.com {
  reverse_proxy 127.0.0.1:8082
}

admin.example.com {
  reverse_proxy 127.0.0.1:8081
}