# Finding out why a game is slow

The engine carries a flight recorder. While a game runs, it writes a small record of what every part of every frame did into a fixed ring in memory: which behaviours ran and for how long, how long physics, rendering and the frame itself took, and, when the run has GPU timestamps, how long each GPU pass took. Nothing is saved or sent unless something asks. When a frame is far slower than its neighbours the recorder saves the last few seconds to disk on its own, and everything below reads from that ring or from those saved captures.

This guide is for a person watching a game in the editor and for a tool or script asked "why is it slow?". They read the same numbers: the editor's Profiler tab and the query API draw from one layout, so a picture an agent is handed is the picture a person sees.

---

## 1. Turning it on and off

The recorder is on in every engine built with the default features, and it costs a few microseconds a frame: a span is one clock read and one 24-byte write into a ring that never allocates once it has filled. It is off in an export built without the `profile` feature, which holds no ring at all.

| Setting | What it does |
|---|---|
| `ARGUS_PROFILE=off` (or `0`, `false`) | Turns the recorder off for that run. Every profile endpoint then answers that it is off, and says why. |
| `ARGUS_PROFILE_RING_MB=<n>` | The ring's size. The default is 8 MiB, about ten seconds of a busy game. |
| `ARGUS_PROFILE_DIR=<folder>` | A run started outside the editor saves spike captures into this folder, and records its input beside them. The editor does the same for you, into `.argus/profile` in the project. |

`docs/PERFORMANCE.md`, "Profiler overhead", holds the measured cost of leaving it on. It is written by a script, with the machine's load beside each number.

## 2. Spikes and captures

The recorder watches every frame. A frame is a **spike** when it is more than 2.5 times the median frame and more than 2 ms over it, or when it takes over 33 ms, or when it is over a scene's own frame budget. It waits 120 frames after a start before it judges anything, and it saves at most one capture every ten seconds, so one long hitch does not fill the disk.

A **capture** is a file named for what raised it, such as `absolute-f300.argusprof`, holding the last 180 or so frames with every span, and a header saying what the game was doing (the scene, the seed, draw and body counts, and the recipe to replay the run). The newest eight are kept. Next to it the recorder writes the same frames as a Chrome trace, `absolute-f300.trace.json`.

You can also save one by hand at any time: press **Capture now** in the editor, or `POST /profile/capture`.

Where they go: `.argus/profile/` inside the project when the editor runs the game, or the folder named by `ARGUS_PROFILE_DIR` for a run started from the command line.

## 3. The editor's Profiler tab

Open it from **Window > Profiler**. It sits in the bottom drawer beside Activity, Jobs, Content, Output Log, History, Library and Animator (its short name on a crowded bar is "Prof"). With nothing running it says to press Play.

- **The headline** says what the recent frames are bound by, in one line: for example "Bound by CPU, script (64%), frame 5.7 ms, p95 6.0 ms", with the analyser's top finding after it. "Bound by GPU" names the pass. Software rendering can make the GPU side look larger than it would on a real card, see section 8.
- **The frame-time strip** has a column for every recent frame. Height is the frame's cost on a linear scale, and the 16.7 ms line is drawn over the bars, so an ordinary frame sits well below it and a hitch towers over it. A bar taller than the scale is cut off at the top with a white cap. The colour says what dominated the frame: script, physics, render or GPU. A spike has a red mark under its cap. **Press a spike** (or any bar) to select that frame; the strip stops scrolling so the frame stays where you pointed, and **Follow** puts the newest frame back.
- **The timeline** draws the selected frame's spans on one clock, with a row for each level of nesting: the frame, its phases, each behaviour call, and a GPU row of passes when the run has them. Press a behaviour's bar to select that entity in the viewport and the Outliner.
- **The flame graph** merges the shown frames by path, widest first, so you can see where the time went over the whole window.
- **Compare with...** picks a saved capture and draws a before strip and an after strip, the change in frame cost with its noise band, and the spans that changed by more than the noise: green for faster, red for slower.
- **Source** switches between the live game and any saved capture, so a spike can be read after the game has stopped.
- **Heat** (while playing) tints each entity's bounds in the viewport by what it costs, one metric at a time: script time per tick, triangles or contacts. Cold blue is cheap and hot yellow is costly, and an entity that cost nothing is not tinted. Triangles and draw counts turn on a few counters that cost something in a hot loop, so the tab turns them on while Heat is showing and off when it closes. **Surfaces** paints the meshes themselves in their heat colour and everything else dark grey. It works with the standard renderer only; the HD renderer shows the boxes, not the surfaces.
- **The status bar** shows **Spikes N** while a game runs and has hitched. Press it to open the tab.

The editor also records your input during Play, into `session-<id>.replay` beside the captures, so any capture from that run can be replayed later (section 6).

## 4. The live overlay

Run a game in a window with `--stats` (or press **F3** in the game window). Under the frame-rate rows the overlay gains two more:

```text
bound by CPU 64% | script-hot: patrol.update is 73% of CPU time
frames [..::-=+*#%@!..] max 36.2 ms
```

The first row says what the last frames are bound by and the top finding; the second is the last 48 frames as characters, `.` for cheap up to `@` for costly, and `!` for a spike. It is recomputed twice a second and costs nothing while the overlay is off.

## 5. The query API

A running game with `--serve` answers these. Every one is a plain HTTP request, and all but the two `POST`s are reads.

| Request | What you get |
|---|---|
| `GET /profile` | Whether the recorder is on, how full the ring is, how many spikes it has seen, and what is on disk. |
| `GET /profile/frames?last=120` | One row per frame: cost, the time in each phase, and a spike flag. What the strip draws. |
| `GET /profile/trace?frames=120` | Chrome trace JSON of the ring; add `capture=<id>` for a saved capture. Opens in the Perfetto viewer. |
| `POST /profile/capture` with `{"reason": "before-fix"}` | Saves the ring now and answers the capture's id. |
| `GET /profile/captures` | The captures on disk, newest first. |
| `GET /profile/capture/<id>` | One capture's header and shape. |
| `GET /profile/blame?by=script` | Who paid. `by` is `script`, `phase`, `entity`, `render`, `physics`, `asset` or `load`. `by=entity&metric=triangles` ranks entities. |
| `POST /profile/detail` with `{"on": true}` | Turns on the counters that cost something in a hot loop (draw groups, script API calls) for 30 seconds. Read again a moment later. |
| `GET /profile/explain` | What the window is bound by and a ranked list of findings, each with evidence, a suggested change, an estimated saving and how to confirm it. |
| `GET /profile/diff?a=<capture>` | A saved capture against the live ring, or against another capture with `b=<id>`: per-span changes with a noise band, and a verdict on each finding. |
| `POST /profile/rerun` with `{"a": "<capture>"}` | Replays the capture's recorded run against the project as it is now, then `GET /profile/rerun/<id>` says how it is going. |
| `GET /profile/image?kind=strip` | A PNG. `kind` is `strip`, `timeline` (add `frame=`), `flame` or `heat` (add `metric=`). Add `capture=<id>` for a saved capture. |

A game with a reader attached pays to publish its state for it. `serve.publish` is that span on the main track, between frames, with the entity count as its argument. `serve.publish.scripts`, under it, is the walk of every behaviour's `ctx.state`, which only happens once something has read an entity (a `GET /entity/...`, an agent, the editor) and stays on after that. A timing run whose harness reads entities measures this cost too; a player's game has no reader.

`explain` is where to start. Each finding has an id such as `script-hot` (one behaviour is most of the CPU time), `script-many` (many small calls add up), `render-cpu`, `draw-bound`, `physics-heavy`, `gpu-pass-hot`, `vsync` (the frame is waiting for the display, so there is nothing to fix) or `unexplained`. It also lists what the window is **not** bound by, so an agent does not go and optimise something that costs 4% of the frame.

GPU time is reported only when the run has GPU timestamps. Without them it is `null`, never 0, and a finding about the GPU says so.

GPU timing keeps its own cost small: when timing every frame costs more than 50 µs of CPU, it times 1 frame in 4, and on a software adapter it starts there. That choice is made per run, so two runs of the same build can differ. A frame it did not time has no GPU spans, which means "not measured", not 0 ms. The Perfetto export's GPU row says which rate the run used ("timing every frame" or "timed 1 frame in 4"). For a measurement, such as an A/B of GPU time, start the engine with `ARGUS_GPU_TIMING=every`: every frame is timed for the whole run. `ARGUS_GPU_TIMING=off` is the other way round: the recorder keeps its CPU spans but nothing is GPU-timed, so a CPU measurement carries none of the timer's own cost (its timestamp writes and query resolve).

The standard renderer's encode spans carry counts in their two args (the Perfetto export's `arg` and `arg2`): `render.encode.prepare` the opaque pass's material-binding floor (the distinct built-in and raw materials and user-shader batch classes its draws used: the fewest material binds any draw order could make; against `.main.opaque`'s bind-group sets, what sorting could save); `render.encode.shadows` the cascades' runs and draw calls; `render.encode.main` the draws that went in a multi-draw and how many multi-draw calls they took; `render.encode.main.opaque` its draw calls and per-draw bind-group sets; `render.encode.main.scene` its draw calls and how many were blended surfaces; `render.encode.main.post` and `.layers` their draw calls; `render.encode.main.resolve` the shadows' and the scene part's bind-group sets; `render.encode.main.finish` the frame's per-draw bind-group and pipeline sets, the state `CommandEncoder::finish` replays and tracks. `render.instances.ism.cull` carries the instanced meshes' visible cells; `render.instances.ism.rows` the groups drawn from kept rows (one per run of visible cells, `Renderer::keep_rows`) and the groups built from the frame's rows; `render.submit.set_instances` carries the most kept rows so far and the kept region's KiB on the GPU, and splits into `.upload`, `.groups`, `.user_rows` and `.draw_order`.

## 6. Checking a fix: the AI loop

The `profile` toolset holds `profile.explain`, `profile.blame`, `profile.capture`, `profile.diff` and `profile.view`. It is not in the default tool list, so an agent asks for it once with `tools.enable` (`profile`), or by calling any of its tools. The usual loop:

1. **Ask why.** `profile.explain` while the game runs. Read the first finding: its evidence names the behaviour or pass, and its confirm line says how to check the fix.
2. **Save a baseline.** `profile.capture` with `action: save`. Note the id it returns. The capture carries a recipe for replaying the run when the game was recording its input, which is always the case for Play in the editor.
3. **Change the project.** Edit the behaviour, the scene or the material.
4. **Diff with replay.** `profile.diff` with `a` set to the baseline id. It replays the baseline's recorded input against the edited project, three times by default, pools the runs and answers per-span changes with a noise band, and a verdict on each finding: confirmed (the saving showed up), partial, not confirmed, regressed, or inconclusive when the noise is larger than the change. A change that alters what the game does, not just how fast it does it, diverges from the recording, and the diff says so instead of pretending the two runs compare. Pass `replay: false` to compare against the live ring instead.
5. **Look.** `profile.view` with `kind: flame` or `kind: heat` returns a picture, held to the same size budget as any capture.

A capture with no replay recipe (a run that was not recording its input) can still be explained and compared against the live ring, but not replayed. Start such a run with `--save-replay file.replay`, or with `ARGUS_PROFILE_DIR` set, and captures from then on can be.

To read a capture without the toolset, use the routes in section 5 directly through `play.query`.

## 7. Perfetto

`GET /profile/trace` and `profile.capture` with `action: export` produce Chrome trace JSON, which the Perfetto viewer at ui.perfetto.dev opens directly, with one row per track (main thread, worker threads, GPU, present, load) and every span. From a saved capture on disk:

```sh
argus profile-trace .argus/profile/absolute-f300.argusprof
```

writes `absolute-f300.argusprof.trace.json` beside it (or where `--out` says). The export can hash script and scene names (`redact`) before a trace leaves the machine.

## 8. What is not measured

- **GPU time on a software renderer.** A CPU-based Vulkan driver (llvmpipe, lavapipe) runs the "GPU" on the same cores as the game, so its pass times include time stolen from the CPU side. Trust the shape, not the milliseconds, and expect a game that is CPU-bound on a real card to look GPU-bound there.
- **GPU time without timestamps.** A run whose device cannot time GPU passes reports GPU time as `null` and its frame cost as the CPU side alone.
- **Per-entity GPU cost.** GPU-driven passes do not attribute time to one entity. The HD renderer reports triangle share times pass time, marked as an estimate. Everything estimated says `method: "estimate"`.
- **HD surfaces.** The heat map's Surfaces button paints meshes in the standard renderer only. The HD renderer does not read the view-mode flag, so there the map is the tinted bounds.
- **Time inside the operating system.** Waiting on the display, a driver stall, another program taking the CPU: they show as a slow frame without a span to blame. The `vsync` and `unexplained` findings say so.
- **Load on the machine.** Every measurement, including the overhead table, is affected by what else the machine is doing. The table records the load average beside each number for that reason.