Metrics
GET /api/metrics — the cost, progress and ladder numbers in Prometheus text, and the contract their names are.
مستندات به انگلیسی نوشته شده است. بقیهٔ سایت به فارسی خوانده میشود.
What the console spent, what it is driving, and how far along it is — in Prometheus text.
curl -s 127.0.0.1:4620/api/metrics | head -20
Everything the cost and progress pages show is on this endpoint too, so a figure can be alerted on rather than watched. It is a plain read like every other GET: no flag turns it on, no token guards it, and the console binds to loopback.
56 families here — two of them, phase_console_session_suspect_total and
phase_console_supervisor_detections_total, are Pro, and the sentence below counts the ones every
tier emits.
54 families ship in both tiers. The list below comes from viewer/server/analysis/metrics.ts
METRIC_FAMILIES, and viewer/test/docs-parity.test.ts holds this document to it — a family added
to one and not the other fails the suite, as does calling a counter a gauge.
Scraping it
# prometheus.yml
scrape_configs:
- job_name: phase-console
static_configs:
- targets: ['127.0.0.1:4620']
metrics_path: /api/metrics
The port is whichever this console is on; phase_console_build_info carries the instance name.
phase-console list prints the port of every console on the machine — two of them are two targets,
told apart by that same label.
Raise scrape_timeout above your first scrape. The endpoint reads every plan's board, so the
first request after a console restart pays the cold read for all of them and everything after it is
served from cache — measured on a 119-plan library: 20 s cold, 10–13 ms warm. That is the same
cost the Plans page pays, not something this endpoint adds, but Prometheus's 10 s default will time
the first one out and retry. phase_console_scrape_duration_seconds reports it, so you can see
which kind of scrape you got.
The families
| Metric | Type | What it answers |
|---|---|---|
phase_console_build_info | gauge | Console version and instance, always 1. |
phase_console_scrape_duration_seconds | gauge | Seconds spent assembling this response. |
phase_console_plans | gauge | Plans, by plan status and whether the operator has closed them. |
phase_console_process_heap_used_bytes | gauge | V8 heap in use by this console process. |
phase_console_process_heap_limit_bytes | gauge | The heap this console may grow to — what --max-old-space-size set, as V8 reports it. The ratio against heap_used is the series worth alerting on. |
phase_console_process_resident_bytes | gauge | Resident set size of this console process. |
phase_console_process_external_bytes | gauge | Memory held outside V8's heap by this process — buffers, and the SSE write buffers a slow client fills. |
phase_console_process_event_loop_delay_seconds | gauge | How late this console's event loop is running — the MEAN over the last complete one-minute window (a mean since boot until control-tower phase 56, which no stall could move). A supervisor that cannot answer promptly is one nothing else can either. |
phase_console_process_event_loop_delay_max_seconds | gauge | The worst event-loop delay in the last complete one-minute window (#75). A stall shows here for a whole window after it ended; a window at a second or more also writes a process.loop-stall log line. |
phase_console_process_event_loop_delay_p99_seconds | gauge | The 99th-percentile event-loop delay over the same window — the tail every request, hook and lease refresh waited behind. |
phase_console_process_handles | gauge | Open handles this process holds — sockets, timers, child processes. |
phase_console_process_uptime_seconds | gauge | Seconds since this console process started. A series that keeps resetting is a crash loop, whatever the other gauges say. |
phase_console_process_sse_clients | gauge | Event-stream clients this console is writing to. |
phase_console_process_sessions | gauge | Claude sessions this console has live right now. |
phase_console_process_pty_sessions | gauge | Terminal (pty) sessions this console is holding open. |
phase_console_phases | gauge | Phases per plan, by board state. |
phase_console_plan_progress_ratio | gauge | Done phases over total phases, 0 to 1. |
phase_console_plan_remaining_weight | gauge | Unfinished phase weight, in the plan sizing units. |
phase_console_runs | gauge | Autopilot runs per plan, by run status. |
phase_console_phase_attempts_total | counter | Sessions the console has launched for a plan. |
phase_console_phase_seconds_total | counter | Wall-clock seconds phases of a plan have run for. |
phase_console_run_blocked_seconds_total | counter | Wall-clock seconds a plan's runs had a phase waiting in the admission queue, by class — what held it: other-run, hand, clock or own-run. Each second counts once, charged to the first of those classes heading any waiting phase, so three phases behind one stranger for an hour are one hour: never the phases' queued time summed. Closed stretches only. A phase behind its own run's live lane is serial work, never queued, so it is not counted at all. |
phase_console_spend_usd_total | counter | USD every session of a plan has cost (the run total). |
phase_console_phase_spend_usd_total | counter | USD attributed to a numbered phase of a plan. |
phase_console_spend_residual_usd | gauge | Run total minus what is attributed to phases. Non-zero means a run file disagrees with itself. |
phase_console_ladder_spend_usd_total | counter | USD the remediation ladder was charged on a plan. A SUBSET of the run total. |
phase_console_ladder_rungs_total | counter | Ladder rungs climbed, by rung and how each ended. |
phase_console_settled_usd_today | gauge | USD settled by phases that ended today, in the operator's zone. |
phase_console_ladder_usd_today | gauge | USD the ladder was charged today - the figure the day cap refuses a rung against. |
phase_console_day_cap_usd | gauge | The ladder's per-day USD cap. Absent when no cap is set. |
phase_console_worktrees | gauge | Console-managed checkouts a plan's isolated run holds. Absent when no isolated run exists. |
phase_console_worktree_disk_bytes | gauge | Bytes those checkouts occupy. Absent where du could not answer. |
phase_console_branch_conflicted_files | gauge | Files a plan's run branch already conflicts on with another live branch. |
phase_console_account_usage_ratio | gauge | An account window's utilization, 0 to 1, by account id and window. |
phase_console_account_burn_ratio_per_hour | gauge | An account window's measured burn, utilization per hour, from a line through the last hour's readings. Absent until measured. |
phase_console_account_wall_seconds | gauge | Seconds until an account window walls at its measured burn. Absent when it is flat, unmeasured, or resets first. |
phase_console_load_average | gauge | The machine's 5-minute load average (control-tower phase 100, #135 G.27). Since one-pool phase 26 (#243) it is the load guard's ceiling, not its reading: the guard holds new admissions on the share of CPU busy over the last 30 s (85 % at the default factor 1.5, scaled by the factor), and on this average only above the threshold below. |
phase_console_load_guard_threshold | gauge | The 5-minute load above which new admissions wait whatever the CPU reads: the guard's factor (1.5 by default, loadGuardFactor) times the machine's cores times 2 (LOAD_CEILING_RATIO). Absent when the guard is off (factor 0). Alert on load_average > load_guard_threshold held for long: nothing new is boarding. A hold on CPU busy alone shows in no gauge; the held admission's reason names it ("the machine is busy — CPU … over the last 30 s"). |
phase_console_lane_reservations | gauge | Lanes kept for a phase (control-tower phase 100, #135 D.15–16), by armed — yes once the lane each waits for has ended and it is holding its scope and one lane slot for its phase; no while that lane is still live. A yes that never falls is a phase that is not coming back for its lane: lift it (unreserve). |
phase_console_account_credit_used | gauge | Credits an account has used this month, in its currency's major unit, by account id and currency. Absent while its credit state is unknown. |
phase_console_log_lines_total | counter | Console log lines written, by level. A line the level dropped is not counted. |
phase_console_journal_appends_total | counter | Journal lines appended across every run this process drove. |
phase_console_journal_overflow_total | counter | Journals that crossed the soft cap and fell back to the terminal reserve. |
phase_console_transcript_shed_total | counter | Transcript records dropped rather than stored, by kind. |
phase_console_git_commands_total | counter | Git commands run through the seam, by verb and whether they exited 0. |
phase_console_git_command_seconds_total | counter | Seconds spent inside git, by verb. |
phase_console_engine_calls_total | counter | Bash engine calls, by script and whether the answer was cached. |
phase_console_http_requests_total | counter | HTTP requests answered, by status class. |
phase_console_shell_commands_total | counter | Other child processes run through the seam, by binary and whether they exited 0. |
phase_console_retention_removed_total | counter | Files retention deleted, by sink. |
phase_console_turn_checks_total | counter | Checks of a person's items, by who wrote the verdict and what it was. |
phase_console_turn_check_usd_total | counter | USD the checking sessions cost, by model. |
phase_console_turn_rounds_total | counter | Rounds that changed Your turn, by what woke them (the clock or a journal line). |
phase_console_turn_handled_total | counter | Things the AI handled instead of asking a person, by source. |
| phase_console_session_suspect_total | counter | Suspicions raised and never acted on, by kind (loop, stuck). |
| phase_console_supervisor_detections_total | counter | What the supervisor detected and what became of each decision (control-tower phase 101, #145), by situation (a SUPERVISOR_SITUATIONS word) and outcome (detected, observed, suggested, escalated, acted, refused) — one count per root cause per decision. A rising acted beside a flat detected is a remedy that keeps being pressed; escalated is work waiting on a person. |
Labels. slug on everything per-plan; state on phase_console_phases
(done · ready · in-progress · waiting · stuck); status and closed on
phase_console_plans; slug and status on phase_console_runs; rung and outcome on
phase_console_ladder_rungs_total; version and instance on phase_console_build_info.
Never a run id — a run id is minted per boarding, so labelling by one would mint a dead series every time the autopilot starts a run, and a scraper keeps those forever. Per-plan is the granularity the questions are asked at.
Three things to know before you alert on these
The names are a contract. Once you scrape this, a rename breaks your dashboard silently, weeks
later. So a family is never renamed or repurposed and a label is never added to an existing family
— new facts arrive as new families. _total is a counter and everything else is a gauge, with no
exceptions: rate() over a mislabelled family is wrong in a way nothing reports.
The money adds up in one direction only. Every session books the same dollars to its run total and to its phase record, and again to the ladder rung that caused it when one did. So:
phase_console_spend_usd_totalis the money. It is the one to sum.phase_console_phase_spend_usd_totalis the same money, seen per phase — equal in a healthy run.phase_console_ladder_spend_usd_totalis a subset of both. Adding it double-counts repair.phase_console_spend_residual_usdshould be0. A non-zero value is a run file disagreeing with itself, which is worth an alert of its own.
The three worktree families are ABSENT, not zero, until a run isolates. phase_console_worktrees,
The counters reset with the process
The ten _total families added in 5.1.0 (server/counters.ts) are process-lifetime: they count
from this console's start and a restart puts them back to zero. That is what a Prometheus counter is
— the scraper owns rate and reset detection — and phase_console_build_info already carries the
version and instance a scraper needs to tell one process's series from the next.
Two consequences worth knowing before alerting on them. A family with no data yet is ABSENT, not
zero, because most are keyed by a value that comes out of the work (a git verb, a script name, a
binary) and a zero would have to invent one. And every one of them is capped at 64 distinct label
values; past that, new values fold into other so the total stays right — a counter keyed by
anything a caller can invent is how a metrics endpoint becomes the thing that runs a machine out of
memory.
phase_console_worktree_disk_bytes and phase_console_branch_conflicted_files report only on runs
working in a checkout of their own (Give this run its own checkout, on the launch form — see
what you control), so on a console that has never isolated
anything the families do not appear at all — absent() is the alert to write, not == 0. Once a run
does isolate, phase_console_branch_conflicted_files is 0 while its branch is clean, which is a
different fact and the one worth alerting on: a non-zero value means a merge conflict is already
sitting between two live branches, and the console raises the same fact as a conflict inbox item
with a one-press remedy. phase_console_worktree_disk_bytes goes absent for one slug where du
could not answer, without taking the other slugs' samples with it.
A counter resets if you delete a run file. That is the only way these go backwards, and
rate()/increase() already handle it. Nothing here is reset by restarting the console: the
numbers are read from the run files on disk, not from process memory.
The forecast family is a line through an hour
The three phase_console_account_* gauges (control-tower phase 92, #141) are the account forecast the
account card and the account bar draw, one sample per account ID and live window — the ID, never the
login's email. The burn is the slope of a least-squares line through that window's readings of the last
60 minutes (FORECAST_WINDOW_MS), measured only once two readings lie ten minutes apart; under 0.5 % an
hour a window reads flat. The wall is where that line crosses 100 % from the current reading, and only
when it does so before the window resets — so phase_console_account_wall_seconds is ABSENT for a flat,
unmeasured or resetting-first window, never zero. An alert on "walls within two hours" is
phase_console_account_wall_seconds < 7200, which is the rule usage-climbing itself pushes on.
What is NOT here
No note, no title, no path, no owner, no session id, no prompt — a scrape is counts, slugs and money. That is what makes it safe to leave open on a laptop, and it is deliberate rather than incidental: if you need the words, the pages have them.
Nothing about cost per model either. It exists per plan (Insights ▸ What this plan cost ▸ By model, and the CSV) but a phase books to the model it ended on, so a fallback mid-run makes the per-model split a summary rather than an accounting — not a number to alert on.