Skip to main content

Health endpoints

Liveness only answers whether the process can serve HTTP. Readiness answers whether its role can currently serve useful traffic. Configure process restarts from /healthz and traffic admission from /readyz. The all-in-one listener routes /healthz to the control plane and /readyz to the data plane. This makes the public liveness check independent of Postgres while readiness covers the inference path. A data plane that rejects a newer manifest keeps serving its last accepted snapshot and remains ready.

Metrics

Configure OTLP export, Prometheus scraping, and alerting with the OpenTelemetry guide.

Request correlation and logs

Every control-plane and data-plane response carries an airmux-generated UUIDv7 in X-Request-ID. A caller-supplied value is replaced. Data-plane usage events use the same request ID, including all attempts in a fallback chain. Production logs are newline-delimited JSON with timestamp, level, logger, event, and message. Logs emitted in a request context also contain request_id; event-specific bounded fields appear at the top level. Start an investigation from the response header, then search application logs and usage events for that value. Development mode uses human-readable logs instead.

CLI diagnostics

Use doctor first to distinguish local profile, control-plane authentication, and gateway readiness failures. gateways list needs instance-scoped data-planes.read permission, so run it with an instance-scoped profile or management key.

Backups

Back up these as one recovery point:
  1. Postgres
  2. Runtime management credential files
  3. Provider secret files
  4. Each gateway’s identity, cached bundles, and SQLite usage outbox
Restrict backup access because the default file secret store contains plaintext provider credentials. Test restoration on isolated infrastructure.

Updates

Compose completes its migration and catalog jobs before starting the control plane. A direct control-plane restart does not run either job. The process refuses to start unless the database is at the expected schema revision. A split gateway can keep serving an accepted cached bundle while the control plane is unavailable. Because pre-1.0 releases do not promise cross-version compatibility between planes, deploy one image digest across all components. Follow Upgrade and rollback to restore the database and state together if verification fails.

Common failures