Production AI needs a runbook, not just a dashboard

A dashboard can tell an operator that latency rose at 10:42. It does not say whether customers received wrong answers, which configuration created them, whether retrying is safe, or who can shut off an automated action. During an incident, those are the questions that consume time.

An AI request may cross input processing, retrieval, a model provider, policy checks, and external tools. The model can return a valid response that is wrong for the workflow. A retry can recover a timeout or repeat a payment. A useful runbook gives these conditions a specific trigger, a safe first action, enough evidence for diagnosis, and an owner who has practiced the procedure.

A dashboard answers a different question

Dashboards support inspection and trend analysis. A runbook describes how to reach a known outcome when a familiar condition occurs.

AWS’s runbook guidance calls for the desired outcome, required tools and permissions, error handling, escalation, an owner, and validation by another team member. That is a stronger standard than a dashboard link followed by “investigate.”

A runbook handles a condition the team understands. A playbook supports open-ended investigation of an unfamiliar failure. The first minutes of a known incident should use the runbook.

Define incidents in workflow terms

Start with user or business impact. “Model quality is down” is too vague for paging. “Verified order status answers fell below the release threshold for 15 minutes” gives the operator a condition to confirm. “The refund tool executed without its required approval” may justify an immediate stop after one event because the consequence is different.

Use a small severity matrix based on scope, consequence, and reversibility. A slow internal draft assistant can often wait for business hours. A system that sends incorrect notices or changes records needs faster containment. Write the rule in observable terms and state who can raise or lower the severity.

For each incident class, record the trigger window, actions that must stop, available fallback, and the owners of technical recovery and the business response. Add specialist contacts where the impact requires them. An API error rate may be the cause, but severity depends on what happened to the work.

Capture enough evidence to reproduce a run

An operator should be able to move from an affected record to its execution path. Carry a run ID across queues, model calls, retrieval, policy checks, and tools. Record version references for the application, model, instructions, retrieval index, tool schema, and policy configuration.

For each step, capture status, latency, retry count, and a safe input and output reference. Tool traces should include validated arguments, approval state, and result. Preserve stable identifiers for retrieval sources and retain human corrections under the applicable data policy.

OpenTelemetry context propagation is a vendor-neutral way to correlate traces, metrics, and logs across service boundaries. Its documentation also warns against putting credentials or personal data in baggage. AI traces need the same care.

Set retention and access rules. Use hashes or controlled references when raw content is unnecessary, and redact secrets before export.

Reproduction should happen in a sandbox with external writes disabled or mocked. The packet needs the versioned configuration and recorded dependency results, not a button that repeats the original side effect. Some live conditions cannot be replayed exactly, but a stable record lets the team separate a model behavior change from stale retrieval, malformed input, or a failed dependency.

Contain first, diagnose second

Do not make an operator prove the root cause while a harmful path remains active. The first runbook steps should limit impact and preserve evidence. Diagnosis follows once the service is in a known state.

Containment may pause one tool, disable an AI route, lower an agent’s action budget, or send work to a manual queue. State what happens to in-flight work and whether it can be retried. If a credential must be revoked, document the downstream effects and required authority.

Preapprove low-risk containment where possible. Higher-impact actions may require a workflow owner or incident commander. Put the contact and decision rule in the runbook.

Evidence preservation is part of containment. Mark the affected time window, snapshot configuration references, and protect the relevant traces from routine expiration. Record every manual change in a live incident log. Otherwise, the response itself can erase the conditions the team needs to understand.

Write procedures around actual failure modes

Begin with failure modes already present in tests, dependency contracts, or prior incidents. Avoid a single “AI is broken” document. Operators need a short path for a recognizable condition.

Condition	Useful signal	First safe action	Escalate when
Model provider timeout or throttling	Provider status, timeout rate, queue age	Stop automatic retries at the set limit and route eligible work to fallback	The queue threatens the workflow’s recovery target
Retrieval is stale or empty	Index freshness, no-result rate, source age	Disable answers that require the affected source and preserve the query set	The system used stale evidence for an external action
Quality falls below its release threshold	Scored production sample, correction rate, critical failure count	Return the affected slice to review and pin the last known-good configuration	A policy failure appears or more slices are affected
A tool performs the wrong action	Action ledger, approval mismatch, user report	Disable that tool, retain action IDs, and block automatic replay	Records, money, permissions, or external messages may be affected
An agent exceeds its expected loop or spend	Step count, repeated calls, token and time budget	Enforce the cutoff and return the case to a person	The cutoff fails or the behavior appears across configurations

Tailor the table to the application. Keep tool names, dashboard queries, commands, and fallback destinations in the detailed runbook.

Page on symptoms that require action

The Google SRE monitoring chapter separates symptoms from causes and argues that every page should be actionable. Use that discipline for AI systems. Page when users are being harmed or a serious condition is imminent, not whenever an internal metric looks unusual.

Traditional signals still cover much of the service: traffic, latency, errors, and saturation. Add semantic signals that represent the workflow. Depending on the system, these may include schema failure, unsupported claims, missing evidence, tool denial, approval bypass, user correction, unusual escalation, or cost per completed case.

Calibrate automated quality scores against domain review and page only when the result has an understood operational meaning. A low-volume service may need scheduled sampling. One verified, high-consequence event can still be an incident when an aggregate metric looks normal.

Every alert should carry the affected service and environment, severity, time window, current owner, dashboard query, runbook link, and the latest deployment or configuration change. If the response is always the same safe mechanical step, automate that step and notify a person of the result. Save pages for decisions that need judgment.

Recover to a known state

“Restart the service” is rarely a complete AI recovery plan. The bad behavior may live in the model selection, prompt, retrieval index, tool definition, policy, or upstream data. Version each piece and define which combinations are known to work.

Recovery may pin the previous configuration, rebuild an index, remove one tool, or keep the AI path disabled while manual work continues. Document prerequisites and expected time for each option.

Use idempotency keys, an action ledger, or a comparable control so operators can distinguish work that failed before an action from work that completed but lost its acknowledgment. Replaying every trace can duplicate messages or record changes. The workflow owner should approve bulk reprocessing.

Verify recovery from the outside. Submit a safe synthetic case or a controlled sample, confirm the workflow result, and check that queues and error rates return to their targets. If releasing a changed configuration, use limited exposure. Google’s canarying guide recommends comparing a time-limited production slice with the control and integrating that evaluation into the release process.

Do not close the incident as soon as a graph turns green. Account for affected work, communicate the current state, and confirm that the fallback queue has an owner.

Make escalation explicit

A routine incident needs a technical responder and a workflow decision owner. Larger incidents may also need a commander who tracks the overall state while another person changes the system. Google’s incident-management guidance separates command, operational work, communication, and planning so responders do not make conflicting changes.

Write escalation triggers in terms of impact and authority: suspected data exposure, an unapproved external action, failed containment, or a missed recovery target. List primary and backup contacts, including the provider’s support route when needed.

The NIST Generative AI Profile recommends incident response and recovery plans that cover the AI value chain, downstream contacts, deactivation criteria, and escalation to the organization’s risk authority. Use the parts that match the system’s consequence. Even a small internal assistant needs someone authorized to pause it.

Prepare communication before an incident. State who updates users, what information can be shared, and how often the status is reviewed. The message should describe impact, workaround, and the next update time without guessing at a root cause.

Test the runbook before it is urgent

Give the document to someone who did not write it and ask them to execute the procedure safely with only the stated access. Missing permissions and private knowledge will surface quickly.

Use a tabletop for failure modes that cannot be triggered safely. Check whether the backup owner can find the document when the usual collaboration system is unavailable.

Keep the runbook in version control or another central store with an owner, last-tested date, and review trigger. Update it when a model, prompt, tool, policy, data source, alert, or team boundary changes. Link post-incident actions back to the exact step that failed or was missing.

Start with the highest-consequence credible failure. Write its detection rule and the first safe action, then have another operator run the procedure. That exercise will produce a more useful operations plan than another dashboard panel.