AI Agent Circuit Breakers: How to Stop Retry Storms, Tool Loops, and Runaway Costs
An AI agent circuit breaker is a stateful runtime control that stops calls when a dependency, tool path, or agent behavior is predictably failing—then permits limited recovery probes instead of flooding the same failure with more traffic.
AI Agent Circuit Breaker: The Quick Answer
An AI agent circuit breaker sits between an agent and something it can repeatedly call: a model provider, retrieval service, browser, database, payment API, code executor, messaging tool, or another agent. It watches recent outcomes for a defined scope. When qualifying failures cross a policy threshold, the circuit opens. New calls are rejected locally, queued, downgraded, redirected, or sent for human review without touching the unhealthy dependency. After a cooldown, the circuit becomes half-open and admits only a small number of probes. Successful probes close it; failed probes reopen it.
The classic pattern comes from distributed systems, but agents add failure modes that ordinary HTTP clients do not capture. A tool may return HTTP 200 while giving malformed or semantically useless data. A model may keep reformulating the same impossible request. A browser agent may revisit the same blocked page. A planning loop may burn tokens while making no measurable progress. A write-capable agent may repeatedly request authority it should never receive. A technically healthy provider can still be the wrong dependency for the task.
This pattern matters because local retry rules can be individually reasonable and collectively destructive. Fifty concurrent runs that each retry three times can still create 150 extra calls against a provider trying to recover. Microsoft’s retry-storm guidance explains how frequent retries can prolong an outage, while its circuit-breaker pattern separates normal traffic, fast rejection, and limited recovery probes. The agent-specific challenge is deciding which outcomes belong in each breaker and what useful work remains possible while it is open.
Retries, Stop Conditions, Rate Limits, and Circuit Breakers Are Different Controls
Teams often use these terms as if they were interchangeable. They solve related problems at different scopes. A retry handles a single transient failure. A stop condition bounds one run. A rate limiter controls traffic volume. A timeout bounds waiting for one attempt. A bulkhead isolates capacity. A circuit breaker decides whether the next attempt should be admitted at all, using recent health state.
| Control | Question it answers | Typical scope | What it does not solve alone |
|---|---|---|---|
| Timeout | How long may this attempt wait? | One model or tool call | Repeated timeouts across later requests |
| Retry with backoff | Should this transient failure be attempted again? | One operation or run | Aggregate retry storms across many runs |
| Stop condition | When must this agent run finish, pause, or escalate? | One task execution | Shared dependency health after the run ends |
| Rate limiter | How much traffic may pass per interval? | User, tenant, service, or endpoint | Whether allowed traffic is likely to fail |
| Bulkhead | How much capacity may one workload consume? | Pool, queue, tenant, or dependency | Recovery decisions for an unhealthy path |
| Circuit breaker | Is this path healthy enough to call now? | Dependency, tool, workflow, tenant, or action class | Correct business logic and safe side effects |
The site’s guide to agentic AI stop conditions defines time, cost, step, evidence, action, and retry limits for one run. A breaker adds a different property: state that survives the failed attempt and influences later traffic. If an email tool returns an authorization failure for one tenant, that run should stop immediately. If the same connector is failing for every tenant, a dependency-scoped breaker may open. If only one customer credential is invalid, a global breaker would be too broad.
Framework ceilings are useful inputs, not complete breaker designs. The OpenAI Agents SDK can raise MaxTurnsExceeded when a run passes its configured turn limit, and it exposes model, tool, and guardrail errors. LangGraph’s recursion limit stops a graph that exceeds its step ceiling. Those mechanisms prevent unbounded execution, but they do not automatically decide whether ten subsequent tasks should avoid the same failing provider or tool.
What Should Trip an AI Agent Circuit Breaker?
The answer is not “every error.” A circuit that opens on ordinary user mistakes will flap. A circuit that counts only HTTP 500 responses will miss agent-specific failure. Start with a failure taxonomy, then assign each category a qualifying event, scope, response, and recovery test.
| Signal | Usually counts toward | Do not treat as | Likely open-state response |
|---|---|---|---|
| 429 with Retry-After | Provider or endpoint capacity breaker | A prompt-quality failure | Honor the delay; queue, route, or degrade |
| Malformed structured output | Model-and-schema compatibility breaker | A network outage | Use stricter repair once, alternate model, or manual path |
| Repeated equivalent tool call | Run or workflow loop breaker | Proof that the tool itself is down | Stop the loop; preserve trace; ask for missing input |
| Authorization denied | Identity, tenant, or action-scope breaker | A transient error worth blind retry | Fail closed and require corrected authority |
| Cost budget exhausted | Run, tenant, or workflow budget breaker | Evidence that provider health is poor | Finalize safely, defer, or request approval |
| Tool timeout after a write | Ambiguous-outcome breaker | Permission to submit the write again | Check status by idempotency key before any retry |
| Evaluator rejects several outputs | Task-model suitability breaker | A universal model outage | Change strategy, narrow task, or escalate |
Notice that a single event can route to different breakers. A 429 from one provider is a capacity signal. Repeated 429s caused by one tenant’s burst may be a tenant-level admission problem. A 429 on a high-impact write does not say whether the prior request committed. Your classification must preserve the operation’s meaning.
Avoid universal trip values copied from blog posts. Thresholds depend on baseline error rate, traffic volume, task importance, dependency behavior, recovery time, and acceptable false positives. A rule based on five failures might be useful at high volume and meaningless at low volume. Prefer a rolling failure ratio with a minimum sample size for shared dependencies; use consecutive-failure rules when even a small sequence is materially abnormal; and allow immediate opening for explicit hard signals such as a revoked credential, prohibited action, or provider instruction to wait.
Closed, Open, and Half-Open States for Agent Workflows
Closed: admit normal traffic and measure qualifying outcomes
Closed does not mean “everything is healthy.” It means the evidence does not justify blocking the path. Requests flow, but the breaker records only the outcomes relevant to its policy. A provider-health breaker may track timeouts and 5xx rates. A semantic breaker may track schema failures after one bounded repair attempt. A loop breaker may compare tool-call fingerprints and progress signals. Mixing unrelated failures into one counter makes the state impossible to interpret.
Use a rolling window or decaying measure so old incidents do not poison current health forever. Store the denominator, not only the failures. Two failures in two calls and two failures in two thousand calls should not be treated alike. Record excluded events too, such as client validation errors, cancellations, and policy denials, so operators can explain why the circuit changed—or did not.
Open: reject locally and perform a deliberate fallback
Open means calls within the protected scope do not reach the dependency. The rejection should be fast, structured, and observable. It should identify the breaker, scope, reason class, state version, next permitted action, and cooldown deadline. Do not return a generic “agent failed” message that invites an upstream orchestrator to retry immediately.
The fallback must preserve truth. A cached answer can be acceptable for a low-risk knowledge lookup if freshness is disclosed. It is dangerous for a bank balance, current permission, inventory commitment, or medical status. A cheaper model can be suitable for summarization but not necessarily for the task whose quality failures opened the original model circuit. A queue can preserve work, but only if deadlines and stale-state rules are explicit. Human review is useful when a reviewer has evidence and authority; it is not a universal overflow bin.
Half-open: send scarce, representative recovery probes
After the cooldown, half-open allows a limited probe budget. The probes should resemble real traffic closely enough to test recovery without causing a thundering herd. Limit concurrency. Prevent every application worker from deciding independently that the deadline has passed. Elect one probe owner or use shared state with an atomic lease. If the probe fails with a qualifying event, reopen immediately. If several representative probes succeed, close gradually or restore traffic in stages.
Microsoft notes that half-open protects recovering services from being flooded. That protection is even more important for agents because one admitted run can fan out into multiple model and tool calls. Probe the smallest useful unit. A health endpoint can confirm network reachability, but it cannot prove that a model can satisfy your schema or that a connector can complete a representative read. Use a safe synthetic request, read-only operation, or shadow evaluation whose side effects are contained.
A Six-Layer AI Agent Circuit-Breaker Architecture
A production design usually needs more than one breaker. Treat the system as layers with different evidence, state, and fallback behavior. A model-provider breaker should not open merely because one tool credential expired. A permission breaker should not close after a latency probe. Separating layers makes recovery safer and alerts actionable.
| Layer | Protects | Example state key | Recovery evidence |
|---|---|---|---|
| Provider breaker | Model API or region | provider:model:region | Successful representative model calls within latency and error policy |
| Tool breaker | Connector or endpoint | tool:operation:tenant | Read-only or idempotent probe succeeds |
| Workflow breaker | Task pattern or graph | workflow:version:task-class | Recorded failure cases pass after a policy or code change |
| Semantic breaker | Model-task compatibility | model:prompt-version:schema | Required output validations pass on a representative set |
| Budget breaker | Spend and scarce capacity | tenant:workflow:budget-window | Budget replenishes or authorized capacity changes |
| Authority breaker | High-impact action path | agent:tool:action-class | Permission, approval, and current-state evidence are restored |
Place admission checks outside model reasoning. The model can describe a proposed action; the runtime decides whether the circuit permits dispatch. Enforce the same decision at every route to the tool, including background workers, retries, callbacks, scheduled jobs, and manual replay. If one code path bypasses the breaker, your diagrams describe a preference rather than a control.
State location depends on scope. An in-process breaker is fast and useful for a single worker, but other workers remain unaware. Shared state in a reliable store supports fleet-wide decisions, atomic probes, and consistent cooldowns, at the cost of another dependency. Service meshes and gateways can protect network calls but usually cannot see semantic failures or agent progress. A practical design combines infrastructure breakers for transport health with application breakers for task-specific meaning.
Version the policy. Store the threshold rule, observation window, minimum sample, cooldown, probe budget, eligible errors, excluded errors, fallback, owner, and rollout state. Include a policy version in every decision event. When a threshold changes, operators should be able to compare behavior before and after the change rather than silently rewriting history.
How to Implement an AI Agent Circuit Breaker Step by Step
1. Draw the call graph and mark side effects
List model calls, retrieval, tools, subagents, queues, approvals, and external writes. Mark each call as read-only, idempotent write, conditionally repeatable, or non-repeatable. Identify fan-out: one agent step might launch several searches or parallel tool calls. If you cannot see the call graph, you cannot place a breaker where it contains the blast radius.
Connect this map to AI agent observability and the site’s OpenTelemetry tracing guide. The current OpenTelemetry GenAI conventions include structured attributes for model and tool operations and warn that recorded content may be sensitive. Capture identifiers and outcomes needed for diagnosis, while applying redaction, access control, and retention policy to prompts, tool arguments, and results.
2. Define qualifying outcomes before thresholds
Create an error classifier with explicit categories: transient transport, persistent dependency, rate limit, client input, authentication, authorization, validation, ambiguous write, semantic quality, loop/no progress, budget, cancellation, and policy denial. Decide which categories count for which breaker. Authorization failures should usually fail closed at the affected identity or action scope, not teach a global provider breaker that the provider is down.
3. Choose the state key and blast radius
A key such as provider:model may be too broad if regions behave differently. A key such as tool:tenant:user:operation may be too narrow to detect a connector outage. Start from the dependency’s independent failure domains and the safest fallback. Support a hierarchy: a global provider circuit can override regional and tenant circuits, while a tenant credential circuit need not affect other tenants.
4. Coordinate retry, breaker, timeout, and idempotency
Set a timeout for each attempt. Classify the result. If it is safely retryable, use bounded exponential backoff and jitter and honor server guidance such as Retry-After. Record every attempt in the aggregate retry budget. Invoke retries through the breaker so an open state stops them. The AWS Well-Architected guidance to control and limit retry calls emphasizes maximum attempts, backoff, jitter, and idempotency.
For writes, pair this design with AI agent idempotency. A timeout after dispatch is an ambiguous outcome. Open an ambiguity circuit for that action, query remote state using the idempotency key or transaction identifier, and refuse blind resubmission. If a multi-step workflow partially committed, use the AI agent saga pattern for compensation rather than pretending the breaker can undo side effects.
5. Design a truthful fallback contract
For each open state, specify what the caller receives: cached data with an age, alternate provider, reduced feature, draft-only mode, queue receipt, human-review case, or explicit temporary failure. Include which actions are prohibited. Do not let the model invent a substitute based on incomplete context. If routing to another model, evaluate that model for the task and preserve the reason for fallback. The existing LLM routing guide explains why lower cost and availability must still be balanced with task fitness.
6. Implement deterministic transitions and decision receipts
function admit(scope, operation, now):
state = breakerStore.read(scope)
if state.mode == OPEN and now < state.cooldownUntil:
return rejectLocally(state.reason, fallbackFor(operation))
if state.mode == OPEN and leaseProbe(scope, now):
breakerStore.moveToHalfOpen(scope)
if state.mode == HALF_OPEN and !probeBudgetAvailable(scope):
return rejectLocally("probe_budget_exhausted", fallbackFor(operation))
return allow()
function record(scope, classifiedOutcome):
state = breakerStore.atomicUpdate(scope, classifiedOutcome)
if classifiedOutcome.opensImmediately:
return open(scope, classifiedOutcome.reason)
if state.mode == CLOSED and policyThresholdExceeded(state):
return open(scope, "threshold_exceeded")
if state.mode == HALF_OPEN and classifiedOutcome.qualifyingFailure:
return reopen(scope)
if state.mode == HALF_OPEN and probeSuccessTargetMet(state):
return closeGradually(scope)
return state
This pseudocode is illustrative, not a drop-in library. Production code needs atomic state transitions, monotonic time, distributed leases, authentication, authorization, tenant isolation, durable event history, clock-skew handling, privacy controls, and explicit behavior when the breaker store itself is unavailable. For high-impact actions, missing breaker state should normally fail closed.
Interactive Breaker Policy Builder
Use this decision helper to choose a starting response pattern. It does not calculate universal thresholds. It helps identify the safety properties the breaker policy needs before you tune values with telemetry and fault tests.
The result should become a design conversation, not copied configuration. Test whether the proposed scope contains the actual fault. Verify that a fallback does not silently lower accuracy, freshness, privacy, or authority. Measure false opens and missed opens. A breaker that never opens may be decorative; a breaker that is often open may be masking a dependency or design problem that needs removal, capacity, or a different workflow.
Test the Breaker as a State Machine, Not Just an Error Handler
Unit tests should cover boundary conditions: the event immediately before the threshold, the event that opens the circuit, a request just before cooldown, the first eligible probe, concurrent probe attempts, enough successes to close, and a failed probe that reopens. Use a controllable clock. Assert the previous state, input event, next state, remote-call decision, reason code, cooldown, and emitted telemetry.
Then run fault injection at workflow level. Return rate limits with and without Retry-After. Add latency without errors. Make a model produce valid JSON with missing required meaning. Return the same tool error under changing wording. Time out a write after the server commits it. Revoke a credential during a run. Fail the breaker store. Start many workers at the cooldown boundary. Confirm that the system does not create a probe stampede.
| Test | Expected state behavior | Critical assertion |
|---|---|---|
| Transient read timeout, then success | Remain closed if policy allows bounded retry | Backoff and jitter occur; aggregate retry budget is updated |
| Persistent provider failures across workers | Open shared dependency circuit | Later calls fail locally without reaching provider |
| Repeated equivalent tool calls | Open run or workflow loop circuit | Trace preserves fingerprints and last new evidence |
| Validation errors from user input | Do not open provider circuit | User receives correction request; provider health is unchanged |
| Write times out after commit | Open ambiguity path | Status lookup precedes any second write |
| Cooldown expires under load | Enter half-open with scarce probes | Only leased probe traffic reaches dependency |
| Fallback model returns lower-quality result | Keep task-specific semantic circuit open | No silent delivery as equivalent quality |
| Breaker store unavailable | Follow declared fail-open or fail-closed rule | High-impact actions do not proceed on unknown state |
Test recovery, not only containment. A system that opens correctly but never closes creates a durable outage. A system that closes after one superficial health check may flap. Rehearse manual override with time-limited authorization and reason logging. Verify that closing a circuit does not replay stale queued tasks without refreshing data, permissions, approvals, deadlines, and user intent. The AI agent checkpointing guide explains why resumed work needs a recovery contract rather than a blind restart.
Operate AI Agent Circuit Breakers With Useful Metrics and Runbooks
Record state transitions as first-class events. Useful fields include breaker name, scope key, policy version, previous and next state, qualifying reason, observation window, sample size, failure ratio, cooldown, probe lease, fallback selected, remote call attempted, trace ID, tenant where appropriate, and operator override. Keep sensitive prompt and tool content out of high-cardinality metric labels. Use protected traces for details.
- Open rate by breaker and reason: reveals unstable dependencies and noisy classification.
- Short-circuited calls: estimates avoided load, latency, and cost without pretending every rejected call would have failed.
- Time open and recovery attempts: shows whether cooldowns and probes match real recovery.
- Probe success and re-open rate: detects premature closure and flapping.
- Fallback outcome quality: measures whether degraded service is genuinely useful and safe.
- False-open review: samples calls that would likely have succeeded, with careful counterfactual limits.
- Missed-open incidents: captures failure spread before the threshold reacted.
- Queue age and stale-task discard: prevents deferred work from becoming silently incorrect.
- Cost per verified outcome: connects spend protection to delivered value, not raw token reduction.
Alert on impact, not every state transition. A tenant credential circuit can create a targeted support case. A provider circuit affecting all production traffic may page operations. A semantic circuit opened during a controlled canary may block rollout without waking an on-call engineer. Define ownership: who can change policy, who receives alerts, who approves manual closure, and who communicates user-facing degradation.
Write the runbook before production. It should explain how to inspect the triggering traces, confirm scope, freeze risky replay, preserve queued work, verify downstream effects, test recovery, close gradually, and review root cause. Connect serious events to the AI agent incident response runbook. A breaker contains one class of failure; it does not replace incident response, compensation, customer communication, or post-incident learning.
Common Circuit-Breaker Mistakes in Agent Systems
Counting every failure in one bucket
Transport failure, invalid input, missing permission, semantic mismatch, and user cancellation have different causes and recovery paths. One counter turns the breaker into an unexplained outage generator.
Using per-process state for a fleet-wide outage
Each worker opens at a different time and sends its own half-open probe. The dependency receives a synchronized wave just as it recovers. Use shared state or coordinated probe leases when the failure domain is shared.
Retrying outside the breaker
An upstream loop catches the circuit-open response and immediately calls again, or a job queue redelivers without respecting cooldown. Standardize the structured rejection and make every retry path circuit-aware.
Sending writes through a fallback without equivalent guarantees
An alternate tool or model may not support the same idempotency, authorization, validation, or audit trail. Fallback is a new execution path that needs its own review—not a synonym for resilience.
Probing with destructive operations
A recovery test should be read-only, synthetic, isolated, or safely idempotent. Do not test whether payments work by charging a real customer or whether publication works by posting publicly.
Hiding degradation from users
A cached or lower-quality answer should not be presented as current and equivalent. Tell users when a function is temporarily limited, what work was saved, and whether they need to retry, wait, or review.
Raising thresholds until alerts disappear
A noisy breaker may indicate bad classification or poor baseline modeling. Raising the limit can convert operational friction into missed containment. Review traces, improve scope, and test changes against recorded incidents.
Treating the breaker as the whole safety system
Circuit breakers reduce repeated failure. They do not prove output correctness, enforce least privilege, validate evidence, secure credentials, obtain human approval, or reverse harmful actions. Combine them with evaluation, observability, idempotency, checkpoints, approvals, and governance.
A Practical Production Rollout
Begin in observe-only mode. Compute the state and emit the decision receipt, but do not block. Compare proposed opens with actual outcomes. Review whether the scope was correct, whether the fallback would have helped, and how many calls would have been short-circuited. This establishes a baseline without pretending historical data perfectly predicts future incidents.
Next, enforce on one low-risk read path. Use conservative scope, a tested fallback, visible user messaging, and an operator override that expires automatically. Run fault injection. Confirm that dashboards distinguish local rejection from remote failure. Verify that queued work does not exceed its useful lifetime.
Then expand to write paths only after idempotency, status reconciliation, approvals, and compensation are ready. For high-impact actions, prioritize authority and ambiguity breakers before availability optimization. A failed read is inconvenient; a duplicated refund, message, deployment, or permission grant can create harm.
Roll policy changes through canaries. The site’s AI agent canary rollout checklist provides a broader release pattern. Track breaker opens, user-visible failures, fallback quality, operator interventions, and downstream corrections. Promote only when the new policy reduces total outcome risk rather than merely improving availability graphs.
Frequently Asked Questions About AI Agent Circuit Breakers
What is an AI agent circuit breaker?
It is a stateful runtime control that watches qualifying failures for a defined scope, blocks new calls when the path is predictably unhealthy, and permits limited recovery probes after a cooldown. It can protect model providers, tools, workflows, budgets, semantic quality, or high-impact action paths.
How is a circuit breaker different from retries?
Retries attempt a failed operation again because the fault may be transient. A circuit breaker prevents attempts when recent evidence says they are unlikely to succeed or safe to repeat. Use bounded retries through the breaker, with backoff, jitter, error classification, and idempotency where needed.
How is a circuit breaker different from an agent stop condition?
A stop condition changes the state of one run when it reaches a time, cost, step, evidence, risk, or retry boundary. A breaker usually retains health state across attempts or runs and decides whether later traffic may call the protected path. The two controls work together.
Should semantic failures open a circuit?
Sometimes. Repeated schema failures, missing required meaning, unsupported citations, or evaluator rejection can indicate that a model-prompt-schema combination is unsuitable. Keep semantic failure separate from transport health, scope it to the task configuration, and verify recovery with representative evaluations.
Where should breaker state be stored?
Use local state for a local failure domain and shared state for a shared dependency or fleet-wide decision. Shared state needs atomic updates and coordinated probe leases. High-impact actions need an explicit rule for what happens when state cannot be read; failing closed is often safer.
How long should the cooldown be?
There is no universal value. Base it on dependency behavior, server recovery signals, business deadlines, traffic, false-open cost, and fault tests. Honor Retry-After when applicable, use longer cooldowns after repeated failed probes when justified, and avoid synchronized recovery traffic.
What should happen while the circuit is open?
Return a deliberate fallback: truthful cached data, an evaluated alternate, reduced capability, draft-only mode, deferred queue, human review, or explicit temporary failure. The response must preserve safety, freshness, authority, privacy, and user understanding.
Can a circuit breaker prevent runaway agent costs?
It can contain repeated calls, tool loops, and fleet-wide retry amplification. A budget breaker can stop or defer work when a spend boundary is reached. It should complement per-run limits, rate limits, queue controls, routing, and evaluation of cost per verified outcome.
Do circuit breakers replace human approval?
No. A breaker decides whether a path is healthy or permitted under a runtime policy. Human approval addresses authority and judgment for particular actions. An open circuit may route a case to review, but the reviewer still needs evidence, competence, time, and the power to act.
Make Failure Containment a First-Class Agent Capability
AI agents turn one request into a sequence of model calls, tool use, state changes, and feedback. That creates leverage, but it also amplifies ordinary failures. A provider slowdown becomes a retry storm. An ambiguous tool result becomes a loop. A validation miss becomes repeated model spend. A timed-out write becomes a duplicate action. A permission problem becomes an unsafe attempt to find another route.
An effective AI agent circuit breaker interrupts that amplification. It classifies failures, remembers health at the right scope, rejects locally when success is unlikely, provides a truthful fallback, and restores traffic through scarce representative probes. It works with—not instead of—timeouts, bounded retries, stop conditions, rate limits, bulkheads, idempotency, checkpointing, observability, approval, and incident response.
The most important implementation decision is not the threshold number. It is the contract around the number: what counts, what scope is protected, what happens when the circuit opens, how recovery is proven, and who owns the result. Build that contract from measured behavior and fault tests. Then keep the decision visible in traces and understandable to users.
Evidence note: This guide synthesizes current first-party documentation from Microsoft Azure Architecture Center, AWS Well-Architected, OpenAI Agents SDK, LangGraph, OpenTelemetry, and OWASP. Illustrative pseudocode and design examples are not universal operating thresholds; validate them against your workload.
