How to Recover AI Agents After a Failure Without Duplicating Effects
Image generated by the Inferama Editorial Team
01

A failure does not always mean the action did not happen

An AI agent can be interrupted after sending a request to a tool but before saving the response. In that interval, the external system may already have created a ticket, sent a message, or changed a record. Losing the response does not prove that the effect failed. If the restarted agent repeats the action without checking, it may duplicate the change.

That is why it helps to distinguish the process state from the state of the external operation. An infrastructure error can cut off communication; a logic error can produce an unsuitable decision; and a tool can reject an input or respond late. These are different causes, and they do not automatically warrant the same recovery.

Conversation history and execution logs can help reconstruct what happened, but they are not necessarily proof that the target system did—or did not—apply a change. The operation ID returned by that system, a reliable status query, or a persisted receipt provides a better basis for deciding whether to continue.

Durable execution stores the intent, confirmed steps, and results needed to resume a workflow outside the process. If the process dies, a new worker can start from the last confirmed checkpoint instead of asking the model to choose the plan again. Resuming means continuing from a known state; retrying means attempting a specific operation again.

02

What state to preserve and how to classify it

Save what is needed to recover the execution without relying on the worker’s temporary memory. At a minimum, record the execution ID, workflow version, decision that led to the call, normalized arguments, stable operation identity, and the provider’s result or receipt. If the external system returns an identifier, preserve that too.

Persist a model decision before executing the tool. Then save each result before starting the next step. This means a restart does not have to generate the plan again and can distinguish a pending call from a confirmed operation. A checkpoint records durable state; it does not guarantee that an external API applied its changes atomically.

The `unknown` state is useful when a response was lost or the process failed between the external effect and the local record. It is not equivalent to `failed`: it means that what happened still needs to be resolved. A provider query may establish that the operation succeeded or failed; if no reliable query exists and repeating it carries risk, it should remain pending reconciliation.

Compatibility matters too. An old checkpoint may no longer make sense if tool definitions or workflow rules have changed. Associate each execution with a compatible version, and reject stale writes through concurrency control. In deployments with concurrent workers, a lease and fencing mechanism can help prevent two processes from advancing the same state, but they must be correctly supported by the storage layer.

Useful recovery states

Observed stateWhat can be concludedNext step
The tool has not been sentThere is no indication that the effect was requestedSend the operation using the identity already persisted
The request was sent, but no response arrivedThe effect may have occurredQuery the destination or look up the receipt
The external effect occurred, but the local checkpoint is missingThe journal is out of dateReconcile using the same key; save the recovered result
The receipt is persistedThe step is confirmed in the journalReuse that result and continue
No reliable query exists, and repeating could cause harmThe outcome is still unresolvedMark it unknown and pause for review
03

Retries with clear limits and conditions

A retry makes sense when the failure appears transient and repeating the operation is safe. A busy service or an interruption before the request was sent may allow another attempt; denied authorization, invalid arguments, or a key conflict usually require fixing the cause rather than trying again. Classify the error before applying a retry policy.

Exponential backoff increases the wait between successive attempts and can include jitter to prevent many workers from calling the same service again at once. This technique controls when to try again; it does not guarantee that the service is available or that the action is correct. Set a maximum number of attempts and a total time limit, and record the reason for each wait.

A stable idempotency key links retries of the same intent when the provider recognizes that key and enforces the corresponding contract. Keep the key unchanged across attempts for the same operation. Generating a new key after a timeout can turn a retry into a second operation. If the arguments change, do not silently reuse a key associated with different content.

When a provider does not offer idempotency, look for a status query or business reference that can be used to reconcile the outcome. If neither option is reliable, stopping may be safer than retrying. Repeating a read-only tool call and repeating a call that sends money, email, or permission changes do not carry the same risk.

Decide before retrying

SituationPrudent decisionCondition
Transient failure before sendingRetry with a bounded delayThe operation has not yet been requested
Timeout after sending a writeReconcile before retryingQuery by key or check the destination’s state
Invalid input or denied authorizationStop and fix the causeDo not try to solve it by increasing the attempt count
Same intent and an idempotent providerRetry with the same keyConfirm the provider’s time window and semantics
Unknown outcome and high-risk effectPause and escalateNo reliable verification is available
04

External effects: idempotency is not rollback

Idempotency and rollback solve different problems. Idempotency aims to prevent repeated attempts for the same operation from producing duplicate business effects, if the executor and provider honor that contract. Rollback attempts to reverse an effect that has already been applied. An idempotent retry does not undo a change.

Not every action can be reversed, either. A reservation may be released; a payment may require a separate refund; an email that has been sent cannot be removed from the recipient’s inbox. In many cases, the alternative is an explicit compensating action, which is a new operation and can fail or require approval.

Before automating a write, identify its effect, how to verify it, and any possible compensation. If an action cannot be reliably undone, limit its scope, require approval where appropriate, or stop the workflow when its status is ambiguous. Do not present a broad data restore as a selective rollback: restoring a backup can affect valid changes made afterward.

A local test pattern can simulate a crash between applying an effect and saving its receipt. The simulation can check that the runtime resumes the plan and queries the external record before duplicating the operation. By itself, it does not prove that a real provider offers idempotency, that distributed operations are atomic, or that two concurrent workers are coordinated.

05

Escalate to a person with useful context

Human escalation should not mean forwarding an unexplained exception. The person receiving the case needs to know what was attempted, what result was confirmed, what remains unknown, and what risk continuing would carry. Include the recoverable state and the pending decision, not just the user’s text.

A practical handoff package can include the execution ID, interrupted step, tool and version, attempts made, classified error, operation key, available receipts, and reconciliation queries performed. Add any current approvals and conditions related to safety or budget that could prevent continuation.

Redact sensitive arguments and restrict log access according to applicable rules. The agent should describe uncertainty concretely—for example, “the request was sent, but the outcome is unconfirmed.” It should not claim that the change happened or did not happen if the status remains unresolved.

The reviewer needs a clear next action: query the destination system, authorize compensation, correct the input, or close the case. If the reviewer decides to retry, record who made the decision, what evidence informed it, and which key was used. That preserves traceability without delegating authority to resolve a high-risk operation to the model.

Handing off an uncertain execution

Use the same context so the team can investigate and decide without reconstructing the history from scratch.

  1. 01Identify the execution, step, and workflow version.
  2. 02Summarize what was sent, how many attempts were made, and what the tool returned.
  3. 03Separate confirmed results from unknown states.
  4. 04State which query or approval is missing and what effects a retry could cause.
  5. 05Record the human decision and return control to the workflow only when the state is resolved.
06

Observability and recovery testing

Observability should make it possible to reconstruct an execution and distinguish model, tool, and infrastructure failures. Record state transitions, the start and result of each call, latency, classified errors, retries, checkpoints, and escalations. Correlate these events with an execution ID and workflow version, without indiscriminately storing sensitive data.

No single metric universally summarizes the resilience of every agent. A team can track how many executions recover, how long unknown states take to resolve, how many cases require intervention, and how many duplicate effects are detected. Define each indicator with its denominator and time window; a recovery rate without those definitions can be difficult to interpret.

Test failure paths, not just the model’s final output. Simulate a crash before sending, a lost response after acceptance, a transient error, a key reused with different arguments, and two workers competing for the same execution. Check that a confirmed step is not repeated and that an unknown state is not presented as confirmed success or failure.

Tests using local files or mock providers verify code decisions under those simulated conditions. They do not automatically validate a production provider’s semantics, database durability, or the effectiveness of distributed coordination. Check the relevant guarantees against the service documentation and test the specific configuration the agent will use.

What to observe in each execution

SignalQuestion it helps answer
State transitionsAt which step was execution interrupted, and what was the last confirmed checkpoint?
Error and toolWas it a rejection, outage, timeout, or logic failure?
Attempts and delaysHow many times was the operation repeated, and what limits applied?
External receipt or identifierCan the effect be verified in the destination system?
Unknown state and escalationHow long did reconciliation take, and who resolved the case?
Workflow versionIs the checkpoint compatible with the code recovering it?
07

A practical recovery protocol

A clear protocol orders the recovery: load the last confirmed checkpoint, check version compatibility, identify the interrupted operation, and classify its outcome. If it was not sent, it can be executed using the persisted identity. If it was sent and no response arrived, query the destination or receipt before deciding. If the operation is confirmed, save that result and continue from the next step.

The policy should specify when retries are allowed, their limits, and which errors require stopping. It should also define who can resolve an unknown state and when additional approval is needed. An explicit stop rule protects against loops and prevents a model from treating silence as permission to repeat.

Test the protocol with a controlled fixture before connecting it to real actions. Then separately validate provider behavior, concurrency conditions, checkpoint retention, and escalation procedures. A durable workflow can take on some of the journaling and replay, but it does not automatically make an external API atomic or exactly-once.

Reliable recovery does not mean every failure can be repaired automatically. It means the system preserves what is confirmed, represents what is unknown honestly, and limits the next step to what can be justified. When there is no reliable way to determine the effect and repeating could cause harm, pausing and reconciling are part of correct operation.

08

Keep exploring

Sources consulted

03

Corrections and transparency

If you spot incorrect or outdated information, send us a correction with the page and source we should review.

Submit a correction