TL;DR — Key Takeaways
-
Retrying is not always safe: A timed-out operation may have succeeded without confirmation. AI agents must verify outcomes before repeating actions that could create duplicate transactions or unintended changes.
-
Verified checkpoints enable safer recovery: Effective recovery preserves confirmed facts and completed actions while identifying uncertain results, invalid assumptions and decisions that may need to be reconsidered.
-
Autonomy requires intelligent failure handling: Production AI agents need clearly defined rules for retrying, resuming, replanning, restarting and escalating when the consequences of continuing are uncertain.
In production support, one of the first lessons you learn is that “try again” is not always a recovery strategy.
I have worked with real-time, data-intensive applications where a single workflow can depend on several systems, integrations and automated steps. When something fails halfway through, the difficult part is often not getting the process running again. It is understanding what already happened before the failure.
Did the downstream system receive the request? Did it process the request but fail to return confirmation? Did only part of the workflow complete? Is it safe to repeat the action?
These questions are familiar to anyone supporting production software. As AI agents take on longer workflows and interact directly with enterprise systems, these questions become even more important.
An agent needs more than the ability to retry. It needs to know what is safe to preserve after something goes wrong.
A Timeout Does Not Always Mean Nothing Happened
Consider a common production scenario.
An application sends a request to another system—the request times out, so the application never receives confirmation. From the application’s point of view, the operation appears to have failed.
But that does not necessarily mean the downstream system did nothing. It may have completed the request successfully before the connection was interrupted.
Repeating the request without checking can create a duplicate action.
Traditional applications have ways of dealing with this problem. Engineers use techniques such as idempotency, reconciliation and state verification to determine what actually happened before deciding what to do next.
An AI agent interacting with several tools faces the same underlying problem, with an additional complication: it may continue making decisions based on what it believes happened.
A generic “retry three times” rule is not enough.
Before retrying, the agent may need to determine whether the previous action failed, succeeded or ended in an unknown state.
Retry and Recovery Are Not the Same Thing
It is useful to separate several actions that are often grouped together as recovery.
Retry means repeating the failed operation while keeping the current state. This can make sense for a temporary network problem when the operation is known to be safe to repeat.
Resume means continuing from the last verified point. If earlier steps completed, there may be no reason to execute them again.
Replan is necessary when the problem is not a failed tool but a failed approach. The information the agent collected may still be useful, but it needs another way to accomplish the objective.
Restart goes further. If the agent’s working context or assumptions can no longer be trusted, continuing from the same state may simply reproduce the problem. The task may need to be reconstructed from reliable sources.
There is also another option that should not be overlooked: escalate.
If the agent cannot determine whether a consequential action succeeded, automatically continuing may be riskier than asking a person to review the situation.
These are very different responses to failure, and choosing among them should depend on what the system knows.
The Last Completed Step May Not Be a Safe Checkpoint
Production systems have also taught me to be careful with the word “completed.”
Imagine a workflow with five steps. The first three complete successfully. The fourth produces an ambiguous result. The fifth depends on the assumption that step four succeeded.
The workflow may have progressed through all five steps, but the last trustworthy point may still be step three.
That distinction matters for AI agents.
Saving an agent’s state after every action does not automatically create a safe recovery point. A useful checkpoint should represent a verified state, not simply the most recent state.
For consequential actions, the system should be able to answer a few basic questions:
- What action was attempted?
- Was its outcome confirmed?
- Did it change an external system?
- Is that action safe to repeat?
- Which later decisions depended on its result?
Without those answers, resuming an agent after a failure becomes guesswork.
Preserve Verified Facts, Not Failed Assumptions
Restarting everything after every problem is not practical either.
An agent may already have collected useful information, validated records or completed actions that should not be repeated. Throwing away all of that work can be inefficient and, in some situations, create additional risk.
A better recovery design separates verified facts from unverified assumptions.
Suppose an agent confirms several pieces of information from reliable sources and then chooses an approach that fails. The failure of that approach does not make the information it collected incorrect.
The agent should be able to replan while preserving what has already been verified.
This becomes particularly important because an agent’s working state can contain very different types of information: confirmed observations, tool responses, assumptions, intermediate decisions and the current plan.
Those things should not automatically receive the same level of trust.
If an assumption turns out to be wrong, the agent should be able to discard it without losing everything else it learned during the task. If a tool result is uncertain, later decisions that depend on it may need to be reconsidered as well.
Recovery is therefore not just about restoring state. It is about deciding which state deserves to be restored.
Design the Failure Path Before Granting Autonomy
AI agent demonstrations naturally focus on the happy path. A user provides an objective, the agent selects tools, performs several steps and returns a result.
Production environments are rarely that predictable.
Connections time out. Permissions change. External systems have become unavailable. Responses can be ambiguous. An operation may succeed even when confirmation fails. A plan that initially looked reasonable may stop making sense halfway through execution.
These are not unusual edge cases. They are part of operating software that depends on other software.
Before allowing an AI agent to make consequential changes, teams should therefore design what happens when the workflow does not go according to plan.
The recovery model should distinguish between when an agent can safely retry, resume, replan, restart or escalate. More importantly, those decisions should be based on verified information about the state the previous attempt left behind.
The question after a failed agent action should not simply be:
“Can we try this again?”
A better question is:
“What from the previous attempt do we know is safe to carry forward?”
As AI agents take on more operational responsibility, knowing how to continue after failure will matter.
Knowing when not to continue may matter even more.
