Your AI Agents Are Not Deterministic. Stop Writing Retry Logic As If They Are.

There is a subtle, expensive, and surprisingly widespread mistake spreading through enterprise backend teams right now. It does not show up in your observability dashboards. It does not trigger your circuit breakers. It does not throw a stack trace. It just silently corrupts outcomes, duplicates side effects, and makes your distributed workflows subtly wrong in ways that are extraordinarily hard to reproduce in staging.

The mistake is this: teams are treating AI agent calls as idempotent because they assume they are deterministic. They are neither. And conflating those two concepts, which are genuinely distinct properties, is quietly poisoning retry logic across agentic workflow orchestration stacks in H2 2026.

I want to be precise about why this matters, where the confusion originates, and what a correct mental model actually looks like. Because the longer this conflation goes unaddressed, the worse the distributed systems debt becomes.

First, Let's Be Exact About What These Words Mean

These two terms get used interchangeably in engineering conversations, Slack threads, and even architecture review docs. They should not be.

Determinism is a property of computation. A deterministic function, given the same inputs, always produces the same output. It is a statement about predictability of result.

Idempotency is a property of operations. An idempotent operation, applied one or more times, produces the same effect on system state as applying it exactly once. It is a statement about safety of repetition.

These are related but orthogonal. A function can be deterministic without being idempotent (think: a pure function that appends a record to a database). A function can be idempotent without being deterministic (think: a DELETE by ID that always leaves the row absent, regardless of what the row contained). And critically for our purposes: an AI agent call is typically neither deterministic nor idempotent.

That second part is the part teams are missing.

How Agentic Workflows Break the Classic Retry Contract

Traditional distributed systems wisdom, codified across decades of work in Temporal, Apache Kafka, AWS Step Functions, and similar orchestration layers, assumes a reliable retry contract. If an operation fails or times out, you retry it. The safety of that retry depends on the operation being idempotent. Send a payment? Use an idempotency key. Write a record? Upsert by a stable ID. Call a downstream API? Expect the vendor to honor your idempotency header.

This contract works beautifully for deterministic, stateless service calls. It starts to fray at the edges with probabilistic systems. And it breaks almost completely with LLM-backed AI agents, for several compounding reasons:

• Temperature and sampling introduce stochasticity. Even at temperature zero, frontier models in 2026 are not fully deterministic across infrastructure restarts, model version patches, or load-balanced inference clusters. The same prompt, retried, can produce a meaningfully different output.
• Agents take actions, not just produce outputs. An agentic step might call a tool, write to a database, send an API request, or trigger a downstream workflow. If the agent step is retried after a partial failure, those side effects may already have occurred. The agent has no awareness of this.
• Context windows are stateful. Multi-step agentic workflows accumulate context. A retry that re-injects a stale or partial context window does not produce a semantically equivalent continuation. It produces a different agent with a different understanding of what has happened.
• Orchestration timeouts do not map cleanly to agent completion. An agent step might time out at the orchestration layer while the underlying LLM inference is still running, or while a tool call is mid-execution. The retry fires into an already-in-progress operation.

The result is a class of bugs that are genuinely novel. Not race conditions in the traditional sense. Not data corruption in the traditional sense. Something more like semantic drift under repetition, where each retry moves the workflow further from its intended trajectory.

Where the Conflation Actually Happens in Practice

Let me describe the failure pattern I see most often in enterprise architecture reviews and post-mortems.

A team builds an agentic workflow using an orchestration framework, whether that is Temporal, LangGraph, Prefect, Dagster, or a homegrown state machine. They model each agent invocation as a workflow activity or task. They configure retry policies on those activities, because that is the correct thing to do for resilient distributed systems. They set retries to three or five attempts with exponential backoff. They feel good about this.

What they have implicitly assumed is that retrying an agent call is safe because, they reason, "the agent will just figure it out again." This assumption smuggles in two sub-assumptions: first, that the agent will produce the same decision on retry (determinism); and second, that re-running the agent step has no net-new side effects (idempotency). Both assumptions are wrong.

The most common concrete failure mode: an agent step that orchestrates a sequence of tool calls fails on the third tool call. The orchestrator retries the entire activity. The agent, now re-invoked, re-executes the first two tool calls before reaching the third. Those first two tool calls may write data, charge an API quota, send a notification, or trigger an external process. They fire again. The system now has duplicate side effects, and the agent is none the wiser.

A subtler failure mode: the retry succeeds, but the agent makes a different decision the second time due to non-determinism. The orchestrator records success and moves on. Downstream steps now operate on an output that is semantically inconsistent with whatever partial state was written during the first (failed) attempt. There is no exception. There is no alert. There is just a quietly wrong workflow outcome.

The Deeper Problem: Framework Abstractions Are Hiding This

Part of what makes this so insidious in H2 2026 is that the tooling ecosystem is actively obscuring the problem. Agentic orchestration frameworks have rushed to offer "reliability" features, automatic retries, checkpointing, and activity-level fault tolerance, all of which are genuinely valuable. But they were designed with deterministic, idempotent workloads in mind. Applying them naively to agentic steps creates a false sense of correctness.

Temporal, for example, is an exceptional orchestration engine. Its workflow replay model is elegant and powerful. But Temporal's replay guarantees apply to the workflow code itself, not to the activities the workflow calls. If your activity is a non-deterministic, non-idempotent agent invocation, Temporal will faithfully retry it without any awareness that doing so is semantically unsafe. The framework is not broken. The assumption is broken.

LangGraph and similar agentic graph frameworks offer node-level checkpointing. Again, genuinely useful. But checkpointing captures the state of the graph, not the state of the external world the agent has already acted upon. A checkpoint that resumes an agent after a tool-call failure will re-execute the tool call, because the checkpoint does not know the tool call already partially succeeded.

The frameworks are doing exactly what they advertise. The problem is that engineers are reading "retry-safe" and hearing "agent-safe." Those are not the same thing.

A Correct Mental Model for Agent Retry Safety

So what does a correct approach look like? It requires decomposing the agent invocation into its constituent properties and handling each appropriately. Here is the framework I recommend:

1. Separate the Decision from the Action

The LLM inference step (the decision) and the tool execution step (the action) should be treated as distinct activities with different retry semantics. The decision step can be retried with the understanding that results may differ, and your system should be designed to tolerate that variance. The action step must be wrapped in explicit idempotency controls at the tool layer, not assumed to be safe to repeat.

2. Make Tool Calls Explicitly Idempotent

Every tool your agent can invoke should be designed with idempotency as a first-class requirement. This means stable, agent-generated or orchestrator-injected idempotency keys that travel with the tool call and survive retries. This is not optional. This is the foundational contract. If your tool layer cannot honor idempotency keys, your agent cannot be safely retried.

3. Checkpoint at Action Boundaries, Not Agent Boundaries

Your durable state should be recorded after each tool execution completes successfully, not after the entire agent turn completes. This means the retry unit should be the next tool call, not the entire agent invocation. Yes, this requires more granular orchestration. Yes, it is more complex to implement. It is also the only approach that is actually correct.

4. Treat Non-Determinism as a Feature, Not a Bug to Suppress

Stop trying to make your agents deterministic through prompt engineering, temperature clamping, and seed pinning. These are leaky abstractions. Instead, design your workflows to be robust to output variance. If two different but valid agent decisions lead to divergent system states, your system has a correctness problem that retry logic cannot solve. Fix the system design.

5. Implement Semantic Idempotency at the Workflow Level

For agent outputs that drive downstream decisions, implement a semantic deduplication layer. Before acting on an agent output, check whether a semantically equivalent action has already been taken in this workflow execution. This is harder than key-based idempotency, but it is the appropriate tool for non-deterministic producers.

This Is Not a Framework Problem. It Is a Mental Model Problem.

I want to be direct here, because I think some teams are waiting for their orchestration framework of choice to "fix" this. That is not going to happen, at least not in the way people expect.

Frameworks can add guardrails. They can expose hooks for idempotency key injection. They can provide better documentation about the limits of their retry guarantees in agentic contexts. Several are already moving in this direction. But no framework can make a non-idempotent tool call idempotent on your behalf. No framework can make a non-deterministic agent deterministic. These are properties that must be engineered into the system, not configuration flags to be toggled.

The mental model shift required is significant. For years, backend engineers have been trained to treat retry logic as a reliability primitive, something you add to make systems more robust. In agentic systems, blind retry logic is not a reliability primitive. It is a correctness hazard. Reliability in agentic workflows requires you to earn the right to retry, by proving that each retried operation is actually safe to repeat.

The Organizational Dimension: Why This Keeps Happening

There is also a team dynamics explanation for why this conflation is so persistent. The engineers building the orchestration layer often come from distributed systems backgrounds. They know retry logic. They trust it. They apply it confidently.

The engineers building the agent logic often come from ML or product backgrounds. They understand LLMs. They know about non-determinism in principle, but they think of it as a model-level concern, not a systems-level concern.

Neither group is wrong within their own domain. But the interface between those domains, the point where an LLM inference result becomes a distributed system action, is where the conflation lives. And because it lives at the seam between two teams, it often belongs to no one. No one owns the question: "Is this agent step actually safe to retry?"

In H2 2026, as agentic workflows move from pilot to production across financial services, healthcare operations, legal tech, and enterprise SaaS, that question is no longer academic. It is a production correctness question with real consequences.

Conclusion: Name the Distinction, Then Engineer It

The fix starts with language. In your architecture reviews, your RFC processes, your incident post-mortems: stop using "idempotent" and "deterministic" as synonyms. Define them precisely. Ask explicitly, for every agent step in your workflow: Is this step deterministic? Is this step idempotent? What are the side effects? What happens if we run this step twice?

If your team cannot answer those questions cleanly, you do not yet have a retry policy. You have a retry wish.

The distributed systems community spent decades learning hard lessons about the dangers of assuming away failure modes. The agentic systems community is about to learn a new set of hard lessons about the dangers of assuming away non-determinism and non-idempotency. The engineers who understand both bodies of knowledge, and who can build the bridge between them, are going to be extraordinarily valuable in the years ahead.

The rest will be debugging production workflows that fail in ways that make no sense, because the retry logic was written for a world where agents behave like functions. They do not. Build accordingly.