The second cost vector of AI agents: durable execution with Temporal

· By José Manuel Román
Contents

Notation: amounts in euros (N €), decimals with a point. The dollar symbol is not used (on this site it is a formula delimiter).

The thesis: agents have two cost vectors, not one

Almost all the cost discourse around agentic AI focuses on a single vector: the cost of inference (the tokens). And that one is solvable: FastMCP + vLLM on local hardware eliminates the cost of inference — you serve the models on your own iron, the cost per token drops to your amortised cost, with no API bill. But there is a second vector that almost nobody measures and that kills more projects: the cost of faulty execution. An agent is not a call, it is a sequence of steps (reason, call a tool, reason, call another one, and so on), and when that sequence fails halfway through with no persisted state, the cost is not linear: it doubles, it spirals with retries and it inflates on every recovery. This article is about that second vector, how durable execution attacks it (with Temporal as the reference), what the state of the art and the papers say, and why it is the real cause of so many cancellations.

The framing that backs the urgency comes from Gartner: it predicts that more than 40 % of agentic AI projects will be cancelled before the end of 2027, because of rising costs, unclear business value or inadequate risk controls (Gartner). “Rising costs” is exactly this second vector: not the cost of serving the model, but the cost of running it badly.

The three expenses of a faulty agent

The cost of faulty execution breaks down into three concrete sources, and it is worth seeing them with numbers:

1 · Failures charged twicefailure at step 15 of 20→ the previous 14 are thrown away+ full retry2 · Uncontrolled retriesretries with no policy→ API/compute billwith no ceiling3 · Inflated contextrecovery rebuildsthe full context→ extra tokens per stepNone is the "base" inference cost: they are the costs of running the sequence BADLY. The vector local vLLM does not touch.Durable execution attacks all three at once with per-step persisted state.

Vector 1: failures charged twice

With no persisted state, a failure at step 15 of 20 costs you the tokens of the 14 previous steps (which you already paid for) plus the full retry from scratch. The wasted cost of an agent with no checkpoint, per failure, is:

$$\text{wasted cost} = \text{tokens of the completed steps} \times \text{cost per token} \times (\text{retries})$$

The longer the sequence and the later it fails, the more expensive: a failure at 75 % of the work throws away 75 % of the cost. Durable execution checkpoints every step: if the worker dies halfway through, another one picks the workflow up exactly where it stopped, not from scratch (Temporal). Temporal uses its Event History as the record of past decisions; on a failure it replays the progress to date and resumes at the exact point (CallSphere). Step 15 fails → step 15 is retried, not the previous 14. The wasted cost goes from “the 14 steps” to “the step that failed”.

Vector 2: uncontrolled retries

Agents that retry with no policy blow up the API or compute bill: a retry loop against a rate limit can generate dozens of billable calls for a single task. Durable execution defines retry policies per activity: hard limits on retries, exponential backoff and timeouts that bound the maximum possible cost of each call. Instead of “retry until it works” (unbounded cost), you define “at most N retries with backoff and a timeout of T” (bounded cost).

The state of the art backs this with data: the ReliabilityBench benchmark evaluates agent reliability under controlled failures and finds that rate limiting is the most damaging failure of all (arXiv 2601.06112). That is, exactly the failure that an unpolicied retry turns into a cost spiral. Bounding retries is not hygiene, it is the defence against the failure mode the literature singles out as the worst.

Vector 3: inflated context on recovery

With no persisted state, the agent rebuilds the full context on every recovery: it reloads the whole conversation/task history to “remember” where it was, paying the tokens of that enormous context on every recovery. With durable execution the state lives in the workflow history, so the next step receives only the context needed for that step, not the whole conversation. The saving is direct: fewer input tokens per step, and no penalty for rebuilding the context after each failure.

A worked example: the cost of one failure

Let us put numbers on the three vectors. A 20-step agent consuming ~3,000 tokens per step (reasoning plus tool call), on in-house inference at ~1.09 €/1M tokens, with a failure at step 15 and a failure rate of 10 % of executions:

ScenarioTokens recomputed per failureCost per failureOver 10,000 executions (10 % fail)
No state (full retry)14 steps × 3,000 = 42,000~0.046 €~46 € wasted
With checkpoint (resumes at 15)0 (only step 15 is retried)~0.003 €~3 € wasted

The absolute figure looks small because inference is in-house and cheap; but the ratio is ~15× less waste per failure, and it scales with the length of the sequence and with the cost per token (were it an external API at 10-30 €/1M, the waste without a checkpoint would be multiplied by 10-30). And this is only vector 1: add uncontrolled retries (vector 2), which can turn one failure into dozens of calls, and the context rebuilt on every recovery (vector 3). The cost of faulty execution is not the inference cost × something small; it is a multiplier that grows with the agent’s complexity, and it is exactly what does not show up in the demo and does show up in the production bill.

How Temporal works: workflows, activities and determinism

To understand why Temporal achieves this, its two primitives:

  • Workflow: the agent’s orchestration logic (the sequence of steps, the decisions). It has to be deterministic: given the same Event History, it produces the same decisions. That is what allows the execution to be replayed after a failure and to arrive at the same point.
  • Activity: every step with side effects (a call to the LLM, to a tool, to an API). Activities are not deterministic (the LLM is probabilistic), so their result is persisted in the Event History: on replay they are not executed again, they are read from the history.

The trick is that separation: the deterministic logic (workflow) is replayed cheaply from the history; the expensive, non-deterministic steps (activities, the model calls) are persisted and not repeated. That is why “completed work is never repeated”: completed activities are read, not recomputed. And retry policies are defined per activity, so each call to the model or to a tool has its own retry limit, backoff and timeout, which is the vector 2 cost control at the right level.

Durable execution: what it is and the landscape

Durable execution is a programming model that guarantees the code completes despite failures: every step is checkpointed automatically, and if a worker dies, another one picks the workflow up where it stopped (Temporal). It crossed into the “early majority” in 2025, with offerings from AWS, Cloudflare and Vercel, driven precisely by AI agent infrastructure (Inngest).

step 1 ✓step 2 ✓… checkpoint per step (Event History) …step 15 ✗failure hereresumes at step 15, not at step 114 ✓Completed work is never repeated: the cost of a failure is that of the step that failed, not of the whole sequence.

The tooling landscape (the 2026 reference architecture for durable LLM agents): Temporal, AWS Step Functions Express, Restate, DBOS and Inngest as durable execution primitives; and the checkpointer model of LangGraph (PostgresSaver, RedisSaver, DynamoDBSaver) on the agent framework side (render.com). Temporal stands out for its workflow-as-code model and its Event History as the source of truth.

OptionModelSelf-hostedNote for sovereignty
Temporalworkflow-as-code, Event Historyyes (open source)the most mature; full on-prem
Restatedurable RPC/handlersyeslightweight, modern
DBOSdurable on top of Postgresyesstate in your own Postgres
Inngestevent-driven, stepsSaaS / self-hostfocus on DX
AWS Step Functions Expressstate machineno (AWS SaaS)tied to AWS, not sovereign
LangGraph checkpointerframework checkpointeryes (your backend)at framework level, not platform level

For a sovereign platform, the ones that are self-hosted (Temporal, Restate, DBOS) keep state in your cluster; the SaaS ones (Step Functions) move execution into someone else’s jurisdiction. Temporal is the choice for maturity and for its complete on-prem deployment.

The cost of durable execution itself

Honesty about the data: durable execution is not free. It has two costs to watch:

  1. Execution overhead: persisting state and replaying history adds latency and compute. The reference architectures put it at around 5-20 % overhead on a load of 100,000 executions/day (render.com).
  2. Price per step: in the offerings with per-step pricing, orchestrating many model calls, rate limit retries and complex interactions generates a large number of billable steps, and the cost can run away (render.com).

The conclusion for a sovereign platform: self-host the durable execution engine (Temporal is open source and deploys on-prem) and the SaaS per-step price disappears, leaving only the execution overhead, a 5-20 % that is paid back many times over by what it saves on the three vectors above. The arithmetic: if a failure halfway through a sequence used to cost you 50-75 % of the recomputed cost, a 5-20 % overhead is a bargain.

Observability: from manual post-mortem to real-time traceability

The fourth element of the equation: the observability layer. The integration with Braintrust and the complete history of every execution (Temporal’s Event History) turn cost analysis from a manual post-mortem, reconstructing by hand what happened and what it cost after a failure, into real-time traceability: every execution is recorded step by step, with its tokens, its retries and its cost, ready to analyse. This closes the FinOps loop of the cost-per-token article, but at the agent level: not just how much a token costs, but how much an execution costs, and where it is wasted.

The complete equation

Putting the pieces together, the cost architecture of a sovereign agent:

FastMCP + local vLLMeliminates the costof INFERENCE (tokens)+Temporal (durable exec.)eliminates the cost ofFAULTY EXECUTION=Sustainable agentbounded cost,sovereign, traceableThey are COMPLEMENTARY: they attack different vectors. The 40% of projects Gartner writes off dies by ignoring the second.+ observability (Braintrust / Event History) = real-time cost traceability.

FastMCP + vLLM running locally eliminates the cost of inference. Temporal eliminates the cost of faulty execution. They are complementary, attacking different vectors of the same problem. Whoever solves only the first (serves cheaply but executes badly) keeps bleeding through the second; whoever solves only the second (executes robustly but pays for the API) keeps bleeding through the first. A sustainable platform needs both.

State of the art and papers

The reader asked for papers, and there are some: agent reliability and cost is an active research area in 2026.

PaperContribution
ReliabilityBench (2601.06112)an agent reliability benchmark: consistency, robustness and tolerance to tool/API failures; rate limiting is the most damaging failure; ReAct more robust than Reflexion under stress
Cost-Efficient LLM Agents (arXiv)the reliability-cost trade-off: routing every decision through the LLM improves accuracy but blows up cost; pre-coded graphs are cheaper but brittle against compound failures
Multi-agent Byzantine Fault Tolerance (2511.10400)BFT consensus (CP-WBFT) to stabilise multi-agent systems under extreme failure rates (85.7 %)
ACRFence (2603.20625)security of agent checkpoint-restore: preventing semantic rollback attacks
The Six Sigma Agent (2601.22290)enterprise reliability via consensus-driven decomposed execution

Two readings for the proposal. First, the reliability-cost trade-off is real and is characterised: you cannot maximise accuracy ignoring cost, or the other way round; durable execution shifts the frontier by making reliability cheaper (checkpoint plus bounded retry instead of recomputing). Second, agent checkpoint-restore has its own security surface (ACRFence): persisting state introduces rollback risks that have to be controlled — durability is not only a cost decision, it is a security one too.

The sovereign angle

For a European sovereign platform the combination fits perfectly and is entirely on-prem: vLLM serves the models (no API cost and no sovereignty given away), FastMCP exposes the tools, and self-hosted Temporal orchestrates with durability (no SaaS per-step price and no execution data leaving for a foreign jurisdiction). The execution history, which describes in detail what your agent does and with which data, stays in your cluster, just like the inferences. Against a stack of agents on external APIs plus a US SaaS orchestrator, the sovereign stack removes both cost vectors and keeps the data under EU jurisdiction. It is the same argument as the rest of the series, applied to the agent layer: bounded, measured and sovereign cost.

FastMCP and the tool layer

The other component of the equation is FastMCP, the layer that exposes tools to the agent via MCP (Model Context Protocol). The agent does not call the APIs directly: it calls MCP servers that encapsulate each tool (a database, a search engine, an internal system) with a uniform contract. The connection with cost and durability: every call to an MCP tool is a Temporal activity, so it inherits its retry policy, its timeout and its persistence. That is, FastMCP defines which tools exist and Temporal governs how they are executed with durability and bounded cost. And because the MCP servers are your own, the tools and the data they touch stay on-prem, never exposed to an external provider. The triad vLLM (model) + FastMCP (tools) + Temporal (orchestration) is a fully sovereign agent stack: the model, the tools and the execution, all under your control and with the cost of both vectors bounded.

Patterns: human-in-the-loop and long sequences

Where durable execution shines most, by agent pattern:

  • Human-in-the-loop (HITL): human approval patterns, essential for an agent’s safety, map directly to the suspend/resume primitives of durable execution, allowing workflows that pause for hours or days waiting for an approval without losing state (Inngest). Without durability, keeping an agent “waiting” consumes resources or loses the context; with it, the workflow sleeps at no cost and wakes up with all its state.
  • Long sequences: the longer the chain of steps, the greater the checkpoint’s saving (a late failure throws away more work). Multi-step agents with tools are the canonical use case.
  • Multi-agent: when several agents collaborate, reliability gets complicated (the BFT papers study this); durable execution provides the consistent state substrate on which to build that coordination.

The pattern that does not need it: a single-step, stateless call (a classifier, a simple extraction). There the durability overhead buys nothing. The rule: durable execution for stateful sequences that fail halfway through; for everything else, a direct call.

How to measure the second vector

To manage the cost of faulty execution you have to measure it, and with the Event History plus observability (Braintrust) the KPIs are within reach:

KPIWhat it indicatesWhere it comes from
Tokens wasted per failurethe cost of vector 1tokens recomputed after a failure
Retries per activitythe cost of vector 2retry count from the Event History
Context size on recoverythe cost of vector 3input tokens after a recovery
Failure rate per workflowhow often the problem occursfailed executions / total
Cost per complete executionthe business figuresum of tokens × cost/token per run

The last one closes the circle with cost per token: not just “how much a token costs”, but “how much an agent execution costs, and how much of that was useful work versus waste”. Without these KPIs the second vector is invisible, and what is invisible does not get optimised, it gets cancelled when the bill surprises you. With them, the cost of every execution is a real-time figure, and the decision to make each workflow durable (or not) is taken with numbers.

Limits and traps (data-driven)

  1. Durability is not free. 5-20 % overhead; on SaaS with per-step pricing it can run away. Self-host Temporal to remove that price.
  2. Not every agent needs it. A single-step, stateless task gains nothing from durable execution; the value is in the long sequences that fail halfway through.
  3. Bounded retries can hide real failures. A retry limit avoids the cost spiral, but you have to alert when it is exhausted, not swallow the failure in silence.
  4. Checkpoint = security surface. Persisting state opens the door to rollback attacks (ACRFence); protect it.
  5. The first vector still exists. Temporal does not lower the inference cost; you need local vLLM for that. They are complementary, not substitutes.

The conclusion the data and the papers support: the cost of agentic AI has two vectors, and the industry has obsessed over the first (inference) while ignoring the second (faulty execution), which is precisely where Gartner locates the cause of 40 % of cancellations. FastMCP + vLLM solves the first; Temporal the second. A sovereign and sustainable agent platform needs to attack both, with observability that turns the cost of each execution into a real-time figure rather than an autopsy.

Closing

The dominant narrative about agent cost is half-lame: it treats the inference cost as if it were the only one, when it is only the one you see in the demo. The one that kills projects, the one Gartner files under “rising costs” inside its 40 % of cancellations, is the cost of faulty execution: failures charged twice, retries with no ceiling and the context rebuilt on every recovery. The engineering that attacks it is not a prompt trick or a better model, it is an execution model: durable execution, which checkpoints every step, bounds every retry and passes only the necessary context. Temporal, open source and self-hosted, does this while keeping state in your cluster, and combined with vLLM (model) and FastMCP (tools) it composes a fully sovereign agent stack where both cost vectors are bounded and the data never leaves the EU. The state of the art and the papers confirm the problem is real and characterisable: the reliability-cost trade-off, rate limiting as the worst failure, checkpoint-restore as a new security surface. For anyone who wants to be in the 60 % of agentic projects that are not cancelled, the recipe is the one in this article: solve both vectors, not one, and measure the second as seriously as the first.

Sources