The most expensive agent failure we've seen isn't an error. It's silence: an agent runs 40 tool calls, each one fast, each one successful — and the platform's hard turn limit fires at call #41. No exception reaches your code. The turn, and every result it produced, is simply gone.
Per-call timeouts don't help, because the problem was never one call. Thirty calls at 8 seconds each blow a 300-second turn, and every one of those calls was individually "fine." The unit that needs a timeout is the turn, and the unit that needs to respect it is every step.
The pattern: one budget, consulted at every step
from turn_budget import TurnBudget
b = TurnBudget(total_s=280, reserve_s=20) # 280s turn, keep 20s to write results
for step in plan:
if not b.can_afford(step.estimated_s):
break # ship what's done, say what isn't
do(step)
b.steps_run += 1
Three details do all the work:
- Reserve time at the end.
reserve_s=20means the budget stops handing out time while you can still write results, checkpoint state, and compose the summary. Agents die mid-sentence when the last 10% of the clock is already spoken for. - Retries consult the same budget. A retry loop with its own timeouts is a budget leak — the fix from the retry post passes
budget_s=b.usableso no call can spend time the turn doesn't have. - Steps declare their cost.
can_afford(estimated_s)before the step, not after the timeout. An optional enrichment step that needs 30 seconds with 25 left doesn't run — and that's a decision, not a death.
The part everyone forgets: ship partial with a status
When the budget runs out, the agent must not go quiet. The loop exits by producing output:
result = {
"status": "PARTIAL", # not "success", not "failed"
"done": b.steps_run,
"skipped": b.steps_skipped,
"summary": summary_of_completed_work,
}
This is the difference between a run that dies at step 41 and one that finishes at step 37 having done 37 steps of useful work and told you. The status field also feeds downstream logic: a partial result should re-enter the queue for the remaining steps, not be treated as done.
Estimating step cost without a crystal ball
You don't need accurate estimates — you need the budget question asked at all. Use a rolling observed average per step type (last N runs), fall back to a conservative default for new step types, and treat can_afford as "fits with the observed p95," not the mean. The budget object's summary() — elapsed, remaining, steps run/skipped — is what you log, so "why was this partial?" stays answerable.
turn_budget module in AgentOps Autopilot (MIT, stdlib only, 24/24 tests + a chaos gauntlet that injects real turn-limit deaths). It pairs with resilient_call and failure_journal: budget exhaustion becomes a structured failure record instead of a vanished turn.Next in the series: the other timeout nobody sets — context window pressure.