An agent that works in a demo and an agent that runs unattended in production are two different systems. The demo has to succeed once, with a friendly input, while you watch. Production has to fail safely thousands of times, on inputs nobody predicted, when nobody is looking. Almost everything that breaks lives in that gap, and none of it shows up in the recorded walkthrough that got the project approved.
The demo lies by omission
When you show an agent booking a meeting or querying a database, you are showing the path where every tool call returned clean data and the model chose the obvious next step. That path is real. It is also rare. In production the same agent meets timeouts, empty results, malformed JSON, rate limits and half-written records. The interesting question is never whether the happy path works. It is what the agent does when a tool returns something it did not expect, because that is the state it will spend most of its life in.
My rule is simple. If you have not watched the agent fail, you have not seen the agent. Before anything touches real users, I want a run log full of broken tool responses and awkward inputs, and I want to see the recovery behaviour with my own eyes.
The failure modes that actually cost you
Across the agent systems I have built and reviewed, the same handful of failures come back again and again.
- Tool errors treated as content. A tool returns an error string, and the model reads it as data. It then confidently reports the error message back to the user as if it were the answer. The fix is structural: tool outputs need a clear success or failure shape, not free text the model has to interpret.
- Infinite and near-infinite loops. The agent calls a tool, does not like the result, calls it again with a tiny variation, and repeats. Without a hard step ceiling this quietly burns money and time until something else times out.
- Hallucinated actions. The model claims it sent the email, updated the record or refunded the customer when no tool was ever called. This is the dangerous one, because the transcript reads as success while nothing happened, or worse, the opposite happened.
- Cost blow-ups. One malformed document balloons the context, retries stack on retries, and a task that normally costs pennies costs pounds. Ten thousand of those a day is a real invoice.
Notice that none of these are model quality problems. A smarter model loops more efficiently and hallucinates more convincingly. These are systems problems, and they need systems answers.
Guardrails that earn their keep
Guardrails are boring and they are what separate a toy from a service. The ones I put in every serious agent:
- A hard step budget. Cap the number of tool calls per task. When the agent hits the ceiling, it stops and escalates rather than grinding on. A loop that cannot run forever cannot bankrupt you.
- Validated tool inputs and outputs. Every tool call is checked against a schema on the way in and the way out. A malformed argument is rejected before it reaches your systems, not after it has written a bad row.
- Idempotency on anything that writes. If the agent retries a payment or a create, the second call must be a no-op, not a duplicate. This single property prevents a large share of the scary incidents.
- Allow-lists, not open access. The agent gets the narrowest set of tools and scopes the task needs. An agent that cannot delete cannot be tricked into deleting.
The goal is not an agent that never makes mistakes. It is an agent whose worst possible mistake is something you can live with.
You cannot fix what you cannot see
Observability for agents is different from ordinary application monitoring. You are not just logging requests and latencies, you are logging decisions. For every run I want the full trace: the inputs, each model turn, every tool call with its arguments and response, the token counts and the final outcome. When an agent does something strange, that trace is the difference between a two-minute diagnosis and a two-day guessing game.
Beyond individual traces, watch aggregate signals that predict trouble: average steps per task creeping up, retry rates rising, cost per task drifting, and the share of runs that end in escalation. These move before users complain. Treat a rising step count the way you would treat rising memory usage, as a leak to investigate rather than a number to ignore.
Humans in the loop, on purpose
The most important design decision in an agent system is which actions require a human to approve them. Reading data, drafting a reply, summarising a document: let the agent run. Moving money, sending an external message, deleting records, changing permissions: put a person in the path. This is not a lack of ambition, it is basic engineering judgement. You gate the actions whose cost of being wrong is high and hard to reverse.
Done well, approval is not a bottleneck. The agent does the tedious work of gathering context and preparing the action, then presents a clear proposal a human confirms in seconds. You get most of the speed with almost none of the tail risk. As the system proves itself on a given action type, you can loosen the gate deliberately, with data, rather than hoping it was fine all along.
Where to start
If you are moving an agent from demo to production, resist the urge to add capabilities and spend that effort on failure handling instead. Give it a step budget, validate every tool boundary, make writes idempotent, log full traces, and gate the consequential actions behind a human. That unglamorous checklist is what turns an impressive demo into something you can leave running.
This is the work we focus on in our AI agent development engagements, and it is usually where the real value sits. If you are weighing up an agent project and want a candid view of what will break first, book a free consultation and we will talk through it.