Most tools now have something called simulation. The question is no longer whether to rehearse, but what the rehearsal has to show you before it does anything real.
Last week, I watched “In The Grey”, an “action-comedy” with Eiza González, Henry Cavill and Jake Gyllenhaal. What struck me was the amount of time spent on dry-runs of their plans — who drives what, where the mines are, where the ditch is and so on. Every single option is planned, simulated, timed and improved on. Most times, we see our hero casually jumping off a boat and diving into a shipwreck that’s 200 meters deep. Heck, my scuba instructor won’t let me get past 18 meters after 2 years of diving. So every time I build a new agent, I want the Sid-and-Bronco version, not the James Bond version: simulated in real-data conditions, telling me what it will do and what it won’t.
A typical Agent is a sequence of actions against your live data, chosen at runtime. So the important part is not whether the code executes but which actions it picks and what it puts in them. And pretty much every Agent designer has arrived at some variation of the same answer: rehearse the thing before letting it loose. We made that case a while back and it is not a controversial position any more; most tools now have something called simulation, preview, dry run or sandbox mode. So the issue is no longer whether you should rehearse but what the rehearsal has to show you before it does anything real. And the ability to do that multiple times, maybe with multiple sets or types of data.
In today’s agentic environments, a message that says “Simulation completed successfully” is of no consequence. Given that we no longer write in Fortran or COBOL or JS, we’re not worried that the code will fail to execute; we’re worried about what it will do while executing perfectly. A run that finishes cleanly while doing entirely the wrong thing reports exactly the same green tick as one that does the right thing. So the useful property of a simulation is not its verdict. It is whether the output is specific enough to be wrong. If you read it and cannot find anything to disagree with, that is not necessarily reassurance — it could be an absence of information. So the standard that a simulation has to meet is this: hand you a list of intended actions detailed enough that a person who knows the business can look at it and say No, not that customer or No, not that amount.
For that to happen, there are at least five things the agentic engine needs to demonstrate.
“Would update 4 records” is a summary of a thing you cannot check. It tells you the scale of the action, but nothing about its correctness. What we need is each intended write with the values it would carry. Which record, which fields, which new values? That is the level at which a human can recognise a mistake — say, that this invoice is going to the parent company rather than the subsidiary, or that this status change closes a ticket that the customer is still waiting on. You can’t see either of those in a count. And such a detailed log has another big benefit: arguments are comparable. Run the simulation twice, maybe a day or a week apart, and a list of specific intended actions tells you what changed in the agent’s behaviour.
This checkbox separates a real simulation from a demo: data changes and outbound messages are not the same category of risk and must be listed separately. A wrong field can be corrected, usually before anyone notices. A wrong email, ticket reply, Slack message or invoice sent to a customer, on the other hand, can’t be recalled. That difference needs to be highlighted, not hidden. So keep them separate and read the sendable list first. It should be short. If it is long, that is itself a finding — you have built something that talks to people more than it probably needs to.
The same distinction shows up everywhere once you look for it: preparing a reply is not sending it, preparing an invoice is not delivering it, a shortlist is not an assignment. Simulation is where you get to see which side of that line each action falls on, before any of them cross it.
This may be one of those “never-thought-to-ask” kind of things: what exactly did the agent read? An agent that reads the wrong slice of data will still produce a perfectly plausible set of intended actions. Every write will look reasonable. Every message will read well. And all of it will be built on the 42 records it happened to look at rather than the 4200 it should have. And you can’t detect that difference by inspecting the output, because the output is internally consistent. So the read scope is a genuine control, not a debug detail: which collections, how many records, filtered how. It is the cheapest way to catch an agent that is confidently working from the wrong week, the wrong project, or the wrong subset of a queue.
If you’ve asked for approval-gates in your Agent, the simulation should show you the run arriving at one and stopping, with details of what would be waiting there and who would be asked. Until a simulated run actually reaches — and stops at — a gate, the gate is only a claim in your Agent intent. A gate placed after the thing it was meant to protect will look good in a settings screen, but a rehearsal is the only place you figure out where it stops.
Watch for the inverse too: if the simulation passes through a stage you expected to be gated and does not stop, you have learned something urgent and fixable.
This last one is the hardest to provide and may be the most valuable to have: the road not taken. A simulation exercises one path — the one the current data produced. The exception path, the empty-result path, the malformed-input path, the one where the customer’s record is missing the field everyone assumed was mandatory: none of those ran. A rehearsal that shows only the happy path, without saying it was only the happy path, encourages you to believe you have seen the agent behave. When you have seen it behave once. IMHO, a simulation that presents one clean pass as complete evidence is seriously suspect.
Let’s check this out in the real world. Open the most recent simulation output your tooling produced and walk through these items:
Yes, “In The Grey” spends a lot of airtime on simulations and tests, unlike most action movies. But as an agent-designer, that actually worked for me 😊.
Bring the most recent simulation output your tooling produced, and the five questions above.