The ROI of an AI agent: measuring it honestly

Most AI agent ROI calculations are run backwards. You build the agent, you watch it work, it feels faster, and you declare victory. The trouble is that "it feels faster" is not a measurement. It's a memory, and a memory has never paid anyone back.

This article proposes the reverse method: measure before you build, separate the real benefits from the imaginary ones, count the full cost, and decide up front the conditions under which you pull the plug. If you're weighing a custom AI agent for your business, this is the frame that tells you whether yours earns its price, instead of leaving you to guess.

It isn't the guide that sells you a dream. It's the one that hands you the numbers to say no when you should, and yes when it holds up.

The short answer, for the busy

Measuring an AI agent's ROI honestly takes three steps most companies skip. First, set a baseline before you build: time the target task for two weeks, or you'll be comparing the agent to a hazy memory. Second, separate three benefit classes and their traps: time freed (which only counts if it's actually redeployed to higher-value work), error reduction, and speed-to-revenue effects. Third, count the full cost: the subscription, human review time, and change management. For a custom agent starting at $947 a month, at a fully loaded labour cost of $35 an hour, the breakeven sits around 27 hours of work freed per month, roughly an hour and a quarter per business day. Below that, wait. And set your kill criteria before you start, the conditions under which you shut the agent down, so the decision isn't made in the heat of a sunk investment.

The baseline, the step everyone skips

Here is the mistake that warps almost every AI agent ROI number: nobody measures the "before". You vaguely know that "email triage eats time", but no one has timed it. So when the agent shows up, you have nothing to compare it against, and you fall back on the impression of a gain, which is always flattering.

The fix is not sophisticated. Before you build anything, take two weeks and measure the target task for real. How many times a day it comes up, how long each occurrence takes, who does it, at what error rate. A spreadsheet and a little discipline are enough. Those two weeks of measurement are the most profitable data in the whole project: without them, you'll never be able to prove the gain, not to your banker, and not to yourself.

The baseline has a second, less obvious effect. It often reveals that the task costs less than you thought, or that it's too irregular to automate cleanly. Better to learn that before you spend than six months later. An honest baseline has already killed good-looking projects on paper, which is exactly its job.

The three benefit classes, and each one's trap

An AI agent creates value in three ways, and each hides a different trap.

Time freed. The benefit everyone cites first, and the most poorly counted. An agent that takes back 30 hours of triage a month only saves you 30 hours if those hours are genuinely redeployed to higher-value work. If the person whose load the agent lightened still costs the same and fills the freed time with something secondary, the gain exists on the spreadsheet but not in the books. Time freed becomes money only when it's redeployed, or when it avoids a hire you would otherwise have made. Name in advance what the person will do with those hours. If you can't answer, the benefit is theoretical.

Error reduction. Quieter, often more profitable. A mis-keyed invoice, an urgent claim buried under three emails, a quote priced wrong: these cost a lot to catch and fix, and sometimes a client. The trap here is the opposite of the last one: you underestimate this benefit because it's invisible when it works. To measure it, you need the "before" error rate on record (the baseline again), or you'll never see the improvement.

Speed-to-revenue effects. The most seductive, and the most dangerous to quantify. Replying to a prospect in two minutes instead of 48 hours moves conversion rates. Sending a quote same-day wins more of them. But nobody serious will promise you a percentage without your data. Treat this class as a plausible bonus, not the line that justifies the project. If your case only holds thanks to an assumed revenue gain, it doesn't hold.

The full cost, beyond the subscription

The ROI denominator gets fudged as much as the numerator. The agent's monthly fee is visible, so that's the one people write down. Two real costs almost always go missing.

The first is human review time. A well-designed agent escalates ambiguous cases to a person: that's a good thing, but it isn't free. Someone handles those escalations, validates edge cases, corrects the agent when it's wrong. In the early months especially, budget a few hours a week. That load shrinks with tuning, but never hits zero.

The second is change management. Your teams have to shift habits, trust the agent, learn when to overrule it. A technically perfect agent nobody uses has a negative ROI. That cost shows up on no invoice, but it's real, and it's often what separates the projects that take off from the ones that stall.

The breakeven, run honestly

Once you have the real benefits and the real costs in hand, the math is simple. We laid it out in what an AI agent costs for a Quebec SMB; here's the tight version.

A custom subscription agent starts with us at $947 a month. At a fully loaded labour cost of $35 an hour (salary plus overhead, for a typical administrative role), the breakeven lands around 27 hours of work freed per month, roughly an hour and a quarter per business day. Below that volume, the math doesn't work, and an honest provider will tell you to wait. Above it, it improves fast, because the agent's cost is fixed while the freed hours pile up.

One figure to keep you from getting sold anything: model inference fees typically run 3 to 7% of an agent's total cost in 2026, not half. If a provider justifies the price with "AI costs", that's a sign they don't understand their own market. The cost is in the human work around the model, never in the model itself. The concrete cases and their arithmetic are in our piece on AI agent use cases for SMBs.

"95% of AI projects fail": what the number actually says

You've probably run into the stat. In August 2025, MIT's NANDA initiative published a report, "The GenAI Divide: State of AI in Business 2025", concluding that 95% of enterprise generative AI pilots produce no measurable impact on financial results. The figure went around the world, often waved as proof that enterprise AI is a bubble.

Read it closely before using it either way. "Fail" here means precisely: no measurable return on the P&L within the studied window. It doesn't capture diffuse efficiency gains or longer-term benefits. The method rests on interviews (around 150 leaders, a survey of 350 employees, an analysis of 300 public deployments), which the report itself calls directional rather than audited accounting, especially since companies are reluctant to report their failures. A real signal, then, not a law of nature.

The useful detail for an SMB sits elsewhere in the same report: projects handed to a specialized vendor succeed about twice as often as internal builds (roughly 67% versus 33%). The lesson isn't "AI doesn't work". It's that how you go about it decides the outcome: a scoped, measured project handed to someone who does this for a living lands on the right side of the stat. A pilot launched with no baseline and no success criteria lands on the wrong side.

Kill criteria, set before you start

Here's the part no provider volunteers, and the one that best protects your money: decide in advance the conditions under which you shut the agent down.

A kill criterion is a line written at the outset. For example: if after three months human review time exceeds the time the agent is supposed to free, we stop. If the error rate on cases handled unattended doesn't drop below a set threshold, we stop. If the team routes around the agent instead of using it, we find out why, and if that isn't fixable, we stop. These conditions set in the cold, before any attachment to the project, are worth ten times a decision made in the emotion of an investment already spent.

The point of a kill criterion isn't to forecast failure. It's to turn the agent into a known-risk bet rather than a vague commitment you no longer dare question. A project you know you can stop cleanly is a project you can start with a clear head. Governing an agent in production, including when it gets things wrong, deserves the same care: we cover it in governing an AI agent in production.

Where to start

The honest sequence has three beats. Measure the target task for two weeks first. Then run the breakeven with your real numbers, full human cost against 12 times the agent's monthly fee. Finally, set your kill criteria before you sign anything.

That's exactly the scoping we do in a first conversation: identify the task with the best cost-benefit ratio, estimate the real gain, and structure the project so it's measurable from day one. 30 minutes, no commitment. And if the conclusion is that an agent isn't justified for you yet, we'll say so plainly.

→ First conversation, no commitment