When to stop an AI pilot: setting kill criteria.
Gartner found organisations with rules for stopping AI pilots were 2.3 times more likely to be top performers. How to set those rules, and how to apply them.
- Topic
- AI ROI
- Product
- Novie Control
- Format
- Guide
- Reading time
- 2 min
- Published
- 5 November 2026
Stop an AI pilot when it misses the result agreed at approval by the review date: cost per task above the limit, value below the baseline, adoption below the target, or a risk that cannot be controlled at a reasonable cost. Set those criteria before the pilot starts. Gartner found organisations with stopping rules were 2.3 times more likely to be among its top performers.
Few organisations have stopping rules.
In Gartner's survey published in October 2026, 31 percent of CIOs had stopping rules for AI pilots and 17 percent had exit criteria for AI in production. Organisations with those rules were 2.3 and 2.6 times more likely to be in its top-performing group. KPMG found that 49 percent of organisations had rephased AI agent deployments when costs outweighed value, which shows stopping is happening, often without a rule written in advance.
TakeawayMost organisations stop AI work only when the bill forces it, and the ones with rules written in advance perform better.
The criteria to set.
| Criterion | Stop if, at the review date |
|---|---|
| Cost per task. | It is above the limit in the business case, and no fix is in sight. |
| Value against baseline. | The measured gain is below the agreed minimum. |
| Adoption. | Fewer people or tasks use it than the case assumed. |
| Quality. | Error or rework rates are above what the work can bear. |
| Risk. | A data, security or compliance risk cannot be controlled at a reasonable cost. |
TakeawayFive criteria cover most decisions: cost per task, value, adoption, quality and risk.
How to apply them.
Write the criteria into the business case, with the review date. Measure monthly so the review has numbers. At the review, the owner proposes scale, fix or stop, and the sponsor decides. Record the decision and the reasons. If it stops, retire it properly: revoke access, set its budget to zero and move any work it was doing.
TakeawayCriteria written at approval, a monthly measure and a recorded decision make stopping routine.
How Novie Control does it
You can see what AI costs to build.Can you see what it costs to run?