Applied AI
How to measure the return on AI beyond hours saved
By Julián Medina · Director of SourcingUp and creator of CommerceUp
Measure preparation, execution, review and rework. The return comes from a better overall outcome, whether that means more capacity, higher quality or faster responses, after accounting for the cost of maintaining the solution.
Establish a meaningful baseline
Compare similar tasks before and after the change. Keep routine cases separate from exceptions so you can see where AI helps. Define the minimum quality an output must meet. Producing more drafts is not the same as resolving more inquiries if every draft needs substantial correction.
Include the less obvious costs
Include integration work, data preparation, training and supervision alongside software subscriptions and usage fees. An initial estimate can compare the value of the capacity freed up with these costs. That capacity is not automatically a cash saving: decide whether it lets the team handle more work, reduce delays or spend time on something else.
Decide what you will do with the result
Before the pilot starts, define what would justify continuing, changing or stopping it. You might aim for faster responses while maintaining the required quality and avoiding more complaints. Keep that definition consistent during the comparison. If the process or tools change, revisit the baseline: a good past result does not establish current performance.
Put it into practice
- Measure comparable cases.
- Count the corrections and interventions.
- Distinguish freed-up capacity from actual cost savings.
- Include maintenance in the evaluation.
The thinking behind this guide
A guide by Julián Medina. The scenarios are illustrative, not measured customer outcomes. My work at SourcingUp.