I was listening to a webinar hosted by Finout at the end of September, with speakers from AWS and GEICO. The question on the table: how can you tell what value you’re getting out of AI? My answer, after listening: first work out whether you’re looking at commodity AI or advantage AI.
Priya Shastri, from GEICO’s FinOps team, gave the example. GEICO has rebuilt its claims process around AI. First, the customer takes a picture of the damaged vehicle and uploads it to an app. The app then reads the number plate and checks the vehicle is insured with GEICO. After that, a platform of AI agents running in parallel moves the claim along. All very cool stuff.
The result, in her words:
“Earlier the claim processing was taking more than four weeks just to do the loss calculation and just to do simple inspections. But now we reduced it to less than 48 hours.”
However, that figure covers simple claims. The bigger collisions still take about two weeks. Even so, that’s amazing for customers.
Great, until it’s the new normal
But what struck me whilst listening to the webinar was this: that’s great, until it’s the new normal. Before long, all of GEICO’s competitors do the same. As a result, the customer comes to expect 48 hours. Then they grumble that it took 52 hours rather than 48, even though it used to take four weeks.
Priya was candid that keeping up is the point. “We want to compete with all other insurance companies who are providing a response in a shorter time,” she said. For GEICO, “the main goal is price, service and retention”.
In other words, that is the language of keeping customers and keeping pace with rivals.
Table stakes: email, fax machines and commodity AI
On the same call Adam Richter, a solutions architect at AWS, reminded us that some AI will be like the introduction of email. Email arrived as a new way of working, and you can’t really quantify the value of that step change. Everyone does it.
Or take the fax machine. Once everyone sends their purchase orders by fax, you need to go and buy a fax machine. You don’t buy it for any particular strategic advantage. Instead, you buy it because that’s the way of working, and it makes business easier. It’s table stakes: the cost of doing business.
A lot of AI is heading the same way. As Priya put it, “almost all organisations somehow have AI tools at this point”.
I feel like an awful lot of the increased IT budget in the next couple of years will be chalked up to “table stakes”.
Commodity AI or advantage AI: which is which?
So the task is learning to tell which is which. Three questions help:
- Do your competitors already have it, or will they within a year?
- Do customers only notice it when it’s missing or slow?
- Can anyone buy it off the shelf?
Mostly yes, and you’re looking at commodity AI: the fax machine. Mostly no, and you may have a real advantage, for now.
For example, run GEICO’s claims process through those questions. Competitors are already responding faster, by Priya’s own account. Customers will notice hour 52. However smart the engineering, it is on its way to being a commodity.
Commodity AI doesn’t need an ROI case, because nobody writes one for email. Instead, it needs the lowest sensible unit cost and controlled risk.
Advantage AI deserves a value case with an expiry date on it, because competitors will catch up.
Cost per outcome: one measure for both
Adam offered a measure that serves both: cost per outcome. Take the total cost of the AI and divide it by the business result. His example: spend $5,000 on AI to resolve 10 bugs and you’re paying $500 per bug. Track that over time and you can judge whether it’s worth it.
He added two warnings. First, “we want to know the total cost of that AI, not just the tokens”. Second, “you have to pick a metric that’s not gameable”. Measure lines of code, for instance, and you’ll get more lines of code.
For commodity AI, cost per outcome is the number to drive down. For advantage AI, it’s the number to weigh against the value while the advantage lasts.
Sorting AI spend into two piles
The GEICO example is in-house development. But AI is coming at us from all directions: personal use of SaaS apps, professional use of SaaS apps, the data centre, the cloud, and AI embedded in technology we already own. Adam’s point was that AI is “growing so fast and touching so many things throughout these organisations” that it’s hard to allocate.
So my proposed first line of attack is to sort AI spend into two piles and treat each one differently.
Commodity AI is the cost of remaining competitive. Advantage AI, on the other hand, is an investment in becoming more competitive.
Commodity AI versus advantage AI at a glance
| Decision | Table-stakes / Commodity AI | Strategic / Advantage AI |
|---|---|---|
| Why buy it? | Because not having it creates friction or disadvantage | Because it might create differentiated performance |
| Business case | Usually none beyond sensible budgeting; treat it like email, cloud storage or productivity software | Explicit value hypothesis, owner and review date |
| How to buy | Standardise, benchmark aggressively, avoid unnecessary customisation, short commitments where possible | Preserve optionality: model choice, data portability, APIs and exit rights |
| What to measure | Adoption, utilisation and unit cost – with the expectation that unit cost falls | Cost per business outcome versus the economic value of that outcome |
| What happens to savings? | They rapidly become absorbed into the operating baseline | Capture them while the advantage exists |
| When to review | At renewal or when market pricing materially changes | When the hypothesis fails or when the advantage becomes commonplace |
| Who owns it? | IT / Procurement / ITAM | Business owner accountable for the outcome, supported by FinOps/ITAM |
| End state | Increasingly cheap, standardised infrastructure | Either sustained differentiation or eventual migration into the commodity column |
Whichever pile it lands in, Priya’s advice was to start measuring cost now. “Don’t wait till the end where you wait to see what is the business value.”
A question for readers: how are you sorting your AI spend?
