The Build-vs-Buy AI Test that Actually Predicts Adoption
Most AI vendor evaluations chase ROI projections. Peak International ran a single, real-world test instead — pitting an internal build against an external platform — and it settled the debate no slide deck could.
70% of enterprise AI initiatives fail. The statistic gets repeated so often that it’s easy to lose sight of what it actually means.
But sit with it for a second: seven times out of 10, an organisation decides AI matters enough to invest in — and it doesn’t work. 54% of C-suite executives say AI adoption is not only failing to deliver but is actively disrupting their organisations. That’s not a technology problem. It’s an organisational one.
I lead global service at Peak International Group, a manufacturer of gas generation systems for laboratories and other applications. Two years ago, we began building an AI initiative to support field service engineers, who spend their days diagnosing and repairing highly technical equipment, often in facilities where getting it wrong has real consequences.
Based on peer feedback, we made one crucial decision that put us on the right track: we classified this as a change project rather than an IT project.
That sounds like a semantic distinction. It isn’t.
Why the Label Matters Before the Budget Does
When AI rollouts are filed under IT, success is measured by whether the software is deployed and running: system uptime, integration, ticket closure — metrics the business already knows how to track. None of that tells you whether your service workforce actually changed how it works. And if it hasn’t, you’ve bought expensive shelfware.
Classifying our rollout as a change initiative meant IT still owned the infrastructure, but a different group owned the outcome. We started managing resistance before there was anything to resist.
We deliberately sought out the engineers most likely to be sceptical of an AI tool showing up in their workflow — the ones with the most tenure, and the most reason to distrust a “solution” imposed from above — and brought them into the process early as our user-acceptance testers. Not to sell them on it, but to ask what would actually make their jobs easier, and let their answers shape what we built.
That single move changed the emotional register of the rollout. Instead of a tool arriving and seeming to remove the need for people to do their jobs, it arrived because the people most likely to reject it had helped design it.
Resistance didn’t disappear, but it stopped being organised. It became individual — and individual resistance is something a good manager can work through in a coaching conversation. Organised resistance is what kills initiatives in committee.
The Test that Settled an Argument no Slide Deck Could
Every AI rollout eventually runs into a build-versus-buy debate, and ours was no exception. We tested an internal tool as an alternative to the external AI platform we were evaluating, keeping the comparison simple and grounded in a real-world scenario.
Like many service organisations, we operate in a regulated, safety-conscious environment. Our technical documentation changes constantly; a bulletin issued last month can override guidance in a manual that’s been correct for years.
We gave both tools the same test: reconcile a core technical manual with a newer bulletin that contradicted it, and return accurate part identification.
The internal build couldn’t do it — it returned an answer based on Product X when we’d asked about Product Y. The external platform distinguished between the two sources and surfaced the correct, current answer.
The One Question Most Organisations Skip
If I had to boil this down to a single test any leader can apply before submitting a business case, it’s this: can you state, in one clear sentence, what success looks like for the person who actually uses this tool every day?
Not what success looks like for the P&L. Not what it looks like in a board deck. What it looks like for the frontline engineer, the customer service representative, the marketer building campaigns — whoever will open this tool on a Tuesday afternoon and either trust it or quietly stop using it.
Most organisations skip this question because the focus is on ROI, when the priority should be something harder to quantify: the user experience of the frontline. You can build a case on numbers alone and hope to hit those metrics, or lead with employee experience and hope finance doesn’t ask hard questions later.
Either shortcut produces a pitch that sounds finished. Neither produces adoption. If you can’t answer that one question clearly, you’re not ready to evaluate vendors — you’re still doing the work that has to happen before that conversation starts.
AI initiatives don’t fail in the server room. They fail in the space between a rollout announcement and the moment someone decides whether the tool is worth trusting.
That space is entirely human, and it’s the one most technology conversations skip past to get to the exciting part. We didn’t skip it — and it’s the reason our initiative is still standing a year later, doing the job we built it to do.
ALSO READ: 6 Marketing Shifts Driving Middle East’s Next Phase
