Spreadsheet · the scorecard
The working-paper version — fill it in while the meeting is happening.
A column for the answer, a column for the evidence, and a status against every question.
Download the scorecard →Free to use · on anyone · including on us
The questions to ask before you commit money to an AI system.
AI made it cheap to look right. It did nothing to make it cheaper to be right.
Anyone can now build something that looks finished in a weekend. A screen that demos well, a document that reads well, an answer that sounds right. What has not become cheaper is a system that is still correct in year three — and from the outside, you cannot tell the two apart.
We assess systems for a living. These are the questions we ask before anyone commits money — to buy a company, to sign a contract, to fund a build. We are publishing them so you can use them yourself: on any vendor, on any internal team, and on us.
You do not need to trust us to use them, and you do not need to be technical. You need to be allowed to ask them, and to insist that every answer comes with something you can inspect. A real system survives these questions. A facade does not — which is why nobody selling one will hand you this list.
- Free
- No email address required
- Nothing to sign up for
Question 1: Is it what they say it is?
The question · then what a real answer looks like
1.1
What actually runs the business — where does it run, and what does it cost each month?
A real answer is a short list: these systems, hosted here, costing this. Be wary of an answer that describes the future system instead of the one running today.
1.2
What is the system built of — as opposed to what the slides describe?
The slide says “AI-powered platform”. Ask what it is made of: which parts exist and run every day, which parts are a person doing the work by hand, and which parts are planned. The gap between described and built is where most of the risk lives.
1.3
What happens when it breaks — and how fast does anyone know?
Ask about the last outage: what failed, who noticed first, how long it took to fix. “It has never gone down” is a warning, not a comfort — it usually means nobody is measuring.
1.4
Can this team ship changes reliably — and is anyone steering?
Ask how a change travels from idea to production: who tests it, who approves it, how often releases happen, what happened the last time one went wrong. If every answer is the same person’s name, you have also just answered question 3.
1.5
Would you inherit an exposure — security, or privacy law?
Ask when the last independent security test was done and what it found. If the system holds personal information, ask how consent, access and deletion are handled — in South Africa that is POPIA, and the exposure transfers with the system, whether or not it was ever discussed in the deal.
1.6
Do they own what they are selling?
Ask what is licensed in from third parties — code, data, models — and on what terms. A system can work perfectly and still not be theirs to sell.
Question 2: Will it survive growth?
2.1
Is this a product — or an internal system with customers attached?
A product takes on the next hundred customers without heroics. An internal system with customers attached needs hand-work for every new one. Ask what it takes, step by step, to bring on a new customer, and how long the most recent one actually took.
2.2
Can the data model carry the product — and who understands it?
Everything the system will ever do rests on how its data is structured. Ask who designed the data model, whether they are still there, and who could change it safely today.
2.3
How much of what was sold is actually used?
Ask for usage numbers — per feature, per team or office — not adoption stories. The gap between what was bought and what is used tells you what the system is really worth to the people who work in it.
2.4
Does it survive the growth the plan assumes — at a cost that still works?
Take the growth plan’s own numbers and ask what happens at that volume: what breaks first, and what each additional customer costs to serve. If nobody has done that arithmetic, the growth plan is a hope with a spreadsheet.
2.5
If it was built for one, what would it take to serve many?
Many systems were built for a single organisation and are later sold as a platform. Making one system safely serve many customers — each seeing only their own data — is one of the hardest re-engineering jobs there is. If that work is still ahead, its cost belongs in the price.
Question 3: Are you buying software, or people?
3.1
Who runs the technology — and in which legal entity?
Ask for names and check which entity employs the people and owns the assets — the code, the servers, the contracts. Deals have come apart on the late discovery that the technology, or the team, sat somewhere else.
3.2
If the three people who know it best resigned next month, what would still work?
Ask who can fix the system, who can change it, and who can explain it. If each answer is a list of one, part of the purchase price is a salary negotiation that has not happened yet.
3.3
Which roles are missing at the size the plan assumes?
Today’s team runs today’s system. Ask which roles the growth plan needs that nobody currently fills — security, data, operations — and what they will cost. That number belongs in the model, not in the surprise column.
Question 4: Is the AI real — and where does it earn its keep?
4.1
Is the AI real, and safe to stand behind?
Ask for a live session on your scenario, not a recorded demo. Then ask the awkward questions: what does it do with an input it has never seen, who checks its output before anything depends on it, and what has it got wrong recently? A team that measures its AI answers in numbers. A team that cannot is telling you their confidence is anecdotal.
4.2
Is the data behind it an asset — or a liability?
AI is worth what its data is worth. Ask whether the rights to use that data are established in writing, how its quality is maintained, and whether it improves with use or quietly decays.
4.3
Where would AI earn its keep first — and in what order?
“We are adding AI” is not a plan. A real answer names specific tasks, the saving or revenue on each, and the order of attack — usually starting with something boring and measurable, not something impressive.
4.4
Where does this sit on a published maturity model — and can they defend the placement?
Ask them to place themselves on a published AI maturity framework and to defend it: how output quality is measured, what the AI is allowed to do without a person, what a transaction costs to run. Honest teams place themselves earlier than their marketing does. That honesty is worth more than the stage number.
Not our opinion
The research behind these questions.
These questions come from our own assessment work, and they line up with what published research says actually kills AI projects. Different houses, different methods, one picture: most AI spending does not survive contact with reality, and the buyer finds out after the money.
95%
of enterprise generative-AI pilots were producing no measurable return.
MIT NANDA · August 2025
42%
of companies had abandoned most of their AI initiatives — up from 17% a year earlier.
S&P Global Market Intelligence · March 2025
40%+
of agentic AI projects will be cancelled by the end of 2027 — escalating costs, unclear value.
Gartner · June 2025
65
data scientists and engineers interviewed on why AI projects fail. Leading causes: the problem misunderstood, the data foundations absent, the infrastructure missing, the people too few.
RAND · August 2024
Those causes are, in order, what questions 1, 2 and 3 test — before the money, instead of after. Question 4 has public yardsticks: the maturity placement in 4.4 draws on the MIT CISR Enterprise AI Maturity Model (MIT Center for Information Systems Research, 2022, 721 companies, four stages), and the discipline behind the rest of question 4 is the same discipline formalised in NIST’s AI Risk Management Framework (NIST AI RMF 1.0, January 2023) and ISO/IEC 42001:2023.
None of this research can tell you whether the system in front of you is real. It tells you the odds — and the odds are the reason to ask.
Five rules · from how we run our own assessments
How to use these questions.
Put the questions in writing before the meeting.
Rehearsed answers survive meetings. Written answers can be checked afterwards, and the difference between the two is itself a finding.
Ask for evidence, not description.
Every answer should arrive with something you can inspect — a screen shared live, a report, a log, a contract. “We have strong security” is a sentence. A test report with a date on it is an answer.
Number the questions, and let none disappear.
At the end, every question has an answer or a written reason why not. The questions that quietly vanish from the agenda have a way of being the expensive ones.
Ask what was found, not what was done.
“We reviewed the architecture” is an activity. “We found these three issues, and here is the evidence for each” is an answer. Pay for answers.
Use these questions on your advisers too — including on us.
Anyone assessing a system on your behalf should be able to show you which of these questions their report answers, and on what evidence. When we do this work, our reports are structured on these questions, and every finding carries its evidence — what we looked at, and what we found. Hold us to that.
If the answers worry you — or you cannot get answers at all.
Talk to us. The conversation is free. Consulting carries a fee.
Talk to technical counsel →If you would rather we asked them
The Technical Review answers these questions on the system in front of you.
Every finding carries its evidence, and the report is written so any competent team can act on it — ours or anyone else’s. Priced by the job, agreed before we start.
What the review covers →This document is general guidance for evaluating technology and AI systems. It is not advice on any specific transaction, and it does not replace legal, financial or technical due diligence performed on your particular case.