What it is
Before you act on AI advice about money, tax or law, check it against a reliable source or with a professional. This applies when you use a chatbot as an adviser: how to price, whether an expense is deductible, what a contract or a regulation requires.
In a large company, advice passes through finance, legal or a manager before anyone acts on it. In a small firm the owner often asks the AI and acts alone. Nobody else is there to catch a wrong answer.
What the evidence shows
- AI advice helped some owners and hurt others. In a randomised experiment with 640 small-business owners in Kenya, half got a business adviser built on GPT-4, used through WhatsApp. The other half got a standard business guide. On average, the AI made no difference to the firms’ results.
- The firms that were already struggling did worse. Owners whose firms were performing poorly at the start did nearly 10% worse with the AI. Owners whose firms were doing well may have gained over 15%.
- The gap came from which advice owners acted on. The researchers found that it did not come from different questions or different advice. It came from the advice each owner chose to put into practice.
The study tested AI advice, not a check on it. No study has yet tested whether checking advice first prevents the harm. The study also used GPT-4 in 2023, in one country.
How to do it
- Decide which questions need a check. Anything about tax, legal duties, contracts, employment, borrowing, or a price change on your main products. Anything that is costly or hard to undo.
- Ask the AI what its answer is based on. Ask for the rule, the law or the source. Then open the source. AI tools can name sources that do not exist.
- Check against an official source. Use the tax authority’s guidance, the text of the regulation, or your bank’s or insurer’s own terms.
- For larger decisions, pay for an hour of a professional’s time. Bring the AI’s answer and ask them to check it. This costs less than their full advice, and far less than a wrong decision.
- Try it small first. If the advice is about pricing or marketing, test it on one product or one month before you change everything.
- Write down what you acted on and what happened. After a few months you will know where the AI’s advice holds up for your business.
What to measure
- How often a check changes the AI’s answer.
- The results of decisions taken on AI advice, against what you expected.
Where it fails
- The check costs time and money. Keep it for decisions that are costly or hard to undo.
- The source looks right but is not. A confident answer with a citation can still be wrong. Open the source.
- The professional is not up to date either. Ask where their answer comes from too.
By company size
- Micro and small: you are the check. Use steps 1 to 4 for every question on tax, law and money.
- Medium: send AI advice on these topics through the person who handles finance or legal matters before anyone acts on it.
Sources
- Otis, Clarke, Delecourt, Holtz and Koning (2026). The Uneven Impact of Generative AI on Entrepreneurial Performance: Evidence from a Field Experiment in Kenya. Management Science (published 10 July 2026). What this study did and found
Related practices
- Keep a person checking anything that goes to a customer or carries legal weight Before AI output leaves the company or is relied on in a decision, a named person checks it against real sources and signs it off. Make the check a step in the process, with time allowed for it. Do first · Effort: medium (weeks, a few people)
- Make the AI show what its answer is based on A score or an estimate with no reason attached is hard to judge. Choose and set up AI tools so that each answer comes with what it rests on: the marked part of the image, the similar past cases, the passage in the document. Do next · Effort: medium (weeks, a few people)
- For each kind of task, test whether a person with AI beats either one alone Do not assume that adding a person to AI, or AI to a person, gives the best result. Test it for each type of task before you settle the process. Do next · Effort: medium (weeks, a few people)