Introduction
Five Cents explores how to recognise when AI gets it wrong, with practical checks you can use before trusting an answer. We’ll cover invented facts and citations, misleading calculations, and biased recommendations. Then we’ll bring those checks together into a repeatable routine.
The key points
The starting point is the difference between sounding convincing and having evidence. A generative AI system can produce fluent, detailed language without establishing that its claims are true. This failure is often called hallucination: an answer contains something false or unsupported, presented as though it were settled.
That doesn’t mean every answer is unreliable. It means polish, precision, and confidence are not reliable measures of accuracy. For anything consequential, ask what would actually establish the claim: a document, a calculation, a direct observation, or an appropriate expert assessment. Watch especially for exact dates, quotations, statistics, or names without checkable support. Another warning sign is an answer that quietly resolves an ambiguity instead of asking what you mean.
Imagine this hypothetical case: you ask for the dissertation title of a little-known researcher, and the system supplies one without a traceable record. The safer response would acknowledge that it cannot establish the title. Some research suggests that evaluations rewarding answers more than admissions of uncertainty can encourage guessing. That is one proposed explanation, not the whole story. Asking the model to separate supported facts from inference can help, but its explanation still needs checking.
Start with citations, because they can make an unsupported answer look independently verified. In an invented example, an AI provides a study title, several authors, a journal, and a digital identifier to support a health claim. First, search the exact title in the journal’s records or a trusted scholarly index. Check whether the title, authors, date, and identifier belong to the same paper.
Then read what the paper actually found. Does it concern the relevant population? Does it show causation, or just an association? A genuine study can still be misrepresented. If you cannot locate the reference, treat the claim as unverified rather than accepting the citation’s scholarly appearance. A citation gives you something to investigate. It is not proof by itself.
Numbers require a different check
Numbers require a different check. Here’s a hypothetical example: a price rises from eighty to one hundred. The increase is twenty, but that is twenty-five per cent of the original eighty, not twenty per cent. The new price is one hundred and twenty-five per cent of the old price. To check it, identify the starting value, subtract it from the new value, then divide the difference by the starting value. Use a calculator independently.
Also check the units and inputs. A correct calculation using monthly income and annual costs can still produce a useless comparison. Finally, examine the conclusion. A price increase alone does not establish why it happened. Arithmetic, assumptions, and reasoning can fail separately, so a calculator fixes only part of the problem.
Bias needs another approach, because an answer can contain accurate facts while applying inconsistent standards. Consider a controlled test using fictional job applicants. Keep qualifications and experience identical, but change only a name or gender cue. Ask for an assessment and suggested interview questions.
Compare whether the system demands more evidence from one applicant, makes different assumptions about leadership, or recommends different levels of responsibility. Look beyond wording to standards and conclusions. One pair of responses does not prove systematic discrimination. Outputs can vary. Repeat the test across several examples, with clear assessment criteria, and in more than one language where relevant.
Differences are signals to investigate, not a complete diagnosis. Human judgment should also be guided by transparent criteria rather than intuition alone. Bring these checks together by isolating the exact claim that matters. Find its supporting source and confirm that the source says what is claimed. Recalculate consequential numbers, including units and assumptions. Seek a primary record or reliable independent confirmation for important facts.
Question the framing
Then question the framing. Was the request ambiguous, or did the answer leave out relevant people or alternatives? Don’t mistake agreement between two AI tools for independent verification. They may repeat the same error. Browsing and confident self-correction are not guarantees either. Match your effort to the stakes. A casual draft needs less scrutiny than advice affecting health, rights, money, safety, or employment, where authoritative guidance or qualified review may be necessary.
The useful habit is to check support, not confidence; verify both calculations and assumptions; and look for inconsistent standards. You don’t need to investigate every sentence equally, but you do need to identify which errors would matter. To build on this, you can generate two new Five Cents podcasts: Checking Scientific Claims and Designing Fair AI Assessments.
And with that, you're up to speed in a few minutes.