I once asked an AI tool for the specific regulation number covering a VA benefit I was trying to explain to a buddy. It gave me a clean, confident answer. Cited the section. Sounded exactly like something a lawyer would say. It was completely made up. Not close. Not outdated. Fabricated, from nothing, delivered with the same tone it uses to tell you the capital of France.
That's the part nobody warns you about when you start using AI. It doesn't hedge the way a person does. A mechanic who isn't sure will say "let me check on that." A know-it-all coworker at least gives you tells — the pause, the over-explaining, the way their story changes when you push back. AI doesn't do any of that. It says wrong things with the exact same confidence as right things, in the exact same voice, at the exact same speed. If you're new to this stuff, or if you've been burned once already and now don't trust any of it, that's the real problem you're solving. Not "is AI good or bad." It's "how do I know when to believe it."
I spent 20 years as an Air Traffic Controller before I built businesses running on AI. In ATC, a confidently wrong answer isn't an inconvenience — it's how people get hurt. So I built habits around double-checking things that sound certain but might not be, and those same habits are what keep AI useful instead of dangerous in my businesses now. None of this requires a technical background. It just requires knowing what to look for.
Why AI sounds so sure of itself even when it's guessing
Here's the thing to understand, and it's not complicated. AI models like ChatGPT and Claude generate answers by predicting the most likely next word based on patterns in enormous amounts of text. They're not looking anything up in a filing cabinet and checking it against a source. They're writing what sounds right based on everything they've seen before. Most of the time, "sounds right" and "is right" line up, because most of what's true also sounds true. But when they don't line up — when the model hits a gap in what it actually knows — it doesn't switch to a different, more careful voice. It just keeps generating in the same fluent, confident tone, because that's the only tone it has.
This is what people mean when they say an AI "hallucinated" something. It's not lying, because lying requires knowing the truth and choosing to hide it. It's closer to a very well-read person who's used to always having an answer, and who has never once said "I don't know" out loud, so they've stopped noticing when they're bluffing. The model isn't aware it's bluffing. It just fills the gap the same way it fills everything else.
Specific numbers, dates, citations, and names are where this bites hardest. Anything vague — "give me ideas for a follow-up email" — has a lot of room to be reasonably right. Anything specific — "what's the exact IRS mileage rate" or "what's the code section for this VA rule" — has exactly one right answer and infinite wrong ones that sound just as clean. That's your first flag right there: the more specific and factual the claim, the more skeptical you should be by default.
The three checks that catch most of it
You don't need a system with twelve steps. You need three habits that take fifteen seconds each.
First, ask yourself if the claim is checkable. Most bad AI answers aren't in categories like brainstorming or drafting or organizing your thoughts — those are low-risk because you're the one deciding what's useful. The danger zone is anything with a specific, verifiable fact buried in it: a law, a regulation, a statistic, a historical date, a person's title, a technical spec. If the answer contains one of those and you're about to act on it or repeat it to someone else, that's your cue to verify it somewhere else before you trust it.
Second, ask the AI to show its work instead of just its conclusion. This sounds simple but it changes the output more than you'd expect.
Try this prompt: Before you answer, tell me how confident you are in this and what part of your answer you're least sure about.
This doesn't guarantee a perfect answer, but it forces the model to slow down and flag the shaky parts instead of presenting everything with equal certainty. You'll often get something like "I'm fairly confident on the general process, but you should double check the exact dollar threshold since that changes." That's gold. That's the model telling you exactly where to point your skepticism.
Third, cross-check with a second source, and ideally a second model. If Claude and ChatGPT give you the same specific fact independently, that's a decent sign it's a real, well-documented thing. If they disagree, or if one of them is vague where the other is oddly specific, that disagreement is the warning. I run several businesses and I've made it a habit to run anything with a hard number or legal claim through a quick second check before it goes into a proposal, an estimate, or anything a client will see.
Watch for these specific tells
There are patterns that show up more often in wrong answers than right ones, and once you know them, you'll start spotting them fast.
Round numbers used with suspicious precision are one. If an AI tells you something happened "in exactly 47% of cases" with no source, that number likely isn't coming from a real dataset — it's coming from what sounded statistically plausible. Real statistics usually come with a source attached, or the model will tell you it's a general estimate.
Citations that feel slightly off are another. A fake citation often looks almost right — a real-sounding journal name, a plausible page number, an author name that could exist. This is one of the most well-documented failure modes with AI models, because they're built to produce fluent, plausible-looking text, and a citation is just a very specific kind of text pattern. If you ask for a source and get something suspiciously tidy, search for that exact source yourself before you cite it anywhere.
Overly tidy timelines are a third one. Real history and real processes have messy edges — exceptions, delays, regional differences. If an AI gives you a clean, step-by-step version of something that in real life is famously bureaucratic and inconsistent, like a VA claims process or a permitting timeline, be suspicious of the tidiness itself. It's smoothing over the mess because smooth is what it's good at producing.
What to do when you catch one
The point isn't to stop trusting AI. It's to know where your leash needs to be short and where it can be long. For brainstorming, drafting, organizing, summarizing your own material, rewriting something in a different tone — that's low-risk territory. Let it run. For anything with a specific number attached to money, law, health, or safety, treat the AI as a fast first draft, not a final answer.
A good habit is building the verification step into your prompt itself, instead of trusting yourself to remember to do it after.
Try this prompt: Answer this, but clearly separate what you're confident is factually accurate from what could be outdated or uncertain, and tell me what I should verify independently.
That prompt does the sorting work for you. You're not asking the AI to be right about everything — you're asking it to be honest about what it's not sure of, which is a much more reasonable ask and one it's actually decent at when you request it directly.
I use this constantly when I'm pulling together anything client-facing, whether it's a proposal, a piece of content, or research for a decision that costs real money. The AI does the heavy lifting on structure and speed. I do the fifteen seconds of skepticism on anything specific. That combination is what makes AI actually useful instead of a liability — not blind trust, and not blind distrust either.
One thing to do today
Pull up the last AI conversation where you got a specific fact, number, or citation you haven't verified yet. Go check it against a real source right now — a government site, a manufacturer spec sheet, a second AI model, anything independent. You'll either confirm it was right, which builds real trust instead of assumed trust, or you'll catch a confidently wrong answer before it turns into a mistake you had to clean up later. Either way, you'll walk away knowing exactly how much to trust that answer next time, instead of guessing.