August 24, 2025

Hallucination Risk Is a Known Quantity, Treat It Like One

Every serious discussion of AI tools eventually gets to the same known limitation: the tool can state something incorrect while sounding exactly as confident as when it states something correct. Treating this as a planning input, rather than an occasional embarrassment, changes how a firm should build its process.

Where the risk concentrates

It is highest on thin, ambiguous, or recent information, a small private company with little public footprint, a very recent event the underlying model has limited data about, or a specific number that requires precise sourcing rather than general description. It is lowest on well-documented, stable, publicly verifiable facts.

Why the confident tone is the actual danger

A person hedging a guess signals uncertainty naturally, 'I think,' 'probably,' a raised eyebrow. A model asked the same question in a mode where it is essentially guessing often states the guess in the same flat, declarative tone it uses for something well-established. The tone gives the reviewer no signal to slow down.

The process response

Treat every factual claim as unverified until traced to a specific source document, regardless of how confidently it reads. Build the process so that source-tracing is a required step, not an optional one a busy reviewer skips under deadline pressure. This is less about catching every error, no process catches every error, and more about making the catch rate high enough that the ones that slip through are rare and low-stakes.

A habit that reduces the risk without slowing work down

Whenever a draft states a specific number, date, or fact that will reach a client, pause and ask 'where does this come from, specifically' before moving on. If the honest answer is a source document, proceed. If the honest answer is 'the tool said so,' that is the exact moment the risk described above is live, and it takes less time to check than to explain later why an incorrect figure reached a client.

Extending the habit to numbers already in a firm's own systems

The same 'where does this come from, specifically' question applies even to figures that originated internally rather than from an AI draft, a number pulled from an old internal memo, for instance, may itself be stale or previously incorrect. The habit of asking the question every time, regardless of the number's apparent origin, is more valuable than assuming AI-produced numbers are the only ones that need checking.

Where this leaves a firm

None of this is complicated in principle, which is exactly why it gets skipped under deadline pressure. The question worth returning to before treating handling client data and AI risk with real discipline as settled is what a careful reader would actually notice if the firm got it right. On the point raised above under “where the risk concentrates,” the answer is usually specific rather than clever: hallucination risk is highest on thin or recent information, lowest on well-documented facts. Firms that build this expectation into how they train new associates find it easier to sustain once experienced staff move on, because the standard lives in a documented habit rather than in one person's memory. The gap between a firm that talks about handling client data and AI risk with real discipline and a firm that actually practices it shows up over several quarters, not in any single engagement, and it tends to show up most clearly in the small, unglamorous checks that a client never sees directly but benefits from anyway.

It also helps to name, plainly, who is responsible for keeping this working once the novelty of a new tool wears off. Someone should own the point raised under “why the confident tone is the actual danger,” check it periodically rather than assume it stays true on its own, and be the person a colleague asks when a new situation does not fit the pattern described here. Put simply: build source-tracing as a required step, not one a busy reviewer can skip. That kind of ownership, named and specific, is a small addition to a firm's process, and it is usually the difference between a good idea that is followed for a month and a standard that actually holds up over a year of real client work.

None of this needs to be elaborate to be effective. A short, dated note in a shared file, reviewed at the next quarterly check-in, is usually enough to keep the responsibility from quietly disappearing when the person who first cared about it moves on to something else.

VIPMarketing Editorial