Hallucination occurs when an AI model confidently generates responses that are factually incorrect, nonsensical, or entirely fabricated, typically because it is
Language models generate the most probable next tokens given their context: they have no built-in fact checker. When the needed fact is missing, ambiguous, or weakly represented in training, the model still produces fluent text, fabricating names, citations, numbers, or APIs with full confidence. Hallucination rises with vague prompts, missing context, long generations, and questions at the edge of training data.
Hallucination is the central trust problem of generative AI: it produces convincing wrongness at scale. Every serious deployment mitigates it: grounding via RAG with citations, structured outputs, eval suites that measure faithfulness, and human gates on high-stakes outputs. Treating hallucination as an engineering target, not a curiosity, separates production teams from demo teams.
No: they're intrinsic to probabilistic generation. They can be driven toward zero for specific tasks through grounding (RAG with citations), constrained outputs, verification steps, and human review where stakes demand it.
Faithfulness evals: compare outputs against source documents or ground truth, score claim-by-claim support (human or calibrated LLM-as-judge), and track the rate over time as prompts and models change.
It supplies the facts in context so the model answers from retrieved sources rather than parametric memory, and citation requirements make unsupported claims detectable. Failure then shifts to retrieval quality, which is fixable engineering.