AI hallucination refers to the phenomenon where a large language model (LLM) or AI system generates information that is factually incorrect, fabricated, or nonsensical, while presenting it with the same confidence as accurate information. The term borrows from psychology, describing outputs where the AI "perceives" content that isn't grounded in reality — effectively making things up.
Hallucinations are an inherent limitation of current generative AI architectures. LLMs predict the most statistically likely next token based on training data, without a separate mechanism for verifying factual accuracy. This means models can produce plausible-sounding but entirely fabricated citations, statistics, legal precedents, medical information, or historical events.
Common enterprise hallucination risks include AI-generated contracts with fabricated legal clauses, medical documentation with incorrect drug dosages, financial reports citing non-existent data, customer service responses giving wrong policy information, and code that references functions or APIs that don't exist (known as "ghost packages" or hallucinated dependencies, which can be exploited by attackers).
IBM research suggests AI hallucinations affect up to 20% of LLM responses in unconstrained settings, though enterprise deployments with retrieval-augmented generation (RAG) and structured prompting can significantly reduce this rate.
Organizations managing AI hallucination risk should implement: human review workflows for AI-generated content in high-stakes contexts (legal, medical, financial); RAG architectures that ground AI outputs in verified document sets; output validation pipelines that cross-check AI claims against authoritative sources; employee training on hallucination risks; and clear policies on when AI-generated content requires human verification before use.