What an AI hallucination is
Language models generate text by predicting likely next words based on patterns learned in training. They do not check a source of truth unless the system gives them one. When a question falls outside what the model knows, or the prompt is vague, it can produce a fluent answer that is wrong.
In healthcare the stakes are higher than in most fields. A hallucinated answer could give a patient wrong instructions or misstate what a plan covers. It could invent a detail in a summary or cite a rule that does not exist. Errors in patient-facing channels carry safety, trust, and liability risk, and they are hard to spot because the tone stays confident.
Three mitigations do most of the work. Grounding, which means answering from the organization's own documents, data, and policies instead of the model's general knowledge. Citations, so staff can check where an answer came from. And human review for clinical or high-stakes content. Narrow scope and a clear handoff to a person help too.