Why do language models hallucinate, and how do you reduce it in an app?
LLM and transformer basics · Beginner 1
Short answer
A model generates likely text, not checked facts, so when it lacks the right information it can produce a confident, plausible answer that is false. In an app you reduce this by giving it the right source text (RAG), asking it to cite and to say when it does not know, and checking outputs.
Deeper answer
Hallucination is a side effect of how models work: they are trained to continue text fluently, and nothing in plain generation checks claims against reality. Gaps in training data, questions about recent events, and very specific details such as citations, numbers and names are where it shows most.
Practical defences: ground answers in retrieved documents and instruct the model to answer only from them; allow and reward "I don't know"; ask for citations and verify that cited text exists; use structured output so code can validate fields; and measure hallucination rate on a test set so you know whether a change helped.
What a strong answer covers
- Models generate likely text, not checked facts
- Missing information leads to confident false answers
- Give source text with RAG
- Ask for citations and to admit not knowing, and check outputs
Answer it out loud in the mock interview about 1 min
Follow-ups to expect
- How would you detect a made-up citation automatically?
- Can RAG remove hallucination completely?
Common mistakes
- Claiming a better prompt alone can eliminate hallucination
- Not giving the model a way to say it does not know
Learn it
Use a chat assistant for a real task and check the answer
- At a high level, what does attention do in a transformer?
- What is a context window, and what happens when a conversation gets too long?
- Your chatbot costs about three times more per reply in Tamil than in English. Why, and what would you do?
- What is DPO, and how is it different from RLHF?
- How is a chat model like ChatGPT trained? What is the difference between SFT and RLHF?
- What does the temperature setting do?
- What is test-time compute, and how can you use it to get better answers?
- What is a token, and why do tokens matter when you build with LLMs?
- What is a large language model, and how does it produce an answer?
- What is perplexity, and what are its limits as a metric?