Skip to main content

A client tells you their AI assistant invented a policy clause and a customer acted on it. What do you do?

Client delivery · Pro 1

Short answer

First contain it: tell the client, correct the customer's information, and block similar answers for now. Then read the trace to find the cause: wrong documents, poor retrieval, or the model answering beyond its sources. Fix that, add the case and similar ones to the test set, and add an automatic check of claims against sources.

Deeper answer

I would look at the full trace: the question, the retrieved passages and the answer. If the clause was not in any passage, the model answered beyond its sources, so I would tighten the instruction to answer only from sources and say when they are silent, and add an entailment or judge check on every claim.

If retrieval missed the real clause, I would fix chunking, filters or the index. I would search logs for other answers on the same topic, report findings and fixes to the client in writing, and agree a review step for policy answers until the checks prove themselves.

What a strong answer covers

Answer it out loud in the mock interview about 1.5 min

Follow-ups to expect

Common mistakes

Learn it

Catching hallucinations: why models make things up and how to detect it

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.