How do you design a product experience when the AI is sometimes wrong?
AI product management · Intermediate 2
Short answer
Set expectations honestly, make output easy to check and edit, show sources or reasoning where possible, and keep the user in control of anything consequential. Offer clear ways to correct, undo or give feedback, and design graceful fallbacks when the model is unsure, such as asking a clarifying question or saying it does not know.
Deeper answer
Patterns from Google's People and AI Guidebook and Microsoft's guidelines for human-AI interaction: explain what the system can do, show confidence where it is meaningful, make it easy to dismiss suggestions, and learn from corrections.
Match autonomy to the cost of errors: suggest rather than act for high-stakes tasks. Then measure: how often users edit, accept or reject outputs tells you where quality falls short.
What a strong answer covers
- Sets honest expectations
- Easy to check, edit or undo
- Shows sources or reasoning
- User stays in control of consequential actions
- Feedback and graceful fallbacks
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- When would you show a confidence score, and when not?
- How would you use user corrections?
Common mistakes
- Hiding that output is AI-generated
- Designing only the happy path
Learn it
- Should we build our own AI feature or buy a vendor product?
- What does a good data strategy for an AI product look like?
- Your AI feature launched, usage is high, but customer satisfaction dropped. What do you do?
- How would you measure the success of an AI feature?
- You launched an AI tool internally and nobody uses it. What do you do?
- How do you decide whether a problem is a good fit for AI?
- As a PM, how do you work with engineers on evals?
- How would you price an AI feature whose cost grows with usage?
- When should a product keep a human in the loop?