Pattern · Safety
Prompt injection defence
Assume text from users, documents and tools may carry instructions, and limit what a hijacked model could do.
Say it in the interview
Models cannot reliably tell instructions from data, so I would not rely on detection. I would limit what a hijacked model can do: least privilege per mode, approval before actions, outbound limits, and untrusted text in marked data sections with a classifier as one more layer.
When to use it
- Any app that reads email, web pages, documents or tool results
- Agents with actions
How it works
- Untrusted content → Model
- Model → may follow hidden instructions → Layers that limit harm
- Layers that limit harm → Least privilege (no send tool in read mode)
- Layers that limit harm → Human approval for actions
- Layers that limit harm → Outbound limits (known recipients only)
- Layers that limit harm → Marked data sections + a classifier
Trade-offs
- Limits damage even when detection fails
- Some useful autonomy is given up
Pitfalls interviewers probe
- Relying on "ignore previous instructions" filters alone
- Giving a summariser the power to send email