A vendor says their ML intrusion detector is 99% accurate. Why might it still drown your SOC in false alarms?
AI security · Intermediate 2
Short answer
Because attacks are rare. If one event in 10,000 is malicious, a model that wrongly flags just 1% of normal events still raises about 100 false alarms for each real attack. Accuracy hides this. I would ask for precision, recall and the false positive rate on data like ours, and test it on our own logs first.
Deeper answer
This is the base rate problem. With a tiny share of positives, even a small false positive rate produces far more false alerts than true ones, so precision is low. Accuracy looks great because predicting 'normal' every time is already almost perfectly accurate.
Better questions for the vendor: precision and recall at the threshold they recommend, the false positives per day expected at our event volume, what data they tested on, and how the model handles drift and attackers who adapt. Then run a pilot on our own logs, label a sample of alerts, and tune the threshold to what analysts can handle. Combine ML scores with rules and context instead of alerting on the score alone.
What a strong answer covers
- Explains the base rate: attacks are rare
- Shows the arithmetic or that false alarms outnumber real ones
- Asks for precision, recall and false positive rate instead of accuracy
- Pilots on own data and tunes the threshold
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- How would you pick the alert threshold?
- What happens when attackers learn how the model works?
Common mistakes
- Accepting accuracy as the measure on rare events
- Trusting the vendor's test results without a pilot on your own data
Learn it
Machine learning for threat detection
- Attackers now use LLMs to write phishing emails. What changes for defenders, and what still works?
- What is attack success rate in AI red teaming, and what are its limits as a metric?
- Threat model an LLM chatbot for a retail bank that can answer questions and show a customer's recent transactions.
- What is the CIA triad, and how would you use it to judge a security problem?
- What is data poisoning, and how would you protect a model you train or fine-tune from it?
- As a security engineer, how would you protect a company's finance team from deepfake voice or video calls from the 'CEO'?
- You find code that builds a SQL query by joining strings with user input. What is the risk and how do you fix it?
- What is the difference between hashing and encryption, and which do you use for passwords?
- An employee's laptop shows signs of malware at 11 pm. Walk me through your incident response.
- Explain least privilege. How would you apply it to a new team's cloud and app access?
- Your SOC wants an LLM copilot to summarise alerts and suggest actions. What are the risks and how do you manage them?
- What is tool poisoning in MCP or agent tools, and how do you defend an agent against it?
- What is MITRE ATT&CK, and how does a detection team actually use it?
- What is a model extraction attack, and how would you defend a model you serve through an API?
- Walk me through the OWASP Top 10 for LLM applications. Which risks would you check first in a RAG chatbot?
- How would you defend a company against phishing? Where do SPF, DKIM and DMARC fit in?
- A teammate wants to load a model file they downloaded from a public hub. What supply chain risks do you check?
- You are asked to red-team your company's customer support chatbot before launch. How do you go about it?
- Tell me about a security incident or near miss you handled. What did you do and what changed afterwards?
- Staff are pasting company data into public AI tools without approval. How would you handle shadow AI?
- Your SOC gets thousands of SIEM alerts a day and analysts are ignoring them. How do you fix alert fatigue?