Skip to main content

What is a model extraction attack, and how would you defend a model you serve through an API?

AI security · Pro 1

Short answer

Model extraction is when an attacker sends many queries to your API and uses the answers to train a copy of your model, stealing its value or helping plan other attacks. Defences: authentication, rate and quota limits, monitoring for unusual query patterns, returning only what users need instead of full probabilities, terms of use, and watermarking where possible.

Deeper answer

Each answer is a labelled training example for the attacker. Detailed outputs such as full probability scores leak more than a single label. A stolen copy can also be used offline to find adversarial inputs that then work against your real model.

Defences come in layers: require API keys tied to accounts; set quotas and rate limits; watch for many diverse or systematic queries from one account or a group of new accounts; reduce output detail; and price or gate bulk access. For LLMs, watermarking outputs and contract terms help with proof and deterrence. Related privacy attacks, membership inference and training data extraction, need limits on what the model memorises, such as deduplicating training data, and testing whether it repeats sensitive records.

What a strong answer covers

Answer it out loud in the mock interview about 1.5 min

Follow-ups to expect

Common mistakes

Learn it

Model extraction and privacy attacks

Practice AI security questions

More on this topic

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.