Skip to main content

How do you decide which model size to use for a feature?

Prompting, RAG or fine-tuning · Pro 2

Short answer

Run your eval set on a few model sizes and compare quality, latency and cost per request. Choose the cheapest model that meets your quality bar for that task. Many products use a mix: a small, fast model for simple steps such as classification, and a larger one only where reasoning quality matters.

Deeper answer

Quality differences between models vary by task: on easy tasks a small model may match a large one; on hard reasoning it may not. Only your own test set tells you.

Routing is a common pattern: classify the request, send easy ones to a cheap model and hard ones to a strong model, or try the cheap model first and escalate when a check fails. Revisit the choice when new models are released, because the price and quality frontier moves quickly.

What a strong answer covers

Answer it out loud in the mock interview about 1 min

Follow-ups to expect

Common mistakes

Learn it

Quality, cost and speed trade-offs

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.