Skip to main content

Your LLM bill doubled this month, but traffic did not. How do you investigate?

System design · Pro 2

Short answer

Break the cost down by feature, model, and input versus output tokens, day by day, to find when and where it changed. Typical causes: a prompt or context that grew, retries or agent loops, a switch to a more expensive model, caching that stopped working, or one customer's heavy use. Fix the cause, then add per-feature budgets and alerts.

Deeper answer

Cost is tokens times price, so a jump with flat traffic means more tokens per request or a higher price per token. Good usage logging, with the feature, model, tokens and cache hits on every call, turns this from guesswork into a quick query.

Look at the date the change started and compare with deployments. A new retrieval setting that sends twenty chunks instead of five, a longer system prompt that broke prompt caching, or a retry bug can each multiply cost. Afterwards, set budgets per feature and alert on cost per request, not just the monthly total.

What a strong answer covers

Answer it out loud in the mock interview about 1.5 min

Follow-ups to expect

Common mistakes

Learn it

Cost control

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.