Skip to main content

How would you set up CI/CD for an LLM application?

Deployment and LLMOps · Pro 1

Short answer

On every pull request run the normal checks (lint, types, unit tests) plus an eval suite: a regression set of prompts with code checks and judge-based scores, compared with the main branch. Block merges that fall below thresholds. Version prompts and model settings in the repository, deploy to staging first, then roll out gradually with monitoring and a quick rollback.

Deeper answer

Evals cost money and time, so run a fast smoke set on every pull request and a fuller set nightly or before release. Cache model responses where inputs are unchanged.

Treat a model version upgrade like a code change: it goes through the same pipeline. Secrets come from the CI system's secret store, never the repository.

What a strong answer covers

Answer it out loud in the mock interview about 1.5 min

Follow-ups to expect

Common mistakes

Learn it

CI/CD with GitHub Actions

Practice Deployment and LLMOps questions

More on this topic

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.