Skip to main content

What is model merging, and when would you use it?

Deep learning · Pro 2

Short answer

Model merging combines the weights of models fine-tuned from the same base into one model, by averaging, SLERP, or methods like TIES and DARE that drop small changes first. Use it to join skills trained separately without forgetting, to serve one model instead of several, or to fit on a device. Always test every task after merging.

Deeper answer

The change fine-tuning makes is a task vector: fine-tuned minus base weights. Adding task vectors combines skills; subtracting can remove a behaviour. Simple averaging works surprisingly well, but tasks can interfere; TIES keeps large changes and resolves sign conflicts, DARE drops changes randomly and rescales.

Unlike an ensemble, which runs several models per request, a merge serves one model at normal cost. Merging needs no GPUs for simple methods. Precise tasks may degrade, so keep separate adapters as fallbacks and check safety behaviour too.

What a strong answer covers

Answer it out loud in the mock interview about 1 min

Follow-ups to expect

Common mistakes

Learn it

Model merging: combining fine-tuned models into one

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.