Skip to main content

Your neural network overfits. What do you try?

Deep learning · Intermediate 2

Short answer

Confirm it first: training loss keeps falling while validation loss rises. Then get more or augmented data, add regularisation such as dropout and weight decay, use early stopping on validation loss, shrink the model, or start from a pretrained model. Change one thing at a time and compare on the same validation set.

Deeper answer

Data augmentation is often the biggest win for images and audio: random crops, flips, colour changes or noise make the model see varied examples. Dropout randomly zeroes activations during training; weight decay penalises large weights; early stopping keeps the checkpoint with the best validation loss.

Also check for problems that look like overfitting: leakage between training and validation, a validation set from a different distribution, or a bug in evaluation mode, such as forgetting model.eval so dropout and batch norm behave as in training.

What a strong answer covers

Answer it out loud in the mock interview about 1.5 min

Follow-ups to expect

Common mistakes

Learn it

PyTorch

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.