What is the difference between linear and logistic regression?
Machine learning basics · Beginner 3
Short answer
Linear regression predicts a number, such as a house price, by fitting a weighted sum of the features with squared error loss. Logistic regression predicts the probability of a class, such as spam or not, by passing that weighted sum through a sigmoid and training with log loss; you then pick a threshold to make a decision.
Deeper answer
Despite its name, logistic regression is a classification model. Its weights are interpretable as changes in the log odds, which is why it is still used in credit scoring and medicine where explanations matter.
Both are strong baselines: fast, explainable and hard to beat when relationships are roughly linear. Both benefit from scaling and regularisation. For more than two classes, logistic regression extends to softmax regression.
What a strong answer covers
- Linear predicts a number
- Logistic predicts a class probability
- Logistic uses a sigmoid and log loss
- A threshold turns probability into a decision
Answer it out loud in the mock interview about 1 min
Follow-ups to expect
- Why not use linear regression for a yes or no target?
- How do you interpret a logistic regression weight?
Common mistakes
- Calling logistic regression a regression model for numbers
- Forgetting the threshold step
Learn it
- How would you tell whether a new model really performs better in an A/B test?
- Explain the bias-variance trade-off.
- What is the difference between a random forest and gradient boosting?
- How does k-means work, and how do you choose k?
- What is data leakage, and how do you prevent it?
- Give an example of feature engineering that made a model better.
- How do you know whether you have enough training data?
- How do you handle a heavily imbalanced classification problem, such as fraud detection?
- How do you deal with missing values in a dataset?
- Walk me through an ML project you did, end to end.
- What is overfitting, and how do you spot it?
- Explain precision and recall with an example.
- What do L1 and L2 regularisation do?
- What is the difference between supervised and unsupervised learning?
- Why do we split data into training, validation and test sets?
- What is the difference between training and inference?