How do you handle a heavily imbalanced classification problem, such as fraud detection?
Machine learning basics · Intermediate 2
Short answer
First pick the right metric: accuracy is useless when 99.5% of cases are normal, so use precision, recall, F1 or the area under the precision-recall curve. Then use class weights or resampling, tune the decision threshold for the business cost of each error, and validate with stratified splits.
Deeper answer
With one fraud case in two hundred, a model that always says "not fraud" is 99.5% accurate and worthless. The real question is the cost of a missed fraud versus a false alarm that annoys a customer, so agree that with the business before modelling.
Techniques: class weights so mistakes on the rare class cost more; undersampling the majority or oversampling the minority, done only inside training folds; models such as gradient boosting that cope well with imbalance; and choosing the threshold on a precision-recall curve instead of using 0.5. Use stratified cross-validation so each fold has some positive cases, and report results at the threshold you will actually use.
What a strong answer covers
- Accuracy misleads; use precision, recall, F1 or PR-AUC
- Class weights or resampling
- Tune the threshold to business cost
- Stratified splits
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- Why can SMOTE inside the whole dataset leak?
- How would you choose the threshold?
Common mistakes
- Reporting accuracy
- Resampling before the train-test split
Learn it
- How would you tell whether a new model really performs better in an A/B test?
- Explain the bias-variance trade-off.
- What is the difference between a random forest and gradient boosting?
- How does k-means work, and how do you choose k?
- What is data leakage, and how do you prevent it?
- Give an example of feature engineering that made a model better.
- How do you know whether you have enough training data?
- What is the difference between linear and logistic regression?
- How do you deal with missing values in a dataset?
- Walk me through an ML project you did, end to end.
- What is overfitting, and how do you spot it?
- Explain precision and recall with an example.
- What do L1 and L2 regularisation do?
- What is the difference between supervised and unsupervised learning?
- Why do we split data into training, validation and test sets?
- What is the difference between training and inference?