How does k-means work, and how do you choose k?
Machine learning basics · Intermediate 1
Short answer
k-means picks k centres, assigns each point to its nearest centre, moves each centre to the mean of its points, and repeats until assignments stop changing. To choose k, try several values and look at the elbow in within-cluster distance, the silhouette score, and above all whether the clusters are useful for the business question.
Deeper answer
k-means assumes roughly round clusters of similar size and is sensitive to feature scale and outliers, so scale features first. It finds a local optimum, so run it several times with different starts; k-means++ initialisation helps.
The elbow method plots within-cluster sum of squares against k and looks for where improvement flattens. Silhouette measures how much closer points are to their own cluster than to the next one. Neither is decisive: for customer segments, the right k is often the number of groups the marketing team can actually act on. If clusters are irregular in shape, consider DBSCAN or a Gaussian mixture.
What a strong answer covers
- Explains assign-to-nearest and move-to-mean steps
- Repeats until stable
- Elbow method or silhouette score to choose k
- Business usefulness decides
- Scale features; sensitive to outliers and shape
Answer it out loud in the mock interview about 1.5 min
Follow-ups to expect
- Why must features be scaled for k-means?
- When would DBSCAN be a better choice?
Common mistakes
- Not scaling features
- Treating the elbow as a precise answer
Learn it
- How would you tell whether a new model really performs better in an A/B test?
- Explain the bias-variance trade-off.
- What is the difference between a random forest and gradient boosting?
- What is data leakage, and how do you prevent it?
- Give an example of feature engineering that made a model better.
- How do you know whether you have enough training data?
- How do you handle a heavily imbalanced classification problem, such as fraud detection?
- What is the difference between linear and logistic regression?
- How do you deal with missing values in a dataset?
- Walk me through an ML project you did, end to end.
- What is overfitting, and how do you spot it?
- Explain precision and recall with an example.
- What do L1 and L2 regularisation do?
- What is the difference between supervised and unsupervised learning?
- Why do we split data into training, validation and test sets?
- What is the difference between training and inference?