Model Selection and Bias-Variance
Explore how to select the appropriate model for customer churn prediction by balancing model complexity with interpretability. Understand diagnosis of underfitting and overfitting using training and validation scores, and learn to predict the impact of model and data changes. This lesson emphasizes clear reasoning about model choice under constraints, supporting actionable stakeholder decisions.
The interviewer sets up the problem: “Predict customer churn from weekly features. Stakeholders want reasons they can act on.” Then comes the question: “Would you use logistic regression or a tree-based model for this churn problem?” The prediction unit is one customer at the end of a week, and the label is whether that customer churns during the chosen future window.
A weak answer is, “I would use gradient boosting because it is more accurate.” That answer overlooks the decision constraints. A complex model can capture interactions, but it may yield unstable results on a limited labeled dataset and make stakeholder explanations harder. It also provides no evidence that the extra flexibility improves validation performance enough to justify its cost.
A stronger answer starts with the constraint. The baseline must produce useful predictions, but it must also support a conversation about why a retention action was proposed, so LogisticRegression is a defensible baseline before we consider a tree-based model. We would begin with LogisticRegression using clearly defined weekly features such as days_since_last_purchase, support_tickets_30d, and price_increase_flag. The model gives a transparent directional explanation. Holding the other included features fixed, a positive coefficient for support_tickets_30d associates more recent tickets with higher predicted churn risk. That association does not establish that tickets cause churn.
A decision tree becomes a reasonable next baseline when the business question suggests thresholds or interactions. For example, churn risk may rise only when days_since_last_purchase > 30 and price_increase_flag = 1. A shallow tree can express that rule directly. The cost is that trees can fit accidental patterns in the training sample as depth grows, so their predictions and explanations can change substantially with different samples.
Baseline rule: Choose the simplest model that meets the decision constraint, then ask whether validation evidence justifies additional flexibility.
A useful answer names what would change ...