Search⌘ K
AI Features
Home
Courses
Grokking the Machine Learning System Design Interview

The Model Selection Framework

Understand a 5-step framework to select machine learning models by balancing accuracy, latency, cost, interpretability, and data needs. Learn to start with simple baselines like logistic regression and escalate to complex models like deep learning or LLMs only when justified. Gain insights into operational trade-offs, infrastructure costs, and effective communication to excel in ML system design interviews.

We'll cover the following...

You are designing a fraud detection system for a fintech company that processes millions of transactions per day. The interviewer asks, “What model would you use?” This is not a recall test for the latest research architecture. It is a design question that affects downstream decisions, including serving infrastructure, latency budgets, training pipelines, and operational cost. Your answer shows whether you can balance model quality with production constraints.

Model selection in ML system design interviews functions as a litmus test for engineering maturity. Candidates who immediately reach for a transformer-based architecture without justifying the added complexity are penalized. Candidates who start with a simple baseline and then articulate precise conditions for escalation demonstrate the kind of production thinking that MAANG interviewers reward. The difference between these two responses is not knowledge of models but fluency in reasoning about trade-offs.

This lesson gives you a repeatable framework for that reasoning. Every model choice you make in an interview should be evaluated along five trade-off axes: accuracy (how well the model performs on offline and online metrics), latency (whether inference fits within the serving SLA), cost (training and serving expenses at production scale), interpretability (whether stakeholders or regulators can understand decisions), and data requirements (how much labeled data the model needs to outperform alternatives). These five axes organize the rest of this lesson and will become your default vocabulary for defending model choices under pressure.

The following decision flowchart captures how these axes interact to guide you from task definition to a recommended model family:

Model selection starts with constraints, defaulting to the simplest viable model before escalating
Model selection starts with constraints, defaulting to the simplest viable model before escalating

With this map in hand, the next step is understanding why the simplest path on that flowchart is almost always the correct starting point.

Start simple and justify escalation

Logistic regression remains the workhorse for click-through rate prediction at companies like Meta and Google. It delivers sub-millisecond inference, full interpretability through direct feature weights, and trivial deployment to thousands of serving nodes. GBDT (Gradient Boosted Decision Tree)An ensemble learning method that builds a sequence of decision trees, where each new tree corrects the errors of the previous ones, producing strong predictive performance on tabular data. models such as XGBoost and LightGBM dominate tabular ranking and fraud detection because they handle feature interactions, missing ...