Semantic Search: Problem Framing and Requirements
Explore how to frame semantic search problems focusing on query understanding, selecting appropriate business metrics, and addressing scale and latency constraints. Gain insights into decomposing search query intent, entity recognition, and contextual disambiguation, while understanding how these elements influence downstream system design decisions in production-scale ML semantic search systems.
Search is one of the most common ML system design topics in interviews at large tech companies. It cuts across multiple layers of the ML stack, from data ingestion and feature engineering to model serving and online evaluation. Interview prompts may ask candidates to design a semantic search for Google, a product search for Amazon, or a people search for LinkedIn. In each case, the interviewer wants to see whether you can frame the problem before proposing a solution. This lesson walks through that framing step for semantic search. We will treat query understanding as an upstream ML problem, identify the product and search-quality metrics that measure performance, and define the scale constraints that rule out simple approaches. These three areas shape the downstream design decisions, from model architecture to serving infrastructure.
The industry-standard architecture for semantic search pairs dense vector embeddings with approximate nearest neighbor (ANN) algorithms. Understanding why that architecture exists requires understanding the problem space first. Candidates who skip problem framing and jump straight to model selection almost always optimize for the wrong objective or violate latency constraints.
Most candidates underprepare for this step.
Query understanding as a prerequisite ML problem
Retrieval quality in a semantic search system is bounded by how well the system understands the query. A query like “apple” is useless to a retrieval engine without context. Query understanding is the upstream ML problem that resolves this ambiguity and routes the request to the right retrieval pipeline. It decomposes into three sub-problems.
Intent classification
Every query carries an implicit intent. Intent classification is a text classification task that assigns a query to one of several predefined categories, most commonly navigational, informational, or transactional, to determine which retrieval and ranking pipeline should handle the request. Consider these examples across platforms.