Multi-Query and Step-Back Retrieval
Explore techniques such as multi-query retrieval and step-back prompting that enhance information retrieval in RAG systems. Understand how query reformulations and abstraction increase recall and ranking quality, and learn to evaluate these methods by measuring Recall@k, MRR, and nDCG. This lesson prepares you to optimize retrieval layers for better LLM application results.
Even with well-tuned chunking strategies and high-quality embeddings, retrieval can still fail silently. The reason is straightforward: when a user types a query, your system converts it into a single vector and searches for nearby chunks in embedding space. If the user’s vocabulary doesn’t match the vocabulary in your indexed documents, that single vector lands in the wrong neighborhood, and relevant chunks never surface. Consider a developer asking “How do I handle errors?” against a codebase documentation system. The actual chunks use terms like “exception handling,” “fault tolerance,” and “retry logic,” none of which share obvious lexical overlap with the original query. The embedding model might partially bridge this gap, but a single query formulation still creates a single point of failure in the retrieval step.
Multi-query retrieval and step-back prompting are two query transformation techniques that address this problem from complementary directions. Multi-query retrieval generates several reformulations of the original query to cast a wider net across embedding space. Step-back prompting abstracts the query to a higher-level concept so that broader, foundational chunks are also retrieved. Both techniques operate in the retrieval optimization layer of the RAG pipeline, transforming the query before it ever reaches the vector store.
Multi-query retrieval mechanics
Multi-query retrieval works by replacing a single retrieval attempt with several parallel attempts, each using a different formulation of the same underlying question. This increases the probability that at least one formulation lands near the relevant chunks in embedding space.
The three-stage process
The technique follows three distinct stages that execute sequentially.
Query expansion: An LLM takes the original user query and generates three to five semantically diverse reformulations. Each variant captures a different angle, synonym set, or level of specificity. For example, “What causes high latency in microservices?” might produce variants such as “How to debug slow response times in distributed systems,” “Network bottlenecks in service-to-service communication,” and “Performance optimization for microservice architectures.”
Parallel retrieval: Each variant query is embedded independently and used to search the vector store, producing separate ranked result sets. The original query is typically included as one of the search queries as well.
Result fusion: The system merges these result sets using Reciprocal Rank Fusion (RRF). RRF scores each document with the formula
...