Understanding Retrieval and Generative Models

We've entered a world where our computers are getting better at understanding and responding to us. Early machines were straightforward. They followed strict rules, like following a recipe to the last letter. Ask for a cookie, get a cookie, as long as the recipe specified it was indeed a cookie.

First, we have retrieval models such as Term Frequency-Inverse Document Frequency (TF-IDF) or Best Match 25 (BM25). Think of a well-organized librarian who knows exactly where every book is placed and which ones contain the information you need. These models are good at sifting through large amounts of data to find and retrieve the most relevant information for the task at hand. They help by pulling data from a knowledge database to provide context or facts necessary for generating accurate responses.

Then, we have generative models like the Generative Pre-trained Transformer (GPT). Rather than retrieving existing information, these models use what they learned during training to generate new content. Given a prompt, they can compose a comprehensive answer, or given a hint of an idea, they can expand it into a detailed narrative.

So, as we go about making these machines smarter, we're teaching them to both retrieve existing information and generate new content. Before we can truly understand retrieval-augmented generation, though, we need to look more closely at each model type.

What are retrieval models?

Retrieval models are built to navigate large amounts of data and find information relevant to a specific query. Unlike models that categorize or classify data based on learned examples, retrieval models focus on matching query criteria to the data they have access to. In a question-answering system, for instance, a retrieval model sifts through a database to fetch the details that best answer the user's question.

In text processing, these models analyze a large corpus and identify passages that most closely relate to the query. In image retrieval, the same principle applies: these models analyze visual content, recognizing objects, colors, patterns, or scenes within a collection to find images most relevant to a given query.

The images below illustrate this: a database of images on the left, an input query image in the middle, and the retrieved match on the right.

One of the main strengths of retrieval models is their efficiency. They can scale across very large datasets without needing to deeply understand or generate anything new. The tradeoff is that they depend on the quality and structure of the data they search. If the right answer isn't in the database, they can't retrieve it.

What are generative models?

Generative models work by learning the joint probability distribution of input features and output labels. Unlike retrieval models, which fetch existing information, generative models learn the underlying patterns in data and use those patterns to produce new content.

In image generation, a generative model trained on animal photos can generate new, realistic images of animals it has never seen. It does this not by memorizing specific animals, but by learning general features like textures, shapes, and colors that define what an animal looks like. The image below shows what this can produce, including some genuinely strange outputs, like a hybrid of a whale and a rabbit, which illustrates both the creative potential and the unpredictability of these models.

In natural language processing, generative models can compose coherent, contextually relevant text. ChatGPT is a prime example. After training on large text corpora, it can produce new sentences, paragraphs, or entire articles that match the style and content of the training data. It does this by learning linguistic structure, vocabulary patterns, and stylistic elements, not by retrieving stored answers.

Generative models vs. retrieval models

While generative models may seem superior on the surface, they are not better than retrieval models in every situation. There are some important differences to consider:

• Complexity and computational cost: Generative models learn the joint probability distribution of inputs and outputs, which is computationally expensive. Training takes longer and requires more resources, especially with high-dimensional data.

• Precision in specific information retrieval: When you need a specific, verifiable fact, retrieval models tend to perform better. They are built to return the most relevant data without generating anything new, which keeps the output accurate and traceable.

• Transparency: Generative models are black boxes. It's hard to know exactly why they produced a specific output. Retrieval models are more transparent. You can trace a retrieved document back to its source and see exactly why it was selected.

Can these models be combined?

The use of retrieval and generative models is not mutually exclusive. Combining them lets each compensate for the other's weaknesses, and that combination is exactly what retrieval-augmented generation (RAG) is built on.

Rather than forcing a choice between the two, RAG uses both. It leverages the generative model's ability to produce natural, coherent responses, while incorporating a retrieval mechanism that pulls relevant, up-to-date information from a knowledge base before generation happens. This means the model isn't relying solely on what it learned during training. It's grounding its response in retrieved facts. The result is output that is more accurate and contextually relevant than either model could produce on its own.