Master Agentic Design Patterns/

...

/

Retrieval Capabilities in Agentic Systems

Retrieval Capabilities in Agentic Systems

Learn to integrate retrieval tools so LLMs can access and synthesize private knowledge on demand.

We'll cover the following...

We’ve seen how LLMs can use tools, respond with structure, and even remember previous messages by maintaining a conversation list—all key parts of building an augmented LLM system. But we haven’t yet tackled one of the most powerful real-world use cases: retrieving knowledge.

Press + to interact

Here’s the setup: you want your model to answer questions not from the public internet, but from your company’s own data—say, an internal policy, an onboarding document, or a niche e-commerce FAQ. These models don’t know that information out of the box, but with the right tools, we can feed it in dynamically at runtime, just like you’d hand a teammate a file mid-conversation and expect them to use it in their response.

Let’s explore how to retrieve local data and hand it off to the model as needed, without retraining, fine-tuning, or even hosting a custom database. We’ll use a simple but powerful approach: define a tool that “searches” our knowledge base (even if it’s just a TXT or JSON file), let the model decide when to call it, and then combine that result with a structured, trustworthy answer. Think of it as giving your LLM an open-book test, and it gets to flip through your content before answering.

When the model doesn’t know what you know

Let’s try something simple. Suppose we ask the model:

“What other courses related to AI Agents is Educative releasing in 2025?”

This seems like a fair question. But here’s the catch: our model doesn’t know the details of Educative’s course catalog and our current plans; it wasn’t trained on our internal dashboards or course listings. So it should reply with something like:

“I'm sorry, I don't have access to Educative's internal content or course offerings.”

Public models don’t know private data, unless you explicitly feed it to them. And that’s what this lesson is about: if we want our AI to speak confidently about internal facts, we have to provide internal context.

Let’s run this prompt and see how the model responds when it has zero inside knowledge:

Press + to interact
Python 3.10.4
from openai import OpenAI
client = OpenAI(api_key=("{{OPENAI_API_KEY}}"))
completion = client.chat.completions.create(
    model="gpt-4.1",
    messages=[
        {"role": "system", "content": "You're a helpful assistant."},
        {
            "role": "user",
            "content": "What other courses related to AI Agents is Educative releasing in 2025?",
        },
    ],
)
response = completion.choices[0].message.content
print(response)

Great, that’s exactly what we wanted. Our course catalog and upcoming launch plans are still confidential, which means the model can’t peek behind-the-scenes. (Although, it's true—we do have some exciting AI agent content in the works.)

But this little demo highlights a massive limitation: out of the box models have no access to your company’s data. For tasks like onboarding support or product-specific chatbots, that’s a dealbreaker. No matter how “smart” the model seems, it’s still operating in a vacuum.

How to create a simple external knowledge base?

Let’s fix that. In the next step, we’ll simulate what it looks like to retrieve that internal knowledge, not from the model itself, but from an external source like a file, or a database. This is where retrieval enters the picture: our LLM doesn’t need to know everything, it just needs to ask ...