Introduction to Agents

The developer can see every inference step, tool call, and shield result. So far, we’ve been using the Inference API to send prompts and receive responses. This is powerful, but it’s also limited since every interaction is a single query. There's no persistence, no memory, no external actions. We are relying on pure language modeling to solve all problems.

But real-world AI applications often need more. They need to reason across multiple steps, retrieve relevant information, invoke tools, and guard against harmful output while maintaining a conversational state.

That’s where agents come in.

Llama Stack agents are self-contained systems that bring together all of these capabilities under a single abstraction. They’re the foundation for building intelligent assistants, domain-specific copilots, document-based Q&A systems, and more.

Why use agents?

Agents allow us to move beyond basic prompt engineering and into composable workflows. Instead of writing a long system prompt that tries to teach the model how to use tools or recall memory, we configure these behaviors explicitly.

Here are a few problems that agents solve:

• You want your assistant to look things up in a document before answering.

• You need to enforce input/output moderation before generating a reply.

• You want to let the assistant call APIs or execute code.

• You need to maintain a session across multiple user turns.

• You want to observe and debug the internal steps of reasoning.

The agent system in Llama Stack makes all of these possible, with a consistent interface and clear semantics.

What is an agent in Llama Stack?

An agent in Llama Stack is a structured orchestration loop that wraps around the base inference API and adds:

• Persistent session state

• Access to memory (e.g. via vector databases)

• Ability to use tools (such as search or code execution)

• Safety shields (for filtering input/output)

• Multi-step reasoning (with feedback loops between tools and inference)

You define an agent once and use it to process multiple user turns, just like a real conversation. The agent “remembers” the session, uses tools when appropriate, and applies safety checks before producing output. ...