From LLMs to AI Agents

Imagine a knowledgeable librarian who has read millions of books but is working in a library with no phone or internet and only a single-page notepad. You ask about yesterday’s stock prices or ask them to schedule a meeting on your calendar. Without outside connections, the librarian faces two problems:

• Their background knowledge does not include yesterday’s news or your current schedule.

• Their notepad can hold only a limited amount of information at once.

Limitations of LLMs

Language generation and interpretation are core capabilities of large language models (LLMs). When used without external tools or stored context, however, these models have several limitations:

• Limited knowledge beyond training: A model’s learned knowledge reflects its training data. It can use new facts supplied in a prompt, but it cannot independently retrieve recent events or private information without an external connection.

  • Practical approach: Developers provide relevant facts in prompts or update a model through additional training or fine-tuning.

  • What remains: Updating model weights can be costly and does not provide live access to changing information. Supplying context helps, but the application must obtain and select the relevant data.

• No built-in access to external systems: A standalone model does not independently browse the web, query a database, or modify an application. It can generate a request to use a tool, but the surrounding application must execute that request and return the result.

  • Practical approach: Developers connect the model to scripts, functions, or APIs that retrieve data or perform actions.

  • What remains: Separate adapters can duplicate work and become difficult to maintain as tools evolve. Clear interfaces and reusable integrations help manage this complexity.

• Context window limitations: A model can process only a bounded amount of input and generated output in a request. Information omitted from the current context is not directly available for that response; this is different from erasing stored conversation history.

  • Practical approach: Retrieval-augmented generation (RAG) retrieves relevant material from an external collection and includes it in the model’s context.

  • What remains: RAG can retrieve stored conversation history as well as documents, but the application must manage that storage. Retrieval alone does not provide an action-execution loop or guarantee that the right information is selected.

• Potential for hallucination: LLMs can generate plausible but unsupported or incorrect statements. This can happen even when relevant context is supplied; it is not limited to situations where a model is unsure.

  • Practical approach: Developers use human review, clearer prompts, and retrieved evidence to check or ground model outputs.

  • What remains: These measures can reduce errors but cannot eliminate them. Applications still need checks on sources, retrieved content, and generated answers, especially when an error would have significant consequences.

Because of these limitations, a standalone model cannot independently fetch live information or execute external actions. It can still analyze supplied documents, write code, classify inputs, and answer questions using information included in its context.

How do external tools and context empower LLMs?

Let’s examine how relevant context and external tools expand what an LLM-based application can do through a few practical examples:

The context-augmented assistant

Picture an AI assistant helping a doctor review a patient’s information. A standalone model can use symptoms or lab results supplied in its prompt, but it cannot independently retrieve the patient’s records. A connected application can fetch authorized records and provide them as context, much like giving the librarian a relevant file to read. This can make a summary more relevant, but does not guarantee clinical accuracy. The doctor must review the evidence and any suggested conclusions.

The toolbox for problem-solving

Consider a coding assistant used by a software developer. Without external tools, it can suggest code and explain concepts using its learned knowledge and supplied context. A connected application can let it run code in a sandbox, read documentation, search a repository, and run tests. The model can then use those results to revise its answer. Tests and documentation provide evidence about a solution, but do not guarantee that it is correct or secure. The application controls which commands and resources are available.

Dynamic decision-making

A productivity assistant can give advice based on the information in its prompt. With authorized access to a calendar or email service, it can retrieve current availability, draft invitations, or create reminders. The application must enforce permissions and any required approval before sending messages or changing calendar entries.

When an application gives an LLM relevant context and tools, it can combine model-generated decisions with external information and actions. The application executes tool requests and returns the results for the model to use:

• Fetching information: The application retrieves documents or current data and provides them to the model. Relevant evidence can improve answers, but retrieved information must still be checked.

• Taking actions: The application executes permitted tool requests, such as creating files, sending messages, or updating a workflow.

• Maintaining memory: The application stores important information and retrieves it into future context. Persistent memory belongs to the system surrounding the model, rather than arising automatically from tool access.

In practice, tools and context expand an application’s capabilities without necessarily changing the model’s learned parameters. The model can use new evidence and execution results, while the application supplies storage, access controls, and the machinery to perform actions.

From LLMs to agents

Developers can combine the approaches described earlier—retrieval, scripts, and APIs—to build agentic systems. The distinction is how the application coordinates decisions and actions, rather than whether it uses a particular integration technique.

• RAG retrieves information for the model. It can be part of an agent, but retrieval alone does not make a system agentic.

• Scripts and APIs execute actions. They can support both fixed workflows and agents, depending on who selects the next step.

• No individual component provides reliable autonomy on its own. An agentic application combines model-driven decisions with tool execution, feedback, and controls. A predefined workflow follows steps chosen by the developer; an agent can select or revise its next action based on the task and the results it observes.

What changes is the control loop: the system uses the model to choose actions, executes them through tools, and evaluates the results before continuing. The degree of autonomy depends on the application’s constraints and approval rules.

What makes a system agentic?

Agentic systems allow a model to choose or sequence actions based on a goal and the results it observes. Tools and dynamic context support this process, while the application enforces limits and executes requests. Common capabilities include:

• Retrieve information (for example, searching the web or querying a database)

• Invoke services or APIs (such as calling a weather service, sending an email, or running a calculation)

• Maintain longer context or memory (by storing information externally and retrieving relevant history or documents into the current context as needed)

Crucially, an agent can choose which tool to request and when, within the application’s rules. This ability to adapt the sequence of actions distinguishes an agent from a fixed workflow. Think back to the librarian: an agentic assistant might identify a gap, request the latest report, review the returned information, and decide whether another step is needed.

Early projects such as Auto-GPT and BabyAGI in 2023 explored model-driven task planning and repeated execution steps. They illustrated the possibilities and difficulties of autonomous applications; their designs and tool capabilities differed, and their early demonstrations did not establish dependable general-purpose autonomy.

Around the same time, consumer AI assistants also began exposing more external capabilities:

• ChatGPT began introducing plugins, including a browsing plugin, in 2023. This was an early example of connecting a conversational model to external information and services.

• Voice interfaces can also connect models to services and APIs. Adding an LLM does not by itself make an assistant agentic; the system must give it a role in choosing or adapting actions.

These developments expanded the range of tool-enabled assistants, from fixed workflows to systems with model-driven action selection.

Limitations of agents

Giving an agent tools and context does not solve every reliability problem. Integrations with databases, cloud services, and file systems still need clear contracts, validation, and access controls. Reusable adapters can reduce work, but each backend’s behavior must be understood.

This approach can create two recurring integration problems:

• Developers may duplicate integration logic when multiple applications need access to the same data sources or tools.

• The proliferation of separate adapters can introduce inconsistent schemas and error signals, causing misinterpretations or failures when applications do not validate tool results.

The integration burden creates a need for shared, well-defined interfaces between AI applications and external capabilities.

A common question is how MCP relates to agentic AI. Agentic AI describes systems that use model-driven decisions to select or sequence actions. MCP provides one way to connect an application to external capabilities. Agents can use MCP or other interfaces, and an MCP-connected application need not be autonomous.

Case study: the medical AI assistant

Dr. Lee uses an AI assistant to review patient information in this illustrative scenario.

• Initial version: The assistant uses only a standalone LLM. It can consider symptoms and lab results that Dr. Lee includes in the prompt, but cannot independently retrieve patient records or current guidelines. It may miss relevant information that was not supplied or generate unsupported conclusions.

• Agentic upgrade: The application gives the assistant authorized tools to retrieve patient records, recent test results, and current guidelines. It can select relevant retrieval steps, review the returned information, and prepare a summary with supporting sources for Dr. Lee to assess. With the required approval, it can also request a follow-up appointment through the clinic’s calendar service. Retrieved data and tool access do not guarantee a correct diagnosis, and clinical decisions remain subject to the doctor’s review.

Questions

• What new retrieval and action capabilities does the agentic upgrade provide, and which information could the standalone LLM already use from a prompt?

• What risks remain from incomplete data, incorrect reasoning, or brittle and insecure integrations? Which checks should the application and Dr. Lee perform?