Understanding MCP Architecture
Explore the MCP architecture by understanding the distinct roles of host, client, and server in agent systems. Learn how MCP's modular, composable, and programmable design allows AI agents to scale, integrate diverse tools, and handle complex workflows efficiently and securely.
Imagine you’re assembling a team of robotic assistants to manage a high-tech office. Each robot needs a workspace, a reasoning engine, and a toolkit for interacting with its environment. Without a standard, every new robot would require hardwiring and custom instructions for each task, such as opening a filing cabinet, sending an email, or logging in to the HR system. This approach would quickly become unmanageable: updates, fixes, and scaling would grow increasingly difficult as the system expanded.
MCP addresses this challenge by replacing brittle, hardcoded setups with a modular, plug-and-play MCP architecture for AI agents. In this lesson, we’ll explore the architecture of MCP-powered agentic systems in practical terms: the role each component plays, how they interoperate, and how this unlocks new possibilities for building scalable, resilient AI applications.
The architecture of MCP
In MCP architecture, each software component (an AI agent, plugin, or backend service) takes on one of three main roles: host, client, or server. These roles specify how components communicate and their responsibilities in the system.
Host
The host is the central orchestrator, the root environment that manages agent lifecycles, user sessions, and connections to tools and data. It might be a desktop app (e.g., Claude Desktop), a web portal, or an IDE such as Visual Studio Code.
Responsibilities
Session management: Maintains state across user interactions (conversation history, preferences, authentication).
Security context: Handles authentication, authorization, and user-specific access tokens. Ensures agents only act within allowed boundaries.
Security note: For sensitive deployments, hosts should also enforce sandboxing for servers, use secure channels (HTTPS/TLS), manage credentials centrally, and log all access for auditing. These are foundational security practices for any production deployment.
Connection orchestration: This involves starting and stopping agent processes, connecting/disconnecting MCP clients and servers as needed, and managing resource cleanup.
Interface bridge: Translates user inputs into agent-understandable requests, and agent outputs back into UI messages.
Capability registry: This keeps track of which MCP servers (tools/resources) are currently available and which agents are allowed to use them.
Hot-swapping and discovery: Dynamically loads or unloads new tools/servers at runtime, enabling true plug-and-play extensibility.
Multi-agent orchestration
In many real-world deployments, a single host may orchestrate multiple concurrent agents within the same application or workspace, each with its own client and specialized capabilities. For example, in collaborative AI workspaces (like ChatDev), different agents may handle distinct tasks or represent different personas, all coordinated by the host. MCP’s one-client-per-server design fits naturally into this kind of multi-agent architecture, making it easier to scale from a single assistant to a team of cooperating agents, each managing its own workflows and tool access.
Architecturally, the host sits at the intersection of user experience and agentic intelligence, ensuring seamless and secure orchestration of all system parts.
Client
The client is a lightweight runtime component embedded in the host. It acts as the agent’s gateway to the outside world, maintaining a one-to-one connection with a specific server. It transmits requests over MCP and returns results to the agent.
Note: Strictly speaking, the MCP specification treats the client as a connector: the main agent logic (the LLM and planner) typically resides in the host, and each client handles protocol communication with one server. In practice, though, the application code that owns these connections (such as the LangGraph agent you’ll build later in this course) acts as the brain of the system. The responsibilities below describe this broader client application, which combines the agent’s reasoning with its MCP connections.
Responsibilities
Task decomposition: Receives user goals (from the host) and breaks them down into subtasks.
Reasoning and planning: Decides which MCP-exposed tools/resources to use for each subtask.
Protocol handling: Issues structured requests (JSON-RPC 2.0) to servers, handles responses/errors, and keeps track of request/response IDs.
Session context: Maintains memory (short-term and long-term), manages chain-of-thought, and tracks intermediate results.
Security enforcement: Calls tools and accesses data as allowed by the host’s policies and current user/session context.
Error handling: Detects and responds to errors by adapting the workflow or prompting the user for alternatives.
Orchestration and composition: Dynamically combines multiple tool/resource calls in multi-step workflows, sequencing actions as needed to achieve user goals.
Transparency and observability: Logs tool usage, decisions, and errors for observability, auditing, and debugging.
In this broader sense, the client application is both the brain (deciding “what” to do) and the conductor (deciding “when” and “in what order” to do it).
Server
The server is a modular component that exposes specific capabilities via MCP. It might interface with a database, filesystem, third-party API, or internal tool. Servers are self-contained and designed for interoperability; depending on their purpose, they can be stateless or maintain session state.
In MCP, each server exposes certain capabilities, called primitives. These fall into three categories: resources, tools, and prompts.
MCP resources: Read-only data sources that provide context or information. This could be a database record, a file, or a document. For instance, a research assistant agent might have a resource server that can fetch scientific articles by DOI, or a vector store that can retrieve relevant text passages given a query. Resources broaden the agent’s knowledge beyond its trained model by plugging in external data on the fly.
MCP tools: The AI can invoke active functions to perform actions or computations. For example, a “Calculator” tool for math, a “WebSearch” tool to perform a web query, or an “EmailSender” tool to send an email. Tools usually have inputs (parameters) and produce outputs (results). For example, an
add(a, b)tool takes two numbers and returns their sum:
MCP prompts: Predefined templates or workflow scripts that guide the AI in particular tasks. MCP prompts are a first-class feature, allowing agents to load structured instructions from a server rather than hardcoding them into the application. Think of these as structured prompts or mini-scenarios that the agent can call up. For example, a “SummarizeDocument” prompt template might be a sequence that instructs the model how to summarize a given text (perhaps using a particular format or step-by-step approach). With prompts as a built-in component, an agent can load a domain-specific prompting strategy from a server when needed. For example:
@mcp.prompt()def commit_message(issue_id: str, change_summary: str) -> str:return f"feat: {change_summary}\n\nResolves #{issue_id}"
Note: Prompts can be dynamic (generated on the fly or retrieved based on context) rather than purely predefined templates.
Because all these capabilities are exposed in a standardized way, the AI agent can compose them. For instance, consider a complex user request: “Analyze this customer feedback and create a report with graphs.” A composable agent might use a resource to fetch all relevant feedback data from a database, then use a tool (like a Python execution tool) to perform sentiment analysis or generate statistics, then use another tool to plot graphs (maybe via a charting API), and finally use a prompt/template to format the findings into a coherent report. All these steps can be orchestrated seamlessly because the agent can invoke each module as needed and then integrate the results. This is a classic example of an MCP workflow in action.
Does an MCP-enabled AI application encompass all three roles (host, client, and server)?
MCP workflow: interacting host, client, and server
Understanding the interplay between host, client, and server is crucial for architecting robust, modular agentic systems with MCP. A sequence of lifecycle events, dynamic discovery, secure communication, and structured error handling defines this interaction.
To see how these pieces fit together, let’s review a typical request, step by step.
A user asks:
“Summarize the latest Q2 sales report and schedule a meeting with the team.”
The process begins when the user’s request enters the host application, which manages the overall session and user interface. The host then passes this input to the agent, which analyzes the instruction. The agent determines that it needs access to the sales report (a file) and the ability to create a calendar event.
At this stage, the host ensures the agent is connected to all relevant servers, each through its own MCP client.
Discovery and registration
The client queries all connected servers during initialization to discover their available tools, resources, and prompts. Each server advertises its capabilities using a standardized schema, enabling the agent to dynamically assemble workflows. The architecture supports real-time extensibility: adding a new server (for example, a bug tracker) makes its features available to the agent without changes to the agent’s code. This is one of MCP’s biggest advantages: capabilities are discoverable and hot-swappable by design.
Invocation
When the agent decides to retrieve the sales report, it sends a structured JSON-RPC resources/read request to the file server:
The file server fetches and returns the document contents to the client.
Next, to schedule the meeting, the client sends a tools/call request for the calendar server’s schedule_event tool:
The calendar server processes this information, creates the event, and sends confirmation or error messages back.
Response
Each server returns a structured response. It can either be the requested data, the result of an action, or a clear error message. The client gathers these results, synthesizes a coherent response, and relays it through the host to update the user interface.
MCP elicitation: Beyond serving data and executing tools, MCP also supports a client feature called MCP elicitation, which is a standardized way for servers to request additional information from the user mid-interaction. For example, if a server needs a missing user preference or a confirmation to complete a task, it doesn’t fail silently or make assumptions. Instead, the server sends an elicitation/create request through the client, which presents the user with a prompt or form to fill in. MCP elicitation supports two modes:
Form mode for collecting structured, non-sensitive data.
URL mode for sensitive interactions like OAuth flows or payment processing, where data must never pass through the MCP client.
This keeps the agent workflow dynamic and interactive while maintaining clear security boundaries.
Protocols and dynamic extensibility
MCP uses JSON-RPC 2.0 as its messaging protocol, enabling consistent method calls, parameters, and matched responses. It supports both local (standard input/output) and remote (Streamable HTTP, which uses Server-Sent Events for streaming) transports. Remote servers are secured with HTTPS and, where needed, OAuth-based authorization; local servers rely on OS-level permissions. A key advantage of this design is hot-swapping: servers can be added or removed at runtime, and clients can discover and adapt to these new capabilities, since servers can notify clients when their list of tools changes.
Pillars of the MCP architecture
Having laid out what MCP is and how it’s structured, let’s explore the three defining characteristics of MCP architecture (modular, composable, and programmable) and what they mean for building robust, intelligent agents.
Modular: MCP utilizes a modular architecture, breaking down AI agents into independent modules like LLMs, tool servers, and memory stores. Each module handles a specific responsibility, can be developed and updated separately, and seamlessly integrates with other modules as long as it adheres to MCP. This modularity supports fast iteration, scaling, and adding new components without affecting the existing system.
Composable: MCP stresses combining independent modules into flexible workflows. Like LEGO blocks, these modules with standard interfaces can be connected to solve new problems without extensive code changes. Agents dynamically chain modules (like database readers, analyzers, and formatters) for tasks. This context-driven mixing and matching allows components like a Gmail server to be reused across agents and contexts without constant custom integrations.
Programmable: The third pillar of MCP is programmability. Agent behavior is not opaque but consists of inspectable, programmable steps. Developers or agents can script workflows using data queries, tool execution, result formatting, and error handling. Agents use tools in a controlled, transparent manner, with each call visible as an explicit step. Programmable agents offer reliability, transparency, and easier debugging, allowing step-by-step observation and refinement. In the future, agents may dynamically enhance their capabilities by creating and integrating new tool modules.
MCP in action
Let’s see how all these roles work together through a real-world workflow. Imagine we have an AI assistant named Ava deployed in an office environment. A user asks Ava: “Please schedule a team meeting for next week and send out invites.” This high-level request requires multiple steps and showcases MCP’s modular, composable, and programmable design.
Understanding the request: Ava’s LLM parses the instruction and identifies subtasks: checking calendars for availability, finding a meeting slot, booking a room, and sending invitations.
Here, each of these capabilities is delivered by a distinct, modular MCP server (calendar, room booking, and email), making the system flexible and maintainable.Querying the calendar (modular interaction): Ava uses the MCP client to send a
get_availabilityrequest to the “Calendar” server.
Because the calendar is a standalone, modular service, it can be swapped, updated, or scaled independently.Finding a meeting room (composable workflow): Ava then invokes the “Room Booking” server with a
find_room(time_slot)request.
This step demonstrates how Ava can compose different modules, dynamically chaining calendar and room booking together to fulfill the larger goal.Sending email invites (further composability): Ava calls the “Email” server’s
send_invite(details)action, providing meeting info. Again, the agent uses composable modules by connecting outputs from one server as inputs to the next.Error handling and adaptation (programmable logic): If no room is available, Ava’s workflow, being programmable, can automatically try the next time slot or prompt the user for alternatives.
Confirmation and response: After orchestrating these modular services in a composable, multi-step plan, Ava returns a confirmation:
“Your team meeting has been scheduled for next Wednesday at 10 a.m. in conference room A. Invitations have been sent to all participants.”
MCP enables Ava to act as a true agent by handling complex, multi-system tasks with clarity and flexibility through its modular components, composable workflows, and programmable logic.
Why is this powerful?
No hardcoded integrations: Ava didn’t need to know how each calendar API or email system works.
Plug-and-play: Any company could swap out the Calendar or Email server for a new provider, and Ava’s composable logic would keep working, as long as the new server exposes equivalent tools.
Error recovery: Ava’s agent logic can automatically adjust, retry, or escalate if a tool fails.
Security: The host decides which servers and capabilities each agent can reach, and each server enforces its own permissions. MCP provides the structure for this, but the enforcement itself is implementation work.
Case study: dynamic scaling with MCP
Sam is a backend engineer at a growing SaaS company. His team’s product relies on a primary PostgreSQL database for customer analytics. As the product evolves, Sam needs to integrate a new, completely different data source: a high-speed Redis cache to store temporary session data.
The challenge (before MCP):
Previously, integrating a new type of data source was a major project. Sam would have to write a custom client adapter for Redis, teach the agent the specific commands and data structures for Redis (which are different from SQL), and then update and redeploy every agent that needed to access it. The agent’s core logic would become cluttered with if-database-is-postgres-do-this and if-database-is-redis-do-that branching. This approach was brittle, slow, and tightly coupled the agent’s logic to the specific tools it used.
The solution (with MCP):
Now, Sam’s team uses MCP. To integrate the new cache, Sam deploys a standard “Redis Server” that exposes capabilities like get(key) and set(key, value) via MCP. He registers this new server with the MCP host.
Once the host connects to the new server, MCP-powered agents can discover these capabilities and use them without changes to their core logic. The model sees tools like get and set described in a standard format and decides when to use them for tasks such as “store temporary data” or “retrieve a key.” The agent doesn’t need to know it’s talking to Redis; it just knows it’s using a standardized capability.
Question
Explain how MCP’s architecture allows Sam to integrate the new Redis cache. Contrast this with the old approach and highlight why MCP’s method is superior for adding new types of tools, not just for scaling existing ones.