Design a Multi-Agent Medical Diagnosis System

In this lesson, you will apply everything you have learned about agentic system design to solve a complex, real-world challenge. You will be guided through a thought exercise to architect a multi-agent “AI Hospital,” a simulated clinical environment for evaluating and improving the diagnostic capabilities of LLM-based doctor agents. This exercise will test your ability to design agent roles, orchestrate collaboration, and build mechanisms for long-term agent improvement.

The challenge of medical diagnosis with AI

Medical diagnosis is one of the most complex and high-stakes tasks in human decision-making. Diagnosing a patient involves dynamic, multi-step reasoning, unlike answering a trivia question or recommending a movie. A doctor must gather a patient’s history, interpret symptoms, order and analyze tests, and then integrate all that information into a coherent diagnosis and treatment plan. Every decision carries weight, and mistakes can have serious health consequences.

Today, most evaluations of large language models (LLMs) in medicine rely on static benchmarks, such as multiple-choice exam questions. While useful, these benchmarks fall short. They measure knowledge recall, but not the interactive process of real-world diagnosis. In practice, medicine is not about choosing the best answer from four options; it is about asking the right questions, handling uncertainty, collaborating with colleagues, and learning from experience.

This gap presents an opportunity: What if we could create a simulated clinical environment, an “AI Hospital,” where LLM-based doctor agents are tested not just on recall but also on their ability to act like doctors? Such a system would ...