Hands-On: Mapping an LLM Application Blueprint
Explore how to design and justify each layer of a large language model application tailored for internal document question answering. Understand requirements, component selection, orchestration logic, and trade-offs to create a defensible architecture that meets performance, compliance, and usability needs.
We'll cover the following...
Every production LLM application begins not with code, but with a blueprint. In the previous lesson, you traced a user query through the canonical eight-layer LLM application stack using a travel reimbursement scenario. You saw how each layer, from the UI to observability, plays a distinct role in transforming a natural language question into a grounded, trustworthy response. Now it is time to move from tracing someone else’s architecture to designing your own.
This exercise mirrors what happens in real-world architectural planning sessions. Engineers gather around a whiteboard, define requirements, select components layer by layer, justify trade-offs, and document their reasoning, all before writing a single line of code. You will do exactly that for a concrete use case: building an internal document Q&A system for a mid-size company’s HR policy knowledge base.
Here are the scenario constraints you will work within. The company maintains roughly 500 PDF policy documents covering topics from parental leave to expense reimbursement. Approximately 2,000 employees need self-service answers to policy questions. Responses must include citations pointing back to the source document. The system must deliver sub-3-second latency, enforce compliance guardrails, and keep all data within the company’s AWS account. There is no single correct architecture for this scenario. The goal is defensible, well-reasoned choices at every layer.
Defining the use case requirements
Before selecting any component, you need to establish the requirements that will drive every architectural decision. Skipping this step is one of the most common mistakes in LLM application design. Teams jump straight to picking a vector database or foundation model without understanding what constraints they are designing against.
The functional requirements for this system break down as follows:
Natural language policy Q&A: Employees ask questions in plain English about HR policies and receive accurate answers grounded in the actual policy documents.
Citation-backed responses: Every answer must reference the specific document and section it was derived from, so employees can verify the information. ...