Enterprise RAG: Problem Framing & Requirements
Explore how to frame the problem and set precise requirements for enterprise Retrieval-Augmented Generation systems. Understand key challenges like answer accuracy, strict access control, and citation traceability that differentiate enterprise RAG from consumer AI. Learn to define critical business metrics such as answer accuracy, citation quality, user adoption, and latency, and discover architectural trade-offs between pre- and post-retrieval filtering approaches to access control. This lesson prepares you to design compliant, secure, and efficient RAG systems tailored to interview scenarios and real-world enterprise needs.
In the previous case study, we designed a fraud detection system built around adversarial classification, catching bad actors in real time. Now the design challenge shifts dramatically. Instead of predicting a binary label, the system must generate a natural language answer, ground it in retrieved evidence, and enforce document-level permissions for every single user. Welcome to enterprise RAG.
A Retrieval-Augmented Generation knowledge assistant answers employee questions by pulling relevant passages from internal documents and feeding them to a large language model (LLM) that synthesizes a cited response. Products like Microsoft Copilot, Google Workspace AI, and Notion AI operate on this principle. This problem appears frequently in MAANG ML system design interviews because it simultaneously tests retrieval pipeline design, data privacy reasoning, and end-to-end metric definition, three skills that rarely overlap in other case studies.
The core design question is deceptively simple. How do you build a system that generates accurate, cited answers from potentially millions of internal documents while ensuring no user ever sees information they are not authorized to access? Unlike consumer search, enterprise RAG operates under strict compliance constraints such as SOC 2, GDPR, and HIPAA. These constraints elevate access control from a nice-to-have feature to a core architectural requirement that shapes every component in the pipeline.
This lesson covers problem framing and requirements. The next lesson builds the data strategy and retrieval pipeline on top of these foundations.
Note: Framing the problem correctly in the first five minutes of an interview often matters more than the technical depth you reach later. Interviewers evaluate whether you identify the right constraints before you start drawing boxes.
Enterprise Q&A requirements
Enterprise RAG systems must satisfy three non-negotiable requirements that distinguish them from consumer-facing generative AI products. Getting any one of these wrong can result in compliance violations, eroded employee trust, or outright system rejection.
Answer accuracy: The system must return factually correct responses grounded in retrieved documents, not hallucinated content. Hallucination risk is amplified in enterprise settings because employees may act on incorrect answers in financial, ...