Decomposing the Problem: The RAG System
Explore how to decompose and design a Retrieval-Augmented Generation (RAG) system tailored for a mid-size investment bank. Understand user workflows, document ingestion challenges, access control enforcement, and how to combine self-hosted AI models to enable secure, accurate natural language querying of a heterogeneous, permission-bound document corpus.
This lesson works through a case study from a mid-size investment bank. The case study names a business pain, specifies some constraints, and asks for a solution without defining what that solution looks like. It represents a common class of AI system problem in enterprise settings: a large, heterogeneous document corpus, meaningful access control requirements, and a desire to make it queryable through natural language.
The case study
Given below is the complete case study of a mid-size investment bank. Read it once before we begin decomposing it.
The case study
The organization: A mid-size investment bank with around 4,000 employees across New York, London, and Hong Kong.
The problem: Analysts, compliance officers, and senior advisors lose significant time every week searching for information across the bank’s internal document corpus. A typical equity research analyst spends two to three hours before writing a note locating relevant prior research, comparable company filings, and internal memos on the sector. Compliance officers manually scan through policy documents and regulatory guidance to answer questions from the business. Senior advisors assembling a pitch book spend half a day locating relevant deal precedents and internal models from prior transactions.
The documents: Documents live in three places: SharePoint for internal research notes, compliance policies, and deal memos organized by client; an email archive for Bloomberg Message and Outlook threads going back eight years; and a shared file server for SEC filing PDFs, earnings transcripts, and older documents that predate SharePoint.
The requirements: All documents and all processing must stay within the bank’s private cloud. No document content, chunk, or query can be sent to any external service. The system must also respect the document permissions that already exist in SharePoint. Different teams and seniority levels have access to different document libraries. If an employee cannot access a document in SharePoint today, they should not be able to receive an answer derived from that document through the AI system. The same access control principle extends to the email archive and file server.
The ask: Employees should be able to ask questions in plain language and get accurate answers from internal documents.
With the ...