A data strategy guide for advanced GenAI applications on AWS
Imagine hiring the world’s most brilliant consultant: an expert with decades of experience, but then providing them with no files, company history, or context about your business. You wouldn’t expect them to solve your most complex problems. They understand general knowledge but don’t know your products or customers.
Deploying generative AI without a data strategy is the same idea. The model can generate answers, but without access to your proprietary data, its responses are generic and often inaccurate. The AI cannot access company-specific knowledge, manuals, or documentation. Additionally, it misses out on short-term interactions, like ongoing conversations or session data, which are not stored.
The model’s insights lack business relevance as they can not use your data.
The real benefit of AI comes from the organization’s unique and proprietary data gives the model the context to generate useful responses. A well-architected data strategy is necessary for insightful and accurate AI outcomes.
This newsletter explores the architectural patterns and AWS services that form the pillars of a modern data strategy. It explains how to build a foundation that transforms generic models into context-aware applications that precisely understand your business.
Providing data to GenAI applications with in-context learning#
A large language model (LLM) needs a carefully constructed and engineered prompt built from multiple layers of context to generate a truly accurate and relevant response, besides the user’s question. This process of providing rich, just-in-time information to the model is known as in-context learning.
An engineered prompt consists of three types of context:
Behavioral context (prompt template): This tells the model how to behave. It includes instructions on its persona, such as “you are a helpful customer support agent.” It also includes the desired output format, such as “Answer in three sentences or less.”
Situational context: This provides the model with short-term memory and user awareness. It includes the user’s conversation history, details about their account, and other real-time information that shapes the immediate interaction.
Semantic context (knowledge base): This is the model’s deep, long-term memory. It’s the vast repository of your organization’s specific knowledge, including technical documents, product manuals, legal policies, or past support tickets, that the model can draw upon to answer complex questions. This knowledge is typically stored and searched for in a vector database.
So, how do we architect a system to deliver this rich context?
