Context Windows, Tokens, RAG, and Fine-Tuning
Explore how generative AI uses context windows and tokens to process text within limits. Understand retrieval-augmented generation (RAG) for accessing updated business knowledge and fine-tuning for consistent response behavior. Learn to select appropriate AI adaptation methods to balance accuracy cost and traceability in business applications.
We'll cover the following...
A complaint assistant cannot reliably answer from a 300-page refund manual pasted into one request. It may also repeat an outdated policy even when the prompt is carefully written. Tokens are chunks of text that a GenAI model processes, and they contribute to usage cost. A context window is the maximum number of tokens the model can consider at one time, including the input and the generated response.
When the combined prompt, policy material, and expected response exceed that limit, the model cannot consider all of them together. Symptoms include truncated input, missing earlier details, incomplete answers, and inconsistent use of policy clauses. Even before a ...