Anatomy of a Request
Explore how to construct and manage AI model requests in Python backend services by using role-based message structures. Understand the importance of system prompts for defining service rules, managing multi-turn conversations through message history, and handling the stateless nature of AI models. This lesson equips you to build well-structured requests that improve AI integration reliability and maintain clear separation between fixed instructions and user input.
The previous chapter gave us the mental model: we send input to a model, receive a generated response, and the same input does not guarantee identical output across calls. Now we look at the structure of the request we send to the model. In many chat-oriented LLM APIs, the request is structured as a sequence of messages rather than a single string, with each message assigned a role such as system, user, or assistant. Understanding this message structure is important when designing backend services that construct, validate, and manage model requests.
Messages and roles
A request to an LLM is a list of messages, and each message carries a role that tells the model who’s speaking. Three roles show up across virtually every provider, though the exact keyword and mechanics can differ slightly, which we’ll address directly in a moment:
system: Instructions that shape how the model should behave for the entire conversation: its persona, its constraints, its output format, the rules of the service it's operating inside.user: The actual input from whoever (or whatever) is asking something: a customer's message, a ticket's text, a question.assistant: The model's own prior responses, included when we want the model to see what it already said earlier in a multi-turn exchange.
That third role is easy to skip over, but it's the answer to a question previously left open: if each call is stateless and the model doesn't remember anything on its own, how does a multi-turn conversation work at all? The answer is that it doesn't, really; what actually happens is that our own code resends the entire conversation so far, including the model's own earlier replies tagged as assistant, every single time. The "memory" is an illusion we construct by resending ...