Search⌘ K
AI Features
Home
Courses
AI for Backend Engineers

An LLM Is an API Call, Not a Model You Train

Explore how large language models operate as API calls rather than trainable models and understand key concepts like tokens, context windows, temperature, and cost. This lesson helps you grasp the request-response pattern, generation controls, and the importance of treating LLMs as external dependencies when integrating them in backend systems.

We'll cover the following...

Previously, we established a clear boundary: use the model for language-oriented tasks and keep deterministic application rules in code.

Before we can apply that boundary effectively, we need to understand what happens when an application sends a request to an LLM. Some of the terminology around LLMs can be unfamiliar to backend engineers, including training, weights, parameters, and fine-tuning.

As backend engineers integrating LLM APIs, we only need enough understanding of these concepts to reason about model behavior and API integration.

We will not train or fine-tune models in this course. Instead, we will send requests to a pretrained model hosted by a provider through an API, much like other hosted services. This lesson introduces the core concepts needed to work with LLM APIs: tokens, context windows, temperature, and token-based API costs.

The shape of the interaction: Request in, text out

Regardless of which provider we use, the course remains provider-agnostic. At a basic level, most LLM API interactions follow the same request-response pattern. In the simplest case, we send a prompt to an API endpoint and receive a model-generated response. A standard request-response call does not require a persistent connection, and the model does not automatically retain context from previous independent requests. Any cross-request state must either be managed by our application or provided explicitly through a provider-supported stateful API.

We can represent that shape without committing to any specific provider's SDK, as the mechanics of an actual client call belong to the next chapter:

Python
request = {
    "model": "whichever model you've chosen — GPT, Claude, Gemini, etc.",
    "input_text": "Summarize this support ticket in one sentence: ...",
    # Controls how the model generates. Which parameter goes here depends
    # on the model: older models use "temperature", newer reasoning models
    # use settings like reasoning effort and verbosity instead.
    "output_control": "see the section below",
    "max_output_tokens": 150,
}
# response_text is whatever the model generated for this request.
# It is not guaranteed to be identical across two calls with the same input.
response_text: str = "..."

  • Lines 1–9: We deliberately use a plain dictionary instead of a real SDK call. Providers structure requests differently, but most model calls contain the same core concepts: the selected model, the input, any supported generation controls, and a limit on the output length.

    • Line 2: We use a description instead of a real model identifier because the principles in this lesson apply regardless of which provider or model we eventually choose.

    • Line 7: This field is ...