Search⌘ K
AI Features
Home
Courses
AI for Backend Engineers

What Actually Changes for a Backend Engineer

Explore how backend engineers should approach integrating large language models as external services with probabilistic outputs. Learn to distinguish language-based tasks suited for AI from critical business logic that must remain deterministic, ensuring reliable and secure backend operations when adding AI features.

We'll cover the following...

Backend engineers are typically familiar with adding external dependencies to a system. We integrate dependencies such as payment gateways, search indexes, and message queues, then reason about their contracts: inputs, outputs, and failure modes. Adding an LLM can feel unfamiliar because LLMs are often presented as fundamentally different from conventional dependencies. From an integration perspective, many of the same principles still apply. However, LLMs introduce enough new behavior that we need to separate familiar integration concerns from LLM-specific ones before we start implementing the integration.

An LLM is a dependency, not a coworker

It's tempting to think of an LLM as a junior developer we're delegating work to, or as a new kind of intelligence joining the team. That framing causes real engineering mistakes, because it invites us to trust the LLM's output the way we'd trust a human teammate's judgment, or to assume it understands our business rules the way a new hire eventually would.

We get better outcomes by treating the LLM the way we treat any other external service: something we call over an API, with a defined input, a probabilistic output, a cost per call, and failure modes we have to handle. The mental model that will serve us for the rest of this course is simple:

We send text in. We get text back. We decide, in our own code, what that text is allowed to do.

That last part is the one new hires to AI integration usually skip, and it's the one this course spends the most time on.

A single horizontal flow diagram with three boxes connected by arrows
A single horizontal flow diagram with three boxes connected by arrows

Tasks the model is genuinely good at

Language models are strong at tasks that involve unstructured, ambiguous, natural language input where the "correct" answer depends on interpretation rather than a fixed rule. In a backend context, that maps to a small set of recurring task shapes:

  • Classify: Assigning a label to a piece of text, such as sorting an incoming support ticket into "billing," "technical," or "account access."

  • Extract: Pulling structured fields out of unstructured text, such as pulling a customer's order number and complaint type out of a free-form email.

  • Summarize: Condensing a long piece of text into a shorter one that preserves the important content, such as turning a twenty-message support thread into a two-sentence handoff note.

  • Rewrite: Transforming text from one style, tone, or audience to another, such as turning an internal engineering note into a customer-facing update.

  • Draft: Producing a first version of a response or document for a human to review, such as a suggested reply to a support ticket.

Notice what all five have in common: the input is language, the task requires judgment about meaning, and the output is still just text that something downstream will look at. None of these tasks require the model to make a final decision. They require it to convert unstructured meaning into a form our application can work with.

Rather than take that faith claim, let's test it directly:

AI Powered
Saved
10 Attempts Remaining
Reset
Statement

Below is a real support ticket. Write a prompt that asks the model to classify it into exactly one of three categories:

billing, technical, or account_access. Try a vague prompt first, then a specific one, and compare the results. Ticket:

"Hey, I tried logging in this morning and it told me my card was declined, but I wasn't even trying to pay for anything, I just wanted to check my invoice history. Now I can't get into my account at all."

Notice what happens with a one-line prompt like "classify this ticket" vs. a prompt that spells out the three allowed categories and asks for just the label. The gap between those two outputs is the entire argument for treating prompts as part of your system's design, which we'll formalize in the upcoming lessons.

Here’s a simple code example that shows this boundary. We aren't calling a real provider yet, so we'll represent the LLM request with a placeholder function. The focus here is the separation between conventional application logic and the LLM integration. For that reason, the example remains provider-agnostic: whether the implementation eventually uses a GPT, Claude, or Gemini model, the architectural boundary remains the same.

Python
def classify_ticket(ticket_text: str) -> str:
    """
    Stands in for a call to an LLM provider (OpenAI, Anthropic, Gemini, or
    any other). The real request/response mechanics are covered in upcoming lessons.
    For this lesson, treat it as a black box: unstructured text goes in,
    a short text label comes back.
    """
    ...
def route_ticket(ticket_id: str, category: str) -> None:
    """Deterministic business logic. No model involved."""
    if category == "billing":
        assign_to_team(ticket_id, team="billing")
    elif category == "technical":
        assign_to_team(ticket_id, team="support-eng")
    elif category == "account_access":
        assign_to_team(ticket_id, team="security")
    else:
        assign_to_team(ticket_id, team="general")

  • Line 1: We define classify_ticket with one narrow responsibility: converting raw ticket text into a category label. It does not make any routing or business decisions.

  • Lines 2–8: The docstring explains that the function represents a provider call whose implementation will be covered later. The design remains provider-agnostic.

  • Lines 11–12: We define route_ticket as a separate function that receives only a ticket ID and category. It does not see the original ticket text or call a model.

  • Line 13: The docstring states that this function contains deterministic routing logic.

  • Lines 14–20: We use ordinary if/elif/else logic to select a team. Given the same category, this function produces the same routing decision and can be unit-tested without calling or mocking a model.

Tasks the model should not own

The opposite boundary is equally important. An LLM should not make the final decision in workflows that require guaranteed, repeatable, and auditable outcomes. Examples include:

  • Authorization and access control: Deciding whether a user is allowed to perform an action. This must be enforced by application logic, never inferred from a model's response.

  • Precise calculation: Computing totals, balances, discounts, or anything involving exact arithmetic that a bug tracker will hold us accountable for.

  • Anything requiring guaranteed consistency: Implementing logic deterministically rather than delegating it to an LLM, if the same inputs and system state must always produce the same output for the system to behave correctly.

  • Final business decisions with real consequences: Refunding a customer, deleting a record, sending an irreversible action. The model can propose an action; the backend must validate and execute it.

We'll come back to that last point in more depth later in the course, once we've built the pipeline to actually make model calls and handle their output. For now, the important habit to build is asking the question before writing any AI-related code: is this a language-understanding problem or a business-rule problem? If it's the former, the model can help. If it's the latter, keep it in ordinary code, no matter how convenient it might seem to just describe the rule to the model in a prompt.

Model output crosses into code, nothing on the right is ever the model's call
Model output crosses into code, nothing on the right is ever the model's call

In this lesson, we drew the line that the rest of the course will keep coming back to: classification, extraction, summarization, rewriting, and drafting are language problems, and the model can genuinely help with them, as we just saw firsthand in the prompt exercise. Authorization, precise calculation, guaranteed consistency, and irreversible actions are business-rule problems, and they stay in the code we write and control. Getting this boundary right before writing a prompt is what keeps an AI feature from quietly becoming an authorization system nobody meant to build.

Next, in “An LLM Is an API Call, Not a Model You Train,” we’ll explore what happens on the other side of that boundary, including tokens, context windows, temperature, and cost. This will give us the vocabulary needed to reason precisely about the dependency we’ve just defined.