The LLM Systems Round
Explore the LLM systems round to understand how Forward Deployed Engineer interviews at OpenAI probe problem-solving and decision-making. Learn to trace requests through authentication, eligibility checks, retrieval, drafting, and validation. Discover best practices for access control, testing, and handling scenarios like stale records or conflicting sources to confidently design and defend AI system components.
Twenty minutes into the technical session, the interviewer slid a report across: an offline retrieval test of the candidate’s Northstar design, called R-18. It covered 40 cases.
27 of 30 eligible cases produced supported drafts.
All 6 stale cases were withheld.
1 of 4 attempted unauthorized reads put a protected passage into the model’s input.
“So, are we ready for a pilot?”
The candidate said yes, with an output filter added to catch unauthorized passages in the final answer. The interviewer asked one more question: “Where was the passage when your filter ran?” It was already in the model’s input. The filter could hide it from the screen. It couldn’t take it back.
Candidates report a technical session on LLM systems as part of the FDE loop, often starting from their take-home and going deeper. Its signature move is following one decision down to the mechanism behind it. This lesson covers what interviewers probe, how to trace a request through its boundaries, and a bank of practice questions in the same style.
What do interviewers probe?
OpenAI’s interview guide says interviews aim to understand how candidates approach problems, make decisions, and communicate reasoning. In an LLM systems conversation, that usually means six areas, each tied to a decision we made.
Area | The Question Behind It |
Agents | Why a fixed workflow, one agent, or several? What does each extra agent cost? |
Retrieval | Where does the access check run relative to ranking and the model call? |
Evals | What are the denominators, and which results were observed versus predicted? |
Latency | Which interval is being measured, and for which requests? |
Cost | What does a task cost under stated assumptions, and what’s left out? |
Failure handling | What happens when a check fails, a call times out, or evidence is missing? |
Keep the customer → problem → build → proof → handoff account intact, and put the decision and its strongest evidence first. That gives the interviewer a clear position to push on.
How do we trace one request?
A common technical follow-up is: "Walk me through what happens when a request arrives." A strong answer explains each processing stage in order and identifies the controls enforced at each boundary.
Authenticate: The application verifies the operator's identity using trusted credentials, never model-generated claims.
Check eligibility: Application code verifies the operator's access permissions and each record's applicability and freshness before protected content is passed to ranking or generation. If access is denied, a record is stale, or eligibility cannot be confirmed, return a generic "unavailable" response without exposing protected content.
Retrieve: Rank only eligible passages. If no eligible evidence supports the request, route it to manual review.
Draft: Generate a response from eligible evidence ...