The LLM Systems Deep Dive
Defend LLM system decisions by tracing the mechanisms behind authorization, retrieval, validation, and failure handling. Lead with a recommendation and its strongest evidence, then answer follow-up questions at the requested depth. Keep task quality, freshness, unauthorized exposure, effect evidence, and operating ownership separate. Quarantine faulty protected-data paths, propose enforceable repairs, and verify them against original observed failures and held-out cases. Compare fixed workflows and coordinated agents on the same workload, stating cost assumptions and latency boundaries. Preserve the customer, problem, build, proof, and handoff account while distinguishing implemented controls from proposals.
We'll cover the following...
“Show me exactly where the access check runs.” A question like that ends the presentation and starts the technical deep dive, where an interviewer follows one decision down into the mechanism behind it. Your Northstar recommendation is an offline-only, operator-reviewed triage draft. Defending it means showing where data is admitted, where model output is checked, and where failures go. The skill is depth on demand: lead with the decision, then go exactly as deep as the question asks.
What do interviewers probe?
OpenAI’s interview guide says its interviews look at how candidates approach problems, make decisions, and communicate their reasoning. An FDE loop usually includes a technical deep dive on LLM systems, though your recruiter’s instructions tell you what your loop actually includes. The FDE role description adds technical delivery, close customer work, and trade-offs between scope, speed, and quality.
Expect questions in six areas, each aimed at a decision you made. On agents: why a fixed workflow, one agent, or several, and what each extra agent costs. On retrieval: where the access check runs relative to ranking and the model call. On evals: what the denominators are, and which results you observed versus predicted. On latency: which interval you’re measuring, and for which requests. On cost: what a request costs under stated assumptions, and what’s left out. And on failure handling: what happens when a check fails, a call times out, or evidence is missing.
Keep the customer → problem → build → proof → handoff account intact, and put the decision and its strongest evidence first. For Northstar, the opening might be:
“I recommend an offline-only, read-only triage draft that an operator reviews. The stale-evidence summary found two wrong admissions among 12 records, so the freshness check needs repair and verification before a pilot. The operating owners are proposed, not yet agreed.”
That gives the interviewer a decision to challenge: where an eligibility check runs, what you observed, and who would support the system.
How does one request flow?
The existing northstar_workflow.py has test records, a local test identity with an entitlement check, a one-call budget, allowed labels, and manual-review routes. It doesn’t yet have the freshness check. A complete workflow would run like this:
Authenticate: The application establishes the operator’s identity from trusted credentials, never from model-generated text.
Check eligibility: Application code checks the operator’s entitlement and the record’s applicability and freshness before any protected text reaches ...