Search⌘ K
AI Features
Home
Courses
Google FDE Interview Prep

Debugging a Gemini-Based Pipeline

Explore how to debug complex Gemini-based pipelines by reproducing failures, localizing defects, repairing normalization issues, and proving fixes with regression tests. Understand the importance of separating error types like schema drift and empty retrieval, apply no-progress guards to avoid redundant searches, and prepare clear explanations for technical interviews in system design.

We'll cover the following...

Request dbg-701 asks, “How often must instrument Q6 be calibrated?” Compare the expected and observed outcomes:

  • Expected: A supported answer of 45 days from authorized manual M8, revision 4.

  • Observed: Three model requests and two searches, followed by rejection of another proposed search because it exceeds the tool budget.

Why did a successful search fail to give the model usable evidence? You will localize the failure, repair one boundary, and prove the repair,the discipline an interviewer needs to see when presenting a broken pipeline.

Inspect the failed request

A stronger instruction,“Always use the retrieved manual and never loop”,looks attractive because repeated calls are visible. Increasing the tool budget looks attractive because the system stopped before answering. Neither change explains what happened to the retrieved passage.

Use the debugging sequence: Reproduce the failure, localize it through traces to retrieval, prompt assembly, tool/interface handling, or model behavior, repair it, and add a regression test. Schema drift, empty retrieval, and loops require different investigations.

Start by separating three observations:

  • Dependency outcome: Both searches completed successfully.

  • Evidence outcome: The model received no normalized passages.

  • Terminal outcome: Orchestration rejected a third proposed search before dispatch and returned BUDGET_EXHAUSTED.

These observations allow several hypotheses. Search could have found nothing, an adapter could have discarded evidence, or prompt assembly could have omitted it. The terminal reason alone cannot choose among them.

Freeze dbg-701

A useful reproduction preserves the relevant input and every version or response that could change the execution. For this independent lab, create a folder named calibration-debugger; no earlier lesson files are imported or modified.

The four files are:

  • calibration_debugger.py: Scripted dependency adapters, defective and repaired normalizers, orchestration, boundary events, and counters.

  • calibration_debug_traces.json: Requests, trusted scope, source snapshots, raw responses, and scripted model behavior.

  • test_calibration_debugger.py: Preservation, compatibility, failure-path, and orchestration regressions.

  • calibration_debug_notes.md: Diagnosis, experiment controls, test commands, actual results, and limitations.

Freeze these values for dbg-701:

{
  "request_id": "dbg-701",
  "question": "How often must instrument Q6 be calibrated?",
  "trusted_scope": {"M8": 4},
  "source_snapshot": "manual-snapshot-4",
  "query": "instrument Q6 calibration interval",
  "prompt_ref": "calibration-prompt-2",
  "application_ref": "calibration-app-1",
  "search_interface_version": 2,
  "raw_response": {
    "schema_version": 2,
    "matches": [{
      "source_id": "M8",
      "revision": 4,
      "content": "Instrument Q6 must be calibrated every 45 days."
    }]
  },
  "budgets": {"model_requests": 3, "tool_executions": 2}
}

The trusted scope admits exactly M8 revision 4; generated arguments cannot expand it. The raw payload preserves what crossed the search boundary before any application transformation.

The scripted model requests search_manual whenever its captured context lacks the exact admitted passage. When that passage is present, it returns the supported 45-day answer. Freeze that conditional response script and the prompt throughout the adapter experiment.

Record raw responses, normalized observations, assembled model context, proposed calls, dispatches, and terminal decisions. For production investigations, keep a protected reference to sensitive evidence rather than copying transcripts into ordinary traces. These lab fixtures reproduce application control flow; they do not reproduce changing live Gemini behavior.

Find the first divergence

A first divergence is the earliest observed boundary whose output violates its expected contract given valid upstream inputs. Finding its location narrows the investigation; explaining the underlying cause requires another step.

Inspect the execution in causal order. Choose the earliest broken boundary and identify the evidence that rules out an earlier failure.

dbg-701: Find the first violated contract
Scope admissionBoundary recordModel requestSearchAdapterSearch and adapterBudget admission
Do you find this helpful?

The complete baseline path is:

  1. Scope admission: Trusted state admits M8 revision 4.

  2. First model request: With no evidence yet, the script proposes search_manual using the frozen query.

  3. First search: The authorized search returns the version-2 payload containing the 45-day passage.

  4. Normalization: The old adapter emits [].

  5. Second model request: Prompt assembly includes the empty normalized observation. The script proposes the same search.

  6. Second search: The same valid passage returns and is discarded again.

  7. Third model request: The script again sees no evidence and proposes search.

  8. Admission stop: The two-execution tool budget is ...