Search⌘ K
AI Features

FDE System Design: Data Pipelines, Integration and Reliability

Design a reliable data integration pipeline for an FDSE System Design round, where users must know whether the data they act on is fresh and complete. Separate source-owned from locally owned facts, handle duplicate, late and conflicting deliveries with revisions, and commit progress, coverage and state atomically. Tie user-visible freshness to source-confirmed coverage, fence stale workers with leases, map failure signals to owners and actions, and keep access checks current for exports.

“The worker is alive, but the data is six minutes old. Can the dispatcher trust the exception list?” A fast dashboard response cannot answer that question. A dependable pipeline must connect accepted source changes, durable progress and the freshness shown to the person making a decision. The design below makes that connection inspectable during failure and recovery.

Can dispatchers use a six-minute-old list?

Dispatchers review delayed shipments for their assigned depot and decide whom to contact. The application never automatically reroutes shipments.

The agreed carrier-data freshness limit is five minutes. Beyond that:

  • Retain the previous list with its original coverage timestamp and a prominent stale state.

  • Disable carrier-dependent contact recommendations until refresh or an approved manual check.

User-visible freshness must follow committed source coverage, with an explicit consequence when that coverage becomes too old. Neither a live worker nor a fast HTTP response establishes source completeness or current physical shipment truth.

The sources support different claims:

  • Official role: The FDSE posting describes customer-facing architecture and design, business-critical data, custom applications and end-to-end execution. This supports operational integration’s relevance, not a particular interview format or scoring rubric.

  • Reported preparation guidance: Third-party guidance emphasizes user-centered pipelines, ingestion, storage, reliability and trade-off defense,not confirmed requirements for an individual candidate.

  • Practice scope: Public reports do not establish a universal Foundry ...