Search⌘ K
AI Features
Home
Courses
Grokking the API Design Interview Crash Course

Design a File Service

Explore how to design a reliable and secure file service API that supports uploading, downloading, deleting, and listing files. Understand choosing REST for client and internal communications, handling metadata and binary files, and meeting nonfunctional requirements like availability and low latency. Learn architectural decisions, API endpoints design, and strategies for scalability and security in real-world file storage systems.

We'll cover the following...

File service overview and requirements

Cloud storage services like Google Drive, Dropbox, and OneDrive offer reliable, scalable storage by transferring files between clients and powerful host systems. From a client's perspective, uploading a file looks like a simple API call, but the underlying system involves multiple frontend and backend services coordinating together.

High-level workflow of a file storage service
High-level workflow of a file storage service

Designing this file storage API requires clearly defined requirements up front.

Functional requirements

  • Upload: Securely upload files to remote storage.

  • Download: Download uploaded files from remote storage.

  • Delete: Allow only file owners to delete uploaded files.

  • List: Allow file owners to list their uploaded files.

Functional requirements of a file API
Functional requirements of a file API

Nonfunctional requirements

  • Reliability: No data loss or corruption during transfer; persistent storage after a successful upload.

  • Security: Appropriate mechanisms for secure data access.

  • Scalability: Handle increasing numbers of users and files.

  • Availability: High availability for on-demand upload and download.

  • Low latency: Minimize upload and download response times.

Nonfunctional requirements of a file API
Nonfunctional requirements of a file API

You're designing a file storage API with operations like upload, download, delete, and list. The system has two communication stages:

  1. Client to API gateway: The client authenticates and interacts via standard HTTP operations (for example, uploading and downloading files).

  2. API gateway to internal services: The gateway coordinates metadata extraction, database updates, file processing, and blob storage across multiple downstream services.

Which communication style (REST, gRPC, or GraphQL) would you choose for each stage, and why? Use the AI assessment widget below to submit your solution and get an interactive response.

Selecting Communication Style

Prerequisites

We recommend that you read about the following concepts before proceeding with this chapter, if you have not already.

1.

Should the support for multiple file encoding schemes (PNG, GIF, SVG, MKV, etc.) be considered a separate functional requirement?

Show Answer
1 / 2

Design overview and decisions

With these requirements established, the next step is to examine the file API's overall operation and component interactions. API design is influenced by business requirements and technology choices, so a combination of architectural styles often works in tandem. The following diagram shows the detailed workflow of the components and services involved in the file API design.

Detailed workflow of a file API
Detailed workflow of a file API

The functionality of these components and services is summarized in the table below.

Components and Services Details

Component or Service

Details


File servers

  • Accept and process file API requests forwarded by the API gateway
  • Split requests into metadata (user and file information) and file content

Processing server

  • Performs encoding/decoding of data into different encoding schemes
  • Encrypt/decrypt data at-rest to prevent unauthorized access

User to file mapping system (UFMS)

  • Maps users to files and where those files can be found in storage

Blob storage

  • Stores binary data objects (file content)

Temporary storage

  • Stores files and objects temporarily before processing

User server

  • Handles user related requests forwarded by the API gateway

SQL database

  • Stores information related to files and users (metadata)

CDN servers

  • Geographically distributed network for serving data to the user from a nearby location instead of serving it from the blob storage


API gateway

  • Forwards client requests to the appropriate application server
  • Authenticates and authorizes client requests
  • Rate-limits client requests to keep the server from overburdening

Client

  • The consumer of the API service
  • Can perform lossless compression before sending a file

Workflow

The workflow is divided into two stages:

  • Client to API gateway: The API authenticates the client and verifies read/write access. After successful verification, the user can upload, download, list, or delete files via HTTP requests.

  • API gateway to downstream services: The gateway forwards requests to the appropriate API services. For uploads, the ...