Detailed Design of Tectonic

We have already discussed the overview of the Tectonic architecture, and now it is time to learn about its components and how they interact with each other.

Client Library

The Tectonic client code needs to map its user’s read/write requests to appropriate Tectonic API calls by interacting with the Metadata and Chunk Stores. As an example, when a user asks to read from a file, the Client Library utilizes the metadata service to find the corresponding data chunks on disks (by series of mapping from name layer to file layer to block layer to the chunks. We’ll learn about this series of mappings a bit later in this lesson). Client code can cache some of this information and then goes to the specific storage node to read the data. All such intelligence makes the Client Library fairly complex.

Tectonic groups chunk to make blocks to amortize the cost of metadata keeping. Blocks in Tectonic are encoded using either Reed-Solomon (RS) encoding, which divides the block into $X$ data chunks and $Y$ code/parity chunks, or are replicated, in which $N$ identical chunks each hold a full copy of the block.

Note: Reed Solomon codes are based on beautiful mathematics such as modular arithmetic and Lagrange interpolation. However, their detailed examination is beyond the scope of this lesson. Reed-Solomon codes are used in many places, such as QR codes, CDs, and other storage and communication media.

Compared to some other distributed file systems (such as GFS), Tectonic only allows a single writer to a file at a time. Tectonic uses locking on files to give exclusive writing permissions to one client. Doing so simplifies writing and replication by using hedged writes (meaning sending the request to a preferred node, if that one fails, we fall back to others). The writing operation is performed using the following steps:

1. When the file is opened for the append operation, the token is added to the file’s metadata. This token will be required for mutations on the file.
2. The writer having the matching write token for that block with the file metadata can update that block.
3. When a new writer comes to update the same block, a new token will be generated and updated in the file’s metadata (and the old one will be deleted). Since the token is updated, the previous writer will no longer have the right to update the block, and the new writer with the matching token can perform the write operation.

Metadata Store

The Metadata Store is a fine-grained partitioningA technique that reorganizes the data blocks into chunks allowing operations for partitioning the chunks system that uses hash-partitioning and ZippyDB servers for metadata storage to store shardA unit of data management replicas for scalability, load balancing, and operational simplicity.