A practitioner's guide to building scalable systems

Building Scalable Systems: Distributed Systems basics to System Design Interviews

We want to give you the tools to compare multiple approaches and pick the one that works for your scenario.

Grokking the Distributed and Scalable Systems

# There's no pendulum in your computer

<img style="float:right; padding:20px" src="/api/collection/5689413791121408/5645316556587008/page/5769975163781120/image/5653942327312384" />


In the late 16th century, Galileo discovered that a pendulum of a certain length swings back and forth in almost 1 second and that is the principle used behind the pendulum clocks. However, pendulum stops after a while due to air friction.


That's why the pendulum clocks (like the one pictured here) have a spring. You need to wind up the string every day or so which stores the potential energy and that keeps the pendulum moving.


But you never wind up your computer, and it doesn't come with a pendulum in perpetual motion. 

Did you ever wonder how your computer clock works? Turns out that your computer has a type of a "pendulum" and a "spring" (metaphorically speaking).

_This brings us to Quartz crystals and crystal oscillators._

# Quartz Crystal and Crystal Oscillator

Let's look at how modern digital clocks work - starting from understanding what a Quartz Crystal is.


## What is a Quartz Crystal?
![](/api/collection/5689413791121408/5645316556587008/page/5769975163781120/image/5737161613639680)

Quartz is a crystal with a unique property. When a small voltage is applied to this crystal, it vibrates at an exact frequency. Different crystals are created with different frequencies.

Typically, the Quartz Crystal used in clocks has a frequency of 32768 Hz (which means it oscillates 32768 times every second).

## What is a Crystal Oscillator

{{INSERT__OSCILLATOR_PICTURE_HERE}}

A Crystal Oscillator is an electric circuit that uses the Quartz Crystal to generate periodic electric pulses. It usually has a battery to generate electricity.

Here's how it works

1. A Crystal Oscillator sends electricity to the Quartz Crystal. 
2. Quartz Crystal vibrates at 32768 vibrations per second.
3. The circuit in the Oscillator counts its vibrations and generates an electric pulse every 32768 vibrations (every 1 second).

### Clock Jitter and Clock Drift in Crystal Oscillators

Don't want to create it into a course in Physics or Electronics so know that Quartz Crystal oscillations exhibit very very small inaccuracies in the short-term and long-term based on the crystal, temperate and many other factors. The most important thing to remember here is that Crystal Oscillations are very accurate but not 100% correct. 

Over time, you might see a little bit of drift in time.

_Now the next question is that where are the Crystal Oscillator used in Computers and that is a good time to look at the Real-Time Clock (which every computer has)._

# What is a Real-Time Clock?
The real-time clock is the battery-powered computer clock that tracks the computer time even when your computer is powered-off. 

The Real-Time Clock on your computer is a battery-backed Crystal Oscillator. It keeps oscillating using the battery even when you shut down your computer or power is disconnected. That's why, when you turn on your computer, you don't have to reset the time.

# Is Time a solved Problem in Computers?
Not so fast. Let's discuss a few examples of problems with current clocks.


## Clock Drift
We already discussed that Quartz Crystal based clocks have an inevitable loss of precision over time especially when it experiences high temperature on instability in the power source. 

This loss of precision causes the clocks to drift.

## Clock Skew
Imagine a new node in the network asked another computer for the current time. Due to unpredictable network delays, the time reported was inaccurate. Now, this correct Node has incorrect time. Over time, clock drift might skew this time even more. 

In practice, I've found that the Nodes in the same network has system clocks reporting very different times where the difference is up to 4-5 minutes.

Because Clock skew is a well-known problem, we have a solution to the problem that doesn't fix the problem but ensures that the clock skew between the nodes in a network remains within a certain threshold.

_It's called Network Time Protocol (or NTP). Let's look at what it is._



# There's no pendulum in your computer

<img src="/api/collection/5689413791121408/5645316556587008/page/5769975163781120/image/5653942327312384" />


In the late 16th century, Galileo discovered that a pendulum of a certain length swings back and forth in almost 1 second and that is the principle used behind the pendulum clocks. However, pendulum stops after a while due to air friction.


That's why the pendulum clocks (like the one pictured here) have a spring. You need to wind up the string every day or so which stores the potential energy and that keeps the pendulum moving.


But you never wind up your computer, and it doesn't come with a pendulum in perpetual motion. 

Did you ever wonder how your computer clock works? Turns out that your computer has a type of a "pendulum" and a "spring" (metaphorically speaking).

_This brings us to Quartz crystals and crystal oscillators._

# Quartz Crystal and Crystal Oscillator

Let's look at how modern digital clocks work - starting from understanding what a Quartz Crystal is.


## What is a Quartz Crystal?
![](/api/collection/5689413791121408/5645316556587008/page/5769975163781120/image/5737161613639680)

Quartz is a crystal with a unique property. When a small voltage is applied to this crystal, it vibrates at an exact frequency. Different crystals are created with different frequencies.

Typically, the Quartz Crystal used in clocks has a frequency of 32768 Hz (which means it oscillates 32768 times every second).

## What is a Crystal Oscillator

{{INSERT__OSCILLATOR_PICTURE_HERE}}

A Crystal Oscillator is an electric circuit that uses the Quartz Crystal to generate periodic electric pulses. It usually has a battery to generate electricity.

Here's how it works

1. A Crystal Oscillator sends electricity to the Quartz Crystal. 
2. Quartz Crystal vibrates at 32768 vibrations per second.
3. The circuit in the Oscillator counts its vibrations and generates an electric pulse every 32768 vibrations (every 1 second).

### Clock Jitter and Clock Drift in Crystal Oscillators

Don't want to create it into a course in Physics or Electronics so know that Quartz Crystal oscillations exhibit very very small inaccuracies in the short-term and long-term based on the crystal, temperate and many other factors. The most important thing to remember here is that Crystal Oscillations are very accurate but not 100% correct. 

Over time, you might see a little bit of drift in time.

_Now the next question is that where are the Crystal Oscillator used in Computers and that is a good time to look at the Real-Time Clock (which every computer has)._

# What is a Real-Time Clock?
The real-time clock is the battery-powered computer clock that tracks the computer time even when your computer is powered-off. 

The Real-Time Clock on your computer is a battery-backed Crystal Oscillator. It keeps oscillating using the battery even when you shut down your computer or power is disconnected. That's why, when you turn on your computer, you don't have to reset the time.

# Is Time a solved Problem in Computers?
Not so fast. Let's discuss a few examples of problems with current clocks.


## Clock Drift
We already discussed that Quartz Crystal based clocks have an inevitable loss of precision over time especially when it experiences high temperature on instability in the power source. 

This loss of precision causes the clocks to drift.

## Clock Skew
Imagine a new node in the network asked another computer for the current time. Due to unpredictable network delays, the time reported was inaccurate. Now, this correct Node has incorrect time. Over time, clock drift might skew this time even more. 

In practice, I've found that the Nodes in the same network has system clocks reporting very different times where the difference is up to 4-5 minutes.

Because Clock skew is a well-known problem, we have a solution to the problem that doesn't fix the problem but ensures that the clock skew between the nodes in a network remains within a certain threshold.

_It's called Network Time Protocol (or NTP). Let's look at what it is._



It's a good idea to understand how computers clocks work along with in order to understand the issues associated with clock drifts

What and Why

Node in a Distributed System

Performance and Numbers

Fault Tolerance and Availability

Understanding TIME in Distributed Systems

Data Centers for software engineers

Scaling Distributed Systems

Load Balancers

Structured Storage at Scale

Sharding in Storage Systems

SQL vs NO-SQL

Blob Storage at Scale

Caching in Distributed Systems

Queues and Asynchronous Processing

Data Analytics and Batch Processing

Logging and Monitoring

Searching at Scale

Conclusion

How do Computer Clocks Work and What's Clock Skew?

There’s no pendulum in your computer