Megastore Deep Dive for System Design
Understand the design and motivations behind Google Megastore, a scalable and ACID-compliant cloud storage system. Learn how it combines NoSQL scalability with relational database features to support interactive applications at global scale. This lesson helps you grasp Megastore's functional and non-functional requirements, architecture trade-offs, and its approach to maintaining consistency, availability, and latency across distributed data centers.
Motivation
As desktop programs migrate to the cloud, interactive online services challenge the storage market to fulfill new needs. E-mails, shared reports, and social networking are developing at an enormous speed, pushing the limits of existing infrastructure. Handling the storage needs of these services is difficult due to the following demands:
- Applications must be extremely scalable due to the vast audience of potential consumers that the Internet brings. Using traditional databases such as MySQL since the datastore allows a service to be quickly developed, but expanding the service to millions of people demands a total overhaul of the storage infrastructure.
- Organizations have to compete for users. This demands speedy product development and time-to-market. Usually, a NoSQL solution with some custom consistency models has its learning curve, and developers might be forcing the solution to the problem that is more amenable to traditional relational databases.
- Low latency is essential for storage systems.
- The application should give the user a consistent data view. The outcome of a change should be seen instantly and indefinitely.
- The services should be highly available. The system should operate uninterrupted despite server or component failures.
Technology options
The demands above have trade-offs. Relational databases offer comprehensive capabilities for easily implementing applications, but scaling to hundreds of millions of people is tough. Although NoSQL datastores such as ...