Initial Design of Quora

Initial design

The design will comprise the following building blocks and components:

• Web and application servers: A typical Quora page is generated by various services. The web and application servers maintain various processes to generate a webpage. The webservers have Master processes and the application servers have Worker processes handling various requests. Master processes distribute work among the Worker processes using a router library. The router library is enqueued with tasks by the Master processes and dequeued by Worker processes. Each application server maintains several in-memory queues for handling different user requests. The following illustration provides an abstract view of web and application servers.

• Data stores: Different types of data require storing in different data stores. We can use critical data like questions, answers, comments, and upvotes/downvotes in a relational database like MySQL as they offer a higher degree of consistency. NoSQL like HBase can be used for storing the number of views of a page, answer scores to rank answers, and the extracted features from data to be used for recommendations later on. It is because recomputations of features is an expensive operation, and HBase can be a good option for storing and retrieving data at high bandwidth. We require high read/write throughput because Bigdata processing systems use high parallelism to efficiently get the required statistics. Also, blob storage is required for storing videos and images posted in questions/answers.

• Distributed cache: For performance