Monoliths, Microservices, and Shared Databases

There wasn’t an easy way to put this into the course, but because we discussed APIs in the “API Endpoints” lesson, there is an implicit assumption that we might have more than one Rails app someday, so we want to spend this appendix talking about that briefly.

When a team is small, and we have only one app, whether we know it or not, we have a monolithic architecture. A monolithic architecture has a lot of advantages. Starting a new app this way has a very low opportunity cost, and the carrying cost of a monolithic architecture is quite low for quite a while.

The problems start when the team grows to an inflection point. It’s hard to know what this point is, as it depends highly on the team members, the scope of work, the change in the business and team, and what everyone is working on. Most teams notice this inflection point months—sometimes years—after they cross it. Even if we know the day we hit it, we still have some decisions to make. Namely, do we carry on with a monolithic architecture? If not, what are the alternatives, and how do we implement them?

In this section, we want to try to break down the opportunity and carrying costs of:      

• Staying with a monolithic architecture.

• Deploying a microservices architecture.

• Using a shared database amongst multiple user-facing apps.

The third option—sharing the database—is usually discussed as an antipattern, but as we’ll see, it’s anything but. It’s important to understand that our system architecture—even if it’s just one app—is never done. We never achieve a state of completeness where we can then stop thinking about architecture. Rather, the architecture changes and evolves as time goes by. It must respond to the realities we face, not drive toward some idealistic end state. 

So, we strongly encourage you to understand monolithic architectures, microservices, and shared databases as techniques to apply if the situation calls for it. It’s also worth understanding that any discussion of what a system’s architecture is has to be discussed in a context. It’s entirely possible to have 100 developers working on 30 apps, some of which are monolithic. within a given context.

Let’s start with monolithic architectures.         

Monoliths get a bad reputation 

If we have a single app, we have a monolithic architecture. In other words, a monolithic architecture is one where all functions reside in one app that’s built, tested, and deployed together.

When a team is small and when an app is new, a monolith has an extremely low opportunity cost for new features as well as low carrying cost. The reason is that we can add entire features in one place, and everything we need access to for most features—the UI, the database, emails, caches—are all directly available.

The larger the team and the more features are needed, the harder a monolith can be to sustain. The carrying cost of a monolith starts rising due to a few factors.

First, it becomes harder to keep the code properly organized. New domain concepts get uncovered or refined and this can conflict with how the app is designed. For example, suppose we need to track shipping information and status per widget. Is that a set of new widget statuses, or is it a new concept? Additionally, if we add this concept, how will it confuse the existing widget status concept?

This domain refinement will happen no matter what. The way it becomes a problem with a monolith is that the monolith has everything—all concepts must be present in the same codebase and be universally consistent. This can be extremely hard to achieve as time goes by. The only way to achieve it is through review, feedback, and revision. Whether that’s an upfront design process or an after-the-fact refactoring, this has a carrying cost.

Another carrying cost is the time to perform quality checks like running the test suite. The more stuff our app does, the more tests we have and the slower the test suite takes to run. If we run the test before deploys, this means we are limiting the number and speed of deploys. A single-line copy change could take many minutes (or hours!) to deploy. 

Solving this requires either accepting the slowdown or creating new tools and techniques to deploy changes without running the full test suite. This is an obvious opportunity cost, but it also creates a carrying cost that—hopefully—outweighs the carrying cost of running the entire test suite. 

Related, a monolith can present particular challenges staying up to date and applying security updates, because the monolith is going to have a lot of third-party dependencies. We will need to ensure that any updates all work together and don’t create interrelated problems. This can be hard to predict.

An oft-cited solution to these problems is to create a microservices architecture, but this trades old problems for new problems. 

Microservices are not a Panacea        

Previously known as a service-oriented architecture (SOA), a microservices architecture is one in which functionality and data is encapsulated behind an HTTP API, built, maintained, and deployed as a totally separate app.

The reason to do this is to solve the issues of the monolith. The internal naming, concepts, and architecture of a service don’t have to worry about conflicting with other services because they are completely separate. A microservice creates a context in which all of its internals can be understood. Taking the status example above, we might create a widget shipping service that stores a status for each widget. That status is in the context of shipping, so there’s no conceptual conflict with some other service maintaining some other type of status.

Microservices also naturally solve the issue of deployment. Because each service is completely separate,  deploying a change, say, in the code around widget shipping, only requires running the tests for the widget shipping service. These tests will certainly be faster than running all the tests in an analogous monolith.

Microservices are particularly effective when the team gets large and there are clearly defined boundaries around which subteams can form. This isolation allows teams to work independently and avoid conflicts when interteam coordination is not required.

This sounds great, right? Well, microservices have a pretty large opportunity cost and a significant carrying cost. In our experience, the carrying cost is relatively stable despite the size of the team (unlike a monolith, where the cost increases forever). The opportunity cost—the amount of effort to establish a microservices architecture on any level—is large. 

The reason the opportunity cost is so large is that we changed the problem of our operations team from maintaining one app to maintaining N apps. There are really only three numbers in programming: zero, one, and greater-than-one. Microservices are, by definition, greater-than-one.

First, we must have clearly-defined boundaries between services. If services are too dependent or not properly isolated, we end up with a distributed monolith, where we do not reap the benefits of separation. For example, what if we made a widget data service that stored all data about a widget. 

When our widget shipping team added its new status, that would have to be added to the widget data service. These two services are now too tightly coupled to be managed independently.

Second, we must have more sophisticated tooling to make all the services run and operate. As we discussed in the “Use the Simplest Authentication System We Can” lesson, our microservices need authentication. That means something, somewhere, has to manage the API keys for each app to talk to each other. That means that something somewhere has to know how one app locates the other to make API calls.

This implies the need for more sophisticated monitoring. Suppose a customer order page is not working. Suppose the reason is because of a failure in the widget shipping service. Let’s suppose further that the website uses an order service to render its view and that order service uses the widget shipping service to get some data it needs to produce an order for the website. This transitive chain of dependencies can be hard to understand when diagnosing errors.

If we don’t have the ability to truly observe our microservices architecture, our team will experience incident fatigue. This will become an exponentially increasing carrying cost as time is wasted, morale lowers, and staff turnover ensues.

We should almost never start with microservices on day one. But we should be aware of the carrying costs of our monolith and consider a transition if we believe they are getting too high. We need to think about an inflection point at which our monolith is costlier to maintain than an equivalent microservices architecture, as shown in the “Graph Showing the Costs of a Monolith Versus Microservices Over Time” figure.

The transition to microservices can be hard. As the necessary tooling and processes are developed, it can be disruptive to the team, as shown in “Graph Showing the Costs of a Microservices Transition” figure, also on the next page.

One way to address the problems of the monolith without incurring the costs—at least initially—of microservices is to use a shared database. 

Sharing a database is viable 

When the carrying cost of a monolith starts to become burdensome, there are often obvious domain boundaries that exist across the team. It is not uncommon for these boundaries to be related to user features. For example, we may have a team focused on the website and customer experience, but we might also have a team focused on back-end administrative duties, such as customer support.

Instead of putting both of these features in one app, and also instead of extracting shared services to allow them to be developed independently, a third strategy is to create a second system for customer support and have