According to a number of sources, Microservices are the latest and greatest way to set up web sites. They're said to be better than SOA and an improvement on the hated monolith. I won't go into describing either the Microservices Pattern or the dreaded Monolith Pattern as they are described in the links above.
What I will do is try to summarise why monoliths are believed to be 'bad' and microservices are believed to be 'good'. That will give me a basis to introduce 'The Real World'. Not one of the longest running TV shows in history but the place we all live and work in.
After I've thoroughly debunked the myth of the monolith (don't hate me for being over-confident), I'll point out the real pattern that exists in most businesses. That pattern is the Solar System Pattern. And trust me, you're probably already using it in your company.
There are a number of reasons that the monolith is thought to be 'bad'. And all these downsides are in fact true.
Microservices are believed to be the solution. Simple, separate, and easy to deploy, these little gems allow many small teams to collaborate with each other within an organization without ever having to worry that there is a clash. Each team owns a service, operating and deploying it on their own schedule.
The benefits are:
And now for the problem. Does the real world accept our principles of beautiful and coherent design? Does it allow us to break the world into small pieces? And, really, should it?
Of course, you answer. The case (above) is compelling.
However, many of the current blog posts about microservices exist in the following matrix:
The first problem in microservices is actually the easy one, but is still hugely hard. It's all about managing a fleet of services. Managing dozens of servers, even with Docker and modern cloud farms, is still hard. They need to be clustered, kept running and scaled. This can take a lot of time to get right. And reliability of distributed systems is hard. See here and here for a flavor.
And then there's the hard problem. Most businesses have complex business domains that interlock tightly. Exploding these into microservices does not produce separate concerns as the domain objects interact. This leads quite quickly to tightly coupled microservices with wide interfaces, the exact opposite of what is desired.
Most real businesses have a hefty domain model with many complex interactions, and a number of systems that need to collaborate over the domain model. Those systems are forced to split on technology grounds because they process completely different workloads, such as online order processing, business intelligence, lead management and real-time personalization. Unfortunately, despite the best efforts of microservices, each of these workloads needs access to the full suite of data. Can you imagine the CFO agreeing not to see the sales numbers because they shouldn't be shared between system boundaries for architectural reasons?
The Solar System Pattern is about accepting the real world. It acknowledges that most businesses will have a complex domain model, that for technical reasons will need to be distributed to a number of systems.
Those multiple systems, that need to have full or substantial knowledge of the domain model, I call Suns. They are the core of a solar system. And in reality, it is very rare for our techie solar systems to have just one sun. The most typical small company binary system looks like 2 suns:
Even slightly larger companies, such as ours, have many suns, all collaborating over the same domain model.
And solar systems have planets. These are the microservices that circle around the sun(s). They are generally smaller and more self-contained with less required knowledge of the domain model. They peel off functionality that really doesn't need to understand large bits of the domain model. They can really adhere to the single responsibility principle.
Despite efforts to limit the amount of domain knowledge that suns need, they are always bigger, more complex and share more data with each other than the microservices that orbit them.
A bit more detail.
What is a sun in the Solar System Pattern? It's a monolith. It's an encapsulation of a complex domain model where the different components don't easily separate from each other.
In a nutshell, a sun is a system that holds data and manipulates a substantial portion of a domain model.
For example in our domain (we are the UK's largest business insurance broker), one of our suns (we have more than one) is our main policy management system. It is a 'quote and buy' system and is used to manage all our customers quotes and policies. It handles when a customer wants to change any aspect of their policy, and deals with all the complexities. Our consultants use it to help our customers. And it provides an array of functionality. We frequently look to split functions out, but find it hard to see good seams.
Inside this sun, sits our domain model. Our domain model has customers with quotes and policies. The policies have endorsements and changes and policy documents. Our customers have answered a series of questions about their insurance needs that led to their being offered and then purchasing a policy. Those questions are based on a questionnaire that was answered. All of these elements are part of our domain model.
And on and on. All of these elements interlink in a domain defined way. We've even created an internal site to document it called Simplypedia. There you can look up what a Policy actually is to us and which other domain objects it connects with.
Sure it would be nice for policies to relate to customers in completely independent ways, but they simply don't (for example, a policy provides insurance to a customer, and therefore needs to know who that customer is).
As I mentioned, we actually have multiple suns, since our quote and buy and CRM system interacts with a number of other systems. For example, it interacts with our Big Data analytics system which wants to analyse the full content of our customer, policy and other domain objects. In order to do that, the analytics system requires a very wide interface with our first sun, the quote and buy system.
And what about the lead management system that prioritises who we contact based on the answers our future customers gave us? It needs all the data as well. It uses that data to prioritise who we contact.
The microservices in our solar system are components with limited or well defined interfaces. We've got a PDF generator that crunches HTML. That's a well defined interface. We've got a mail queue system that sends out quote emails that get pushed to it from the main transactional sun. These are tight and well contained interfaces and we can just deploy them at will. And in fact, we hardly ever have to deploy them.
One of the big challenges of real businesses that are iterating rapidly (or not so rapidly) is to keep the domain model up to date. Domain changes are typically cross-cutting concerns and suffer from having to be implemented everywhere that the domain model objects that are changing are used. In practice, this means updating each Sun to communicate, store and interact with the changed domain model. The more places you have to do this, the harder it is.
For example, we recently changed the name of Product in our domain model to Vertical. That small change needed to be rolled out to every system (all Suns) that knew about Product so that it now knew about Vertical. This meant for us: our quote and buy system, our analytics system, our lead management system and our renewals system to call out the major ones. That entailed the modification of APIs, data migration, schema changes and other machinations in order to flow the change through. If we hadn't done the work, we would have had systems with inconsistent domain models, a sure cause for extensive pain. For example, it leads to people not understand a legacy term that is used in only one system and therefore misusing the data.
More generically, the cost of not flowing the domain changes through is that different systems will have different representations of the business. Changes to the domain model are inherent to all businesses, and the maintenance issue is inescapable. They used to call this metadata management but now we call it domain driven design. Microservices don't help here, other than if they are able to avoid knowing about the domain model entirely, in which case they don't need updating. If the microservices are intimate with the domain model, then the more microservices there are, the more changes are needed.
Especially since the reasons we have multiple suns (let alone microservices) is that we need to separate them for technical (functional or performance) reasons.
There's no getting around your domain model. Typically it is tightly linked and just doing object oriented decomposition on it is challenge enough. When you start to want to separate it into multiple microservices, you start having to think about distributed transactions and numerous other mechanisms, just to be able to create 'small' services. This can be worth it, but typically because performance or some other metric has become too much of a problem. Martin Fowler says that complexity is that measure, but I'm not convinced. Artificial boundaries between services don't mean that the services have been correctly separated, and even worse, if a cross-service domain change comes along, the amount of work can skyrocket in a microservices architecture.
So dismantle your majestic monolith at your peril. You've got a domain model to keep alive.
Want to know more about what it's like to work in tech at Simply Business? Read about our approach to tech, then check out our current vacancies.Find out more
6th Floor99 Gresham StreetLondonEC2V 7NG
Sol House29 St Katherine's StreetNorthamptonNN1 2QZ
© Copyright 2020 Simply Business. All Rights Reserved. Simply Business is a trading name of Xbridge Limited which is authorised and regulated by the Financial Conduct Authority (Financial Services Registration No: 313348). Xbridge Limited (No: 3967717) has its registered office at 6th Floor, 99 Gresham Street, London, EC2V 7NG.