Microservices, Integration and building your digital enterprise

It's time to rethink about your enterprise IT eco system. Technology space is going through a period of major revamp and whether you accept it or not, it is changing the way people do business. You may be a software architect of a multi billion enterprise or the only software engineer of a small startup organization which is trying to figure their way out in the business world. It is essential to know about the direction of the technology space and make your moves accordingly. At the 33000 feet, enterprises throughout the world are moving (most of them are already moved) towards digital technology. You may have already brought several third party systems in to your IT eco system and they are functioning well within your requirements. All is running well and your organization is profitable. 

All is well. Why bother about these hypes? Let me tell you one thing. The world of business is moving so fast that a billion dollar company today will become an organization with a debt on their account within a very short period. There will be a new startup offering some cool ideas and they will grab all your customers if you don't provide them the innovation the world is demanding. It is hard to do the innovations without having a proper infrastructure to deliver them to your customers. That is where you need to plan your IT eco system thinking not only about today but next few years. 

Having said all the above facts, there is always one thing which is stopping you from bringing these innovations to your organization. That is none other than the budget. Your boss might say "Well, that is a cool idea. Can you do it for free?" Well, you can, upto some extent. There are several open source solutions which you can use to bring innovations to your enterprise. Let's focus more about the methods rather than the budget. 

Let's think about a scenario where your organization is going to expose new business functionality to your customers through APIs such that web clients and mobile applications can consume your services through these APIs. To provide this new functionality, you need to integrate with different internal systems and you are going to develop new set of services to cater the business requirements. You have the following requirements to deliver your new business functionality to your customers.

1. Providing APIs
2. Integrate different systems
3. Develop new services

There can be more requirements like monitoring, etc. But let's focus on the major requirements and start building your system. API management has been there for some time and there are so many open source and proprietary vendors from which you can select an offering. For the integration of systems also there are many. The real interesting thing is how can I develop my services? As you may have already heard, there is a new concept looming in within the software industry for developing services. That is Microservices Architecture (MSA). You can read about MSA and it's pros/cons almost everywhere. The concept of MSA is that you develop your services in a way that they can be developed/deployed and maintained in a loosely coupled, self contained way. The basic idea is that you should build your services in a way that those services provide real business functionalities as a self contained service. You can take down that service without shutting down your entire system but only that specific service. There are several micro services frameworks available as open source offerings and here is a list of promising MS frameworks.

1. WSO2 MSF4J - http://wso2.com/products/microservices-framework-for-java/
2. Spring Boot - https://spring.io/blog/2015/07/14/microservices-with-spring
3. Spark - http://sparkjava.com/
4. RestExpress - https://github.com/RestExpress

You can use any of the above mentioned frameworks for developing your new services. These services might expect messages with different formats and you need an integration layer to deal with these different types of messages. The below picture shows the architecture of your digital enterprise which consists of the previously mentioned key components (API, Integration, Services).

Sometimes there is a misconception about MSA that it will throw away the integration layer and built on top of "dumb pipes" for message exchange. This is not correct specially when you have more and more micro services, you need an integration layer to deal with different message types and protocol types. You need to keep this in mind and plan for future requirements rather than thinking about a simple service composition scneario where you can achive all the communication using "dump pipes".

One of the main areas of interest of MSA is the deployment strategy and the invlolvement of DevOps. You can deploy your micro services in containers such that they can be bring up and down whenever necessary without affecting other components. When it comes to integration solutions, they are like more solid components in your architecture which does not need to bring up and down so frequently. You can deploy them in a typical Hardware or Virtual infrastructure without worrying about containerization.

Once you have this kind of architecture, you can achieve the following key benefits which are critical in the modern business eco system.

1. Ability to expose your services to multiple consumers (through APIs) in a rapid manner
2. Ability to come with new services in a quick time (Micro services deployment is rapid)
3. Ability to connect with third party systems without worrying about their protocols or message types (Integration layer)

Finally, you can add analytics and monitoring in to the picture and make your system fault tolerant and well monitored for any failures. That would be a subject matter for a separate article and I will stop right here.