The Litmus Test of Enterprise Tech: Change-readiness
Business change is constant and instant, Tech teams needs to be in start-up mode
Markets are getting wider, scaling out is the new norm, so is the adoption of emerging tech
Shrinking Time to Market calls for rapid development in distributed environments backed by continuous deployment
Microservices key value prop: Change-friendliness
Their contexts and boundaries are defined in terms of distinct business capabilities
They are right-sized, invariably small (think scope of service, not lines of code)
They are independently deployable in line with business needs: for instance a new feature or a bug fix would be deployed immediately as also tracked for performance and behavior.
Deployment decisions are choreographed in collaboration with service owners eliminating the need for orchestration across multiple teams which is invariably arduous.
Service owners are free to choose the technologies and persistence mechanisms for building and operating individual services with consensus agreement on cross team parameters like log aggregation, monitoring and error-diagnosis.
Services collaborate with each other using technology-agnostic network calls
They offer a cost-effective and low-risk test bed for evaluating the effectiveness of new technologies in production environments
What does that mean for business?
Scaling is ‘on-demand’ and cost-effective, in line with business needs
Independent deployment, with quick failure isolation and rollbacks, ensures quick value delivery to the market
Ready-to-deploy business capabilities make customer engagement more holistic across different channels
Smaller codebase means significantly lower risk and cost of replacing or rewiring software (vis-à-vis the typical monolith compromise of coping with core modules running on redundant technologies)
Microservices: Here’s the deal
How they deal with change in business requirements
Unlike in monoliths, responsibilities are decomposed in respective services defined by business capabilities, hence change affects only the given module, data segments are API-encapsulated while service overlaps are mapped through higher-order services or hypermedia
How they deal with business capability enhancements or modifications
Bounded contexts enable independent deployment of the impacted service(s) without disturbing business capabilities residing in other services. This eliminates the need for time-consuming and costly regression tests of the monolith universe.
How they deal with situations where business abstractions are dependent on low-level services outside their bounded contexts
‘API gateway’ inverts the dependencies between clients and microservices in this scenario. The secondary abstraction is declared by the high-level abstraction within its service interface, and is implemented by the dependant collaborating services through several means - reducing network chattiness, performing protocol translations, concurrently aggregating service responses and transforming service responses in specific end-user formats
A closer look at API gateways
In the microservice universe, a client’s RESTful HTTP request to individual services can be a painful user experience given the plethora of requests to different services.
Enter the API gateway which tailors APIs to client’s network capabilities.
For instance, a desktop client may make multiple calls while mobile client would make a single request. The API gateway will proxy finely-grained desktop requests to corresponding services while handling coarse-grained mobile requests by aggregating multiple service call results.
Outcome: optimized communication between clients and applications while encapsulation of microservice details.
API Gateways ease the evolution of microservice: whether two microservices merge or one is partitioned into two, updation would be made at the API gateway-level, the clients on the other side of the gateway would be impervious to the change.
Why Microservices call for Continuous Deployment
Microservices are highly amenable to change and continuous deployment makes it rapid and reliable.
Microservices make deployment easier, so that it becomes faster and frequent.
Faster deployment ensures faster feedback from the market.
Faster feedback ensures timely improvements – making it more responsive and secure.
Why Microservices and Polyglot Persistence go together
The microservice approach of multiple teams managing discrete services naturally implies database capability within each service, else coupling at the data level would defeat the very purpose of autonomy.
Using multiple data stores invites eventual consistency which is a known compromise in most businesses. Even relational databases settle for eventual consistency when data is sent to or read from remote systems like value chain databases.
Like how RDBMS uses event streams for constructing reliable views, the microservice world uses event sourcing for triggering service updates from ‘events to log’
The trade-off in favor of high availability becomes even more acceptable when compared to the Multi-Version Concurrency Control issues of the relational world.
Embracing Microservices: No cookbook in the kitchen
Every organization is different – you can’t mirror success stories, or even failures for that matter
How to ensure whether microservices are fit for purpose – the adoption challenge
Microservices demand a paradigm shift - Cultural, Structural and Functional
Compelling benefits come bundled with significant complexities
The Adoption challenge
Greenfield projects
When businesses need to evolve rapidly, the monolith environment may work best for managing small number of applications to deliver the firm’s competitive edge. Microservices however would be helpful to startups in building a minimum viable product.
Brownfield projects
When established businesses need to scale rapidly across large, complex applications, microservices becomes fit for purpose but given the tangled dependencies between applications, an incremental approach to the evolution is highly advisable:
Re-engineer applications based on business priority in phase one
Build API gateways to interface with monolith applications that talk to inefficient database models
Perform minimal database modifications to maintain stateful connections
Re-engineer balance applications in fast-track mode using phase one templates and components
Spilt monolith services into microservices
Normalize relational data models and embrace efficient database models using polyglot persistence
Getting in microservice mode
Pre-adoption Diagnostics
Defining core business capabilities for decomposition into services
Dissecting services in terms of business, processes, constraints, channels and data behaviors to be able to ‘group those things that change for the same reason’
Identifying capabilities to bridge skill gaps of technical team on emerging technologies and best practices
Building the Microservices organization
Aligning the technical architecture to the business organization
Defining service boundaries, appointing service owners and custodians based on optimal team size and maximum productivity.
Promoting autonomy of service owners while enforcing rules for and monitoring implementations of ‘what the services should expose and what they should hide’
Complexities can be overwhelming
Issues of an expanding estate
Host of services
Scores of Processes post resilience
Several Interfaces
Network latency
Network failures
Need for value-added solutions
Versioning and Message serialization
Load balancers and messaging layers
Remote procedure calls
Backward compatibility and functional degradation
Use of home-grown code and off-the-shelf products for high-level automation, roll-outs based on service dependencies
Asynchronicity
Need for solution-minded teams
Visionary technical architects, Competent Custodians, Highly adept DevOps team
Database professionals conversant with polyglot persistence scenarios
Intelligent co-ordination between teams
Democratic governance
Questions that demand credible answers… a partial list
How does one move forward in the absence of standards?
How does one form the Microservice team – domain experts, technical architects, emerging tech champions, networking wizards…How does one bridge the skill gap?
How does one choose technologies, techniques, protocols, frameworks, and tools?
How does one approach design, development and deployment?
How does one identify services?
How does one define service boundaries?
How does one chip off services from the monolith structure?
Why and when does one split services, merge them, or create new services?
How does one deal with service interfaces?
Is there a way to ensure a uniform way to call services?
How does one ensure the quality of code bases? small doesn’t automatically guarantee quality
Can one build coherence and reusability across different components?
How does one tackle the accidents post the ‘first version’ release?
How does one avoid versioning nightmares?
How does one adopt a culture of individual deployment s before evolving into continuous deployment mode:
given the monolith legacy of single unit deployments and regression tests
Summing up
Microservices are an evolutionary phenomenon, not a ready-to-deploy solution
Microservices will ensure measurable value to your organization only if they are Fit for Purpose – irrespective of whether the project is Greenfield or Brownfield.
Microservices Vs. Monolith is not a black and white David Vs. Goliath scenario both have distinct value props.
Microservices are naturally attuned to the virtues of heterogeneous data stores and Continuous Deployment
Microservice trade-offs should be guided by respective trade realities, not by the experiences of other organizations