There is a lot of interest in how Mule supports emerging patterns like CQRS (Command Query Responsibility Segregation), so I wanted to create a series of blog posts discussing an insightful approach. Over the course of the series so far, we described the initial problem at hand and how to solve it using CQRS and API-led Connectivity. Next, we designed and implemented the synchronous Query API application followed by the implementation of the asynchronous Command API application with a composable API architecture.
This is how our CQRS and API-led example ends. In this last post of the series, I describe the missing puzzle piece: we need to keep the two backend systems Neo4j and Supplier Relationship Management (SRM) application in a synchronized state.
We opted for eventual consistency because we were more interested in availability and capabilities for lightning-fast querying.
This post covers a traditional case for integration which contains orchestration, batch processing, data mapping, and adapters. I love the power of API-led connectivity when implemented right in an organization, but there are, of course, traditional integration tasks at hand. However, the good news is Mule supports those traditional tasks with its strong roots in the ESB space enabling ESB patterns.
A significant advantage of the Mule runtime engine today is the capability to deliver traditional and modern approaches to integration in one unified solution.
Looking at the image of our solution, it is quite impressive what we achieved:
- Created clean and separated pieces of integration and business functionality with a clear purpose built for reuse (Bounded Contexts)
- Modernized the existing legacy application and enhanced it with a RESTful API abstraction for mobile cases
- Applied the Command Query Responsibility Segregation (CQRS) pattern to allow for high-speed data query and a tailored data format to be easily parseable for the client
- Implemented batch and orchestration capabilities for keeping the legacy and Neo4j databases in sync
Let’s take a quick look at our solution and its design. You can find the running code on GitHub.
As there are supplier parts to be synchronized from a relational database model into a graph model, some timing considerations need to be applied. The order matters for implementing data correctly from the legacy DB to the Neo4j graph database.
The Mule runtime engine helps me abstract processes into smaller reusable modules. For example, the Sync Parts or Sync Suppliers flow and orchestrate them in the way I need them to be executed.
Batch Processing
What would a batch process look like?
Let’s look at the Sync Parts process we referenced above in the orchestration section.
In the Input section, our process queries the legacy DB for the result set of all parts.
In the next batch step, it processes all records contained in the DB result set. It does that by generating the Neo4j merge query based on the record information and sending it out to Neo4j.
The merge query for the synchronization of parts looks like this:
And by implementing a RESTful wrapper for Neo4J and adding it to Anypoint Exchange, we make it reusable by other developers. These developers can build on this template to accelerate any new project on Neo4J and help improve the template when feedback is incorporated.
Conclusion
API-led connectivity is a powerful concept and a helpful tool to implement modern architecture styles like microservices, event-driven and CQRS, or traditional patterns like an ESB or batch. The ability to do both on one runtime is unique to MuleSoft.
Beyond technology, you also need to think about your operational model, which includes people and process. Many of our customers have found that a Center for Excellence (CoE, ICT) will not scale as they become increasingly agile.
To scale with API-led connectivity by aligning IT teams across Central IT and Line of Business IT, we recommend businesses to devise a Center for Enablement that is more focused on self-service and evangelism. Feel free to read more about reorganizing IT teams and embracing digital transformation here. Good luck and have fun with CQRS implementations!
