As an integration framework and broker, Mule ESB is the platform of choice for implementing and deploying enterprise integration solutions and related services. In this series of posts, I’m going to look at situations beyond the traditional integration scenarios where using Mule ESB has enabled the implementation of effective and elegant solutions.
In this first installment, I will talk about using Mule ESB as a frontal load-throttling middleware for applications that are not able to handle sudden peaks of requests, as illustrated in the following diagram.
In this scenario, burst of requests are generated by clients and need to be processed by an application. These requests either do not expect any response, like TCP packets sent over a socket one after the other, or expect a simple response that is not dependent of their actual processing, like a basic acknowledgement. The application is unfortunately not designed to perform asynchronous processing: it is designed to process messages synchronously, as they come. Hence the need for load throttling.
Implementing such a load-throttler is incredibly easy with Mule and this because Mule, at its very core, follows the design principles of SEDA. The Staged Event-Driven Architecture, or SEDA, “refers to an approach to software design that decomposes a complex, event-driven application into a set of stages connected by queues” (Wikipedia). Such a design allows an application to degrade gracefully under load, as queues get filled up and consumed by workers whenever they regain the capacity to do so.
Let’s look at a simple example where the target application accepts messages over HTTP and takes one second to process each request. Let’s use curl and a simple loop to confirm that it behaves as expected:
Indeed, around ten seconds have been spent performing these ten HTTP POST operations on the slow application
In Mule, let’s now configure a simple HTTP bridge service, as shown here after:
In this bridge, the inbound and outbound stages are decoupled, as per SEDA’s design. Hence the fact that communicating with the outbound application is a slow operation will not affect the capacity of the inbound phase to accept messages rapidly. Let’s confirm this with running the same command line operation, but this time targeting the throttler HTTP service instead of the application:
Now we’re talking! In 300 milliseconds, Mule has accepted all the incoming POST requests and started to dispatch to the slow application in parallel. But what would happen if Mule would crash while still dispatching messages to the slow application? With the current configuration: messages will be lost. If this is not an option, we can add an intermediate persistent VM queue between the HTTP inbound and outbound endpoints in order to gain durability of the messages pending dispatching. Here is how we would do so:
If you re-run the timed command line to send messages, you’ll notice that adding the VM queue intermediary doesn’t affect the overall performance of the solution.
Before we deem the mission as being accomplished, we must be very clear on one thing: the order of messages will not be guaranteed with this solution. If you trace the requests that hit the application, you’ll notice that the overall order is respected in the sense that request #9 arrives generally after request #1, but request #1 and #2 will arrive in disorder. In software, like in life, you can’t have it all: by getting lax on accepting and dispatching requests, we gained performance but lost ordering…
Moreover, there are still load limits we can hit if the peak of incoming requests is high enough to saturate the buffer used to accumulate them. In that case, we could gain further scalability by rolling out additional Mule instances and have a network load balancer spread the load on them. Our slow application will remain oblivious of all the upfront traffic intensity, leaving to Mule the responsibility of sweating it out!
Of course, what have been demonstrated with HTTP in this example could be done with other Mule transports. So have fun and don’t be scared of peak times anymore!
