Agent-Based Synchronous HTTP Request Handling (A Recipe)

motif

In the vast majority of cases, requests are processed synchronously: the operation that the client wants to perform on the targeted resource is executed by the same thread and the result is returned right away. This is usually done by connecting the layer directly to the service layer.

This post demonstrates a slightly different approach where HTTP requests are first sent to a messaging layer, then processed by dedicated agents whose responses are eventually returned synchronously to the client that is blocked waiting.

Some typical use cases for this agent-based approach include:

  • Situations where the number of threads able to concurrently process requests is less than the number of threads potentially coming from the web tier,
  • Multiple Mule application deployments where a single HTTP entry point is exposed and processing agents are available in different applications.
  • Bridging from the DMZ to your private LAN where /AMQP is used for internal communication.

The approach demonstrated in this blog is intended for processing requests that do not take a long time to process. Requests that trigger long running processes are better handled with a completely asynchronous approach (ie. disconnect request submission from result deliverance).

As you will see in the coming examples, the messaging layer we use is JMS. These examples would work the same with AMQP. We will start with a very basic single-agent setup and evolve to a more versatile one.

Single-agent service

The first example we’re going to look at consists in a simple service that performs a single operation: capitalizing all the words of a sentence. Here is the relevant snippet (the full is available at the end of this post):

Let’s try this service:

This promises to be a hit! If we take a closer look at the configuration we notice that:

  • A request-response bridge is used to synchronously connect an inbound HTTP endpoint to an outbound JMS one,
  • A pair of transformers take care of transforming the payload from its initial byte stream form to a String and also of sanitizing all incoming HTTP headers so they don’t conflict with JMS naming conventions,
  • Mule takes care of wiring everything together, performing synchronous requests over JMS via a temporary reply-to queue.

With this configuration in place, we can configure the number of available HTTP threads independently from the number of  JMS consumers.

Now what would happen if we want to roll-out more similarly awesome services? A naive approach would consist in adding a pair of {bridge, agent} for each new service. In the next section, we’ll see a better way.

Dynamic-agent service

The main idea is to use a single upfront bridge that can send request to a different JMS queue, hence a different actor, based on the URI used by the client that performed the HTTP call. This can easily be achieved by using a Mule expression, as shown here:

The somewhat cryptic Groovy expression takes care of turning request path into queue names, ie transforming:

into:

With this new bridge in place, the previous capitalizer agent still works as-is but is now accessible under a different path:

We can then roll-out a new and equally crucial agent:

This new agent  is automatically accessible, but from a different sub-path:

This would work the same if the frontal bridge and each agent would be in different Mule applications, effectively exposing all the agents behind a single HTTP endpoint. An advantage of this architecture is that each application can have a different life-cycle, for example allowing the hot replacement of agents.

Try it out!

The recipes in this article are intended to water your mouth and show you what’s possible do with a few lines of configuration in Mule.

The complete configuration, with both services in it, is available here and can run be run with Mule ESB 3.2 CE.

 


We'd love to hear your opinion on this post


4 Responses to “Agent-Based Synchronous HTTP Request Handling (A Recipe)”

  1. Why not use jetty’s Continuations which handles requests asynchronously? I have mimicked Micorosoft’s DirectPush technology using jetty’s Continuations and works perfectly.
    I have used it to control Apple’s iPone/iPad remotely using Provisioning command of ActiveSync protocol.

    Wrote article and there is source code available.

    http://www.enterpriseios.com/story/2011/06/14/Open_Source_iOS_Device_Management

  2. This is exactly what i did in my last project. It’s not only good for keeping different life-cycles it’s also a way to achieve path based mule apps where you don’t have to take a new port for every app with an http interface.
    Nice post David!

  3. @Haruhiko: Jetty’s continuations could be used with the above solution, as it would only take care of the web-tier threading model. All what’s downstream would remain similar.

    @Tomas: Thank you for sharing your experience!

  4. Hi David,

    Thank you for your comment.

    If the server will hold request/response for every connected client, it would need the same number of thread as the number of connection it is holding. As per jetty’s document,
    If there are 1000 simultaneous connections, there will be 1000 threads to handle which will consume over 256MB of memory.

    However, when jetty’s Continuations is used, the number of threads needed is governed by the time to generate each response. So the same 1000 could be only handled by 1 thread. That said, If the task performed downstream takes much time to generate the responses, combining jetty’s Continuations with the solution above can reduce memory consumption, I believe.