Reading Time: 16 minutes

Overview

Once you have a working Mule ESB application you may be wondering how fast it can run. Here we will discuss a simple method for measuring the throughput of your application using Apache JMeter.

Bear in mind there are many ways to improve performance (simple changes can yield great performance boosts). We will explore them in greater detail in a follow-up blog post covering Mule application tuning.

Prerequisites

  • Mule ESB 3 must be installed (using a clean installation to begin with is probably a good idea).
  • JMeter must be installed. Latest version, if possible. You can get it at http://jakarta.apache.org/jmeter/
  • Have an application to test which uses an inbound HTTP endpoint and a request-response exchange pattern (for simplicity, we will use one of the examples bundled with Mule ESB).
    • Even if in this case we will use HTTP, you can choose from several different transports supported by both Mule ESB and JMeter.
    • Among other transports, you can also use JMS, JDBC, etc.
latest report
Learn why we are the Leaders in API management and iPaaS
Read reports

Set up

For more accurate results, you can use two systems. One to run Mule (“server” from now on) and another one to consume your application’s services (“client” from now on). However, if you are willing to trade some accuracy for simplicity , you can run both client and server on the same machine.

Server Setup

  1. Install Mule
  2. Start Mule.
  3. Install application to be tested.
    • In this case, we will use the “Loan Broker” example application that comes bundled with the Mule distribution as our application to be tested.
    • To install, just copy $MULE_HOME/examples/loanbroker-simple/mule-example-loanbroker-simple-*.zip to $MULE_HOME/apps
    • Mule will extract the zip file and hot-deploy the application on its own. This usually takes less than 15 seconds.
    • Open http://localhost:11081 in your favorite browser, to check the app is running.

Client Setup

  1. Check connectivity with the application running on the server.
    • If the client machine is the same as the server, open http://localhost:11081 in your favorite browser.
    • If client and server are on different machines, use the same URL as above, but replacing localhost with the server’s ip address.
  2. Install JMeter.
  3. Run JMeter.
  4. Create a test plan (as follows) or load a pre-existing one.

Creating a Simple Test Plan

We will use JMeter to generate a high load on the application. This will consist on the following general steps:

  • A certain number of threads will concurrently submit requests to an HTTP endpoint and wait for a response from the Mule application.
  • Every time a thread receives a response form Mule, it will record some data and repeat the whole cycle a certain number of times.
  • With that log data, the throughput will be calculated.

Let’s implement this test plan in JMeter…

  1. Add a thread group to the test plan. This will create threads to call the application.
    1. Right click on “Test Plan”, then left click on “Add”, “Threads (Users)”, “Thread Group”.
    2. Set the thread group name to something suitable, like the name of the app. In this case, “Loanbroker Simple”.
    3. Set the action to be taken after a sampler error to “Continue”. This will keep the test going if an error is detected, taking note of what happened.
    4. Set the thread properties to values you may expect in production.
      1. Set Number of Threads (users) to 20. This means 20 threads will be simultaneously consuming the application.
      2. Set Ramp-Up Period to 0. This means all threads will be created immediately when the test starts.
      3. Set Loop Count to a suitable number that will make the test run in a reasonable amount of time.
        • In this case, a suitable number would be 25.
        • Bear in mind that the total number of requests sent will be the number of threads times the loop count. In this case, 5000 requests.
      4. Leave “Forever” unchecked.
        • Otherwise an infinite number of loops will run until you stop it manually using “Stop” from the “Run” menu.
      5. Leave  “Scheduler” unchecked.
    5. Your configuration so far should look like this…
  2. Set up the HTTP request
    1. Right Click on the  Thread Group you just created and then left click on “Add”, “Sampler”, “HTTP Request”.
    2. You can leave the HTTP Request name to its default value, or change it if you want to.
    3. Set up the Web Server we will point to.
      1. Set Server Name or IP to the server machine’s ip. If using the same machine as server and client, use “localhost”.
      2. Set Port Number to 11081. This is the port on which the “Loanbroker Simple” application will be receiving our requests.
    4. Leave timeouts blank.
    5. Set HTTP request parameters
      1. Leave protocol blank (it will default to http).
      2. Set method to “GET”.
      3. Leave Content Encoding blank.
      4. Set path to /
      5. We will not be sending any Parameters with the request in this case. If we wanted to, we would just press “Add” and complete the necessary information.
      6. Leave all other parameters to their default values.
  3. Add a Summary Report to our test plan, which will provide an overview of the results.
    1. Right Click on the  Test Plan and then left click on “Add”, “Listener”, “Summary Report”.
    2. You can leave the Summary Report’s name as it is or change it if you want to.
  4. Save your test plan and you’re done.

Testing the Performance of your Application

To test the performance of your application, follow the next steps:

  1. Check initial preconditions
    • Mule has just been started on the server.
    • Preferably, the only application running on Mule is the one you want to test.
    • The client has JMeter open with the configuration we just wrote loaded and with the “Summary Report” open (with data from zero samples).
  2. Be ready to take a look at the “Throughput” column of the Summary report.
  3. “Warm up” the server with the first run.
    1. Click “Run” menu and then “Start”.
    2. The throughput number should increase and approach almost asymptotically to a certain value.
    3. The error rate should be zero (or near zero, depending on the behavior of your application).
    4. The throughput will not begin at top speed since the server is “cold” and resources such as memory, connections, etc. were not allocated initially to support the full load.
    5. Once the throughput is relatively steady, we can consider the server is “warmed up”.
    6. The number of samples (loops times threads) should be high enough to let you reach a relatively steady throughput.
      • If a steady speed was not achieved, follow these steps to retest with more loops:
        1. Reset JMeter data by going to the “Run” menu and clicking on “Clear All”.
        2. Increase the loop count of your threading profile. Save the JMeter test plan.
        3. Restart mule.
        4. Go back to the first step of the “Warm Up” Process.
    7. We will discard this first throughput number, since the “Warm Up” phase is used to reach a relatively steady state and not to take measurements. Do NOT restart Mule.
  4. Execute more runs (at least three) to get more data points. For each run, follow these steps:
    1. Reset JMeter data by going to the “Run” menu and clicking on “Clear All”.
    2. Click “Run” menu and then “Start”.
    3. As the test runs, this time the throughput should remain relatively steady. If it changes too much it means something is wrong.
    4. When the test has finished, be sure to write down the final throughput.
  5. Post-process the data (you may want to use a spreadsheet for this one)
    1. By now you should have at least three throughput numbers.
    2. The average throughput of your system will be the average of these numbers.
    3. You should also compute the standard deviation of the throughputs, to validate your test results.
      • If the standard deviation is small compared to the final throughput (say, less than 15%), that means the throughput numbers were in the same vicinity, validating the final result you just computed.
      • If the standard deviation is large compared to the final throughput (say, more than 15%), that means the throughput numbers were scattered too much, invalidating your final result. This means you will have to start the test all over again, or examine any interference that may have happened during the test.
  6. Now you have the average throughput of your Mule application. Remember, this number should always be interpreted along with the hardware, OS, JVM, Mule config, etc. that was used to produce it.

Ready for more?

Now you know how to set up a simple performance test. Feel free to play around with JMeter and Mule. Here are some ideas to try out…

  • Change the number of threads consuming your app and see how it affects the throughput / error rate.
  • Change the loop count and see if the throughput remains steady.
  • Use a tool like UN*X “top” to see the change in memory consumption of the java process running Mule ESB.
    • Is it steady or is it rising steadily?
    • If it keeps rising, there may be a memory leak.
  • Try sending larger requests.
  • Try other transports, such as JMS, JDBC, etc.
  • Increase/decrease the amount of JVM memory for the Mule process and see how it affects the throughput.

Final thoughts

Now you know the basics of measuring the performance of a Mule ESB application. Chances are that your initial throughput can be improved greatly by making some simple changes to your configuration.

Stay tuned, as we will go deeper into “Mule performance tuning” in future blog posts.