Error handling testing with Byteman


exception handling code is not always an easy task. You either need to setup the external conditions that cause the exception to be raised or generate mocks to get the same results.

A third option is using a bytecode injection tool like Byteman. Using scripting language you can insert custom behavior into the application code under test. The main use cases for Byteman are:

Since Byteman 1.5.0 we can use BMUnit, a new packcage that allows easy integration with JUnit. BMUnit takes care of installing the agent on the JVM and loading the rules before running the tests.

An example using Mule

The following example consists of a test to check Mule’s FTP transport behavior when the FTP server replies with 421 error code (Service Not Available). As Mule uses Apache Commons FTPClient to interact with the FTP server, one way to mimic an 421 error is throwing an FTPConnectionClosedException on invocation of the method retrieveFile from FTPClient.

Let’s see what’s going on here in more detail:

  • @RunWith(BMUnitRunner.class) annotation loads the byteman agent and the rules defined by @BMRule/s and @BMScript/s annotations.
  • the test extends AbstractFtpServerTestCase, included in Mule Ftp transport test classes. The superclass takes care of setting up an FTP server besides initializing mule using the configuration specified by getConfigResources(). For more information on Mule Functional Test package you can check Mule documentation.
  • @BMRule allows us to enter the rule definition inline, which is helpful for short rules like this one. For more complex rule you can write them in a separate file and use the @BMScript annotation. Understanding this simple rule is pretty straightforward: when the targetMethod (retriveFile(String, OutputStream)) of targetClass (FTPClient) is invoked, perform action (throw an FTPConnectionClosedException) as long as the condition (true) holds. The condition is mandatory, so if we do not want to specify any in particular, we have to enter the value true.
  • if Mule is behaving properly the underlying transport exception should be wrapped in a subclass of MuleException, as declared by the @Test annotation.
  • the testFtpServiceNotAvailable method adds a file to the ftp server and tries to retrieve it through Mule FTP endpoint.

I’ve just scratched the surface with this example. I hope you find it useful, and in case you have not heard of byteman before you can have a new tool in your kit.


We'd love to hear your opinion on this post

3 Responses to “Error handling testing with Byteman”

  1. Cool blog. Thanks =)

  2. Hi Santiago,

    I’m curious if you were able to get that example to run?

    I had to include this in my test class which extends FunctionalTestCase:

    public void explicitSetupCall() throws Exception
    //Hack to force a call to onSetup. Byteman Unit Runner requires explicit annotation.

    The BMUnitRunner seems to require explicit annotations, e.g. wouldn’t recoginise tests without @Test, and wasn’t recognising the JUnit setup in AbstractMuleTestCase. (and hence wasn’t running the mule context initilisation before trying to execute tests.)

    Let me know if there’s something I’m missing,

  3. Hi Neil,

    The problem you describe seems to be caused by mixing JUnit3 and 4 in the test class hierarchy.
    At that moment I run the example using a Mule 3 snapshot that already had the functional test classes migrated to JUnit4.

    As AbstractFtpServerTestCase had some changes in Mule 3.2 I’m attaching a new gist that works against that version: