This is a guest blog from a member of our developer community. Dr. Roger Butenuth is a Senior Java Consultant at codecentric, he has been using Anypoint Platform for five years, with projects ranging from building simple SOAP routing/transformation to introducing the API-led approach to a Fortune 500 company.
Building Mule applications differs from coding in Java. Instead of typing all your code (with a lot of CTRL+space completion), you have a powerful graphical language. But we should not forget: it’s still programming. The result is a Mule application, but on the conceptual level it’s still a program. What do all experienced programmers know? Untested programs contain bugs, so you need to test them. And as we all know, most programmers like to act lazy, which is why we invented automated test frameworks to avoid redundant work.
So it’s no surprise we find a testing framework called MUnit in Anypoint Platform. It matches the Mule architecture; instead of typing program code, you design your tests with the same graphical language that is used to implement Mule applications.
In this blog, I will share some tips on how I use MUnit for testing, and demonstrate a useful connector I’ve built – the AssertObjectEqualsConnector – that makes a key aspect of MUnit testing even easier.
A little background on MUnit
Frameworks like JUnit and MUnit have been developed for unit tests (as the name suggests), but can be used on the other levels too. This means they can test at several levels:
- Unit: A Java class or method, a Mule flow
- Component: A Java package, a microservice, or a Mule application
- Integration: A complete environment/stage, consisting of all connected systems (Mule, microservices, etc.)
In addition to executing tests and collecting the results, a testing framework has other responsibilities:
- It has to provide mocks for components, which should not be executed during the test. This is especially needed for unit and component tests.
- The result delivered by the code under test has to be checked, usually with assertions.
- For mocked parts and for whitebox testing, you may want to spy into your data and check it with assertions.
MUnit supports all of these aspects, so testing is easy and there is no excuse for not testing your flows. For the rest of this blog, I will concentrate on assertions, which are not only the most crucial point in tests, but sometimes the most difficult one.
Example Application
To illustrate my point, I will use a simple example application implementing a REST API consisting of one HTTP GET request with two parameters: weight and size of a person. It calls an external SOAP service to compute the Body Mass Index (BMI) of that person. The BMI and a description (e.g. Normal/healthy weight) are returned as a JSON object.
Additionally, the BMI and the current date are sent to a queue (again, as a JSON object). For the sake of simplicity, this is only a VM queue. In a real application, this would be something else––such as a JMS queue or a Kafka topic.
Most of the application is generated from the RAML API. The core functionality is implemented in the following flow, which will be called by the APIkit Router (You find the complete example on GitHub):
The first DataWeave component creates the SOAP requests and fills it with the flow variables provided by the APIkit router:
%dw 1.0
%output application/xml
%namespace bmi http://codecentric.de/calculate-bmi/
---
{
bmi#CalculateBmiRequest: {
bmi#length: flowVars.size as :number,
bmi#weight: flowVars.weight as :number
}
}
The SOAP response is parsed with another DataWeave component and stored as a Java object (Map), so it can be consumed several times without any streaming issues:
%dw 1.0
%output application/java
%namespace bmi http://codecentric.de/calculate-bmi/
---
{
bmi: payload.bmi#CalculateBmiResponse.bmi#bmi,
description: payload.bmi#CalculateBmiResponse.bmi#comment
}
The next message enricher feeds the queue, while preserving the original payload. The last DataWeave produces the HTTP response (JSON).
Unit testing with mock
You may ask: “why should I write a test for such a simple application?” As a developer, I can use the preview of DataWeave to check the result of the first transformation:
That may be good enough for a simple transformation, but the real world is more complex. Flow control and error checking mechanisms such as when / otherwise, null checks, and default values create many paths within a transformation, which should be tested with different inputs.
Many people start with a first test, generated by a right click on the APIkit router:
Everything is already there, all we have to do is adjust the values of the input parameters and the file content of the expected JSON.
Are we ready? Not really. We have created an integration test, not a unit test because the flow under test depends on an external web service. You can write these kind of tests with MUnit, but they should be the second step.
First, you should write a unit test where you mock the web service call. In most cases, mocking an external system leaves a gap in your test. You are not checking the message where it enters the connector to the external system. This can be achieved with a spy, resulting in the following test:
Making the test less brittle
Is this a good test? What happens if a new version of DataWeave changes the formatting of the JSON? What happens if somebody changes the order of map elements? What happens if the namespace tag is changed from “bmi” to “ns0”? All of these changes break the test. But the code is still correct!
The problem is this: comparing JSON or XML character by character is too strict. The comparison should happen on the object- or content-level, but not on character-level. Can you achieve this with MUnit? Sure, but with a little bit of work.
Instead of using Assert Payload on JSON/XML, you can parse it with DataWeave to Java and check each attribute with Assert Equals.
The DataWeave transformations produce Java and consist of a simple “payload” as an expression. The asserts compare the given value with a MEL expression, e.g. “#[payload.bmi].” Unfortunately this does not work within the spy because within a spy we are not allowed to modify the payload.
Now we have a stable test that is independent of formatting, order of attributes, and namespace tags. The only drawback is its verbosity. You need an assert node for every attribute, which results in large tests for JSON/XML that consist of more than a handful of attributes.
Can we do better? Of course –– use the extendability of Anypoint Platform!
DevKit to the rescue
If you want to connect to a system which MuleSoft does not provide a connector for, then you can simply write your own using DevKit. Here, we don’t want to “connect” to another system, but we do want a simpler way to compare JSON or XML. We can use DevKit as an elegant and easy tool to implement an adapter between a Mule flow (or MUnit test) and some Java logic implementing the comparison. I’ve done exactly that with the AssertObjectEqualsConnector, it is available on GitHub for the community.
For XML, there is no need to reinvent the wheel. The open source library XMLUnit does everything we need––XML comparison on a semantic level. The “strictness” of the comparison is configurable, the connector I have written supports three levels. For details, see the documentation on the GitHub page of the AssertObjectEqualsConnector. For JSON, I didn’t find a suitable library, so I implemented a comparison library by myself.
In the following section, I will skip the implementation details and focus on showing some usage examples. To use the connector in Anypoint Studio, just install it by adding the update site linked on Github. The connector is on Maven Central, so you use it in a mavenized project. The dependency is added by Anypoint Studio automatically. Comparing JSON is configured like this:
The expected JSON value and the actual value are provided by MEL expression, like in a simple Assert Equals. However, formatting (e.g. whitespace, newlines, etc.) and the order of elements in a map no longer matter. If you want to check the order in a map, there is an option for that; but you shouldn’t have to do so when you use JSON in a reasonable way.
If you want to make sure the result contains some values, but still ignores additional values, then just set the flag “contains only on maps.” That way, additional properties in the actual value don’t break the test.
These two flags work on a global level. With path patterns, you can configure this only for part of the object tree (You can find more details on the Github page). Here, I will show one example, in which the JSON sent through the queue contains a date, generated by this DataWeave in front of the VM endpoint:
%dw 1.0
%output application/json
---
{
bmi: payload.bmi,
date: now as :string { format: "yyyy-MM-dd" }
}
The date will change the next day, so it has to be excluded from the test. In this simple example, we could do so with the “contains only” policy. Another approach is to set the flag “ignore” on that property. In the MUnit XML that looks like this:
<assert-object-equals:path-options> <assert-object-equals:path-option>['date'] ignore</assert-object-equals:path-option> </assert-object-equals:path-options>
Each option entry starts with a path specification, followed by one of the flags “ignore,” “contains_only_on_maps,” or “check_map_order.” The setting is valid for this property and the whole tree below; in case the value is an object, not just a literal.
A path specification can be more complex, e.g. “[‘addresses’][#][‘street’].” Strings are regular expressions that match property names, numbers match list entries, and # matches any list entry. Here, we first select the property “addresses,” whose value is a list (matched by #), which contains objects, where the property “street” is selected. For more information, check out the GitHub repo!
Conclusion
MUnit tests are a good fit within the Mule toolset; you can use the same graphical technique for designing flows and tests. Mocks are quite important if you don’t want to depend on external systems. Within spies, you can assert your assumptions to ensure flow internals are as you expect. Assertions on XML and JSON objects can lead to brittle tests or let the size of your test explode when you do it with the given tools. Fortunately, DevKit allows users to implement their own extension––making the entire process much easier.
