All hail DataWeave!
One of the major changes in Mule 4 is the introduction of DataWeave as our primary expression language. Although this may seem like a radical change, I’ll explain some of the reasons behind our decision, and why this represents a major leap forward. I’ll also share some examples and answer a question that is likely on many readers’ minds: “what about MEL?”
Why We Chose DataWeave
Let us start with an example. A few weeks ago, I decided to create a Slack app to check the status of our tests on Jenkins. This meant I had to integrate Slack and Jenkins APIs to create a personalized experience for our team – exactly what MuleSoft is all about. After reading some docs on each API, I decided to start building the app:
- First, I used an HTTP listener to receive Slack commands, and a choice router that decides which action should be taken based on the input; in this case, I either retrieve the test status or explain how to use the command
- Then, I created my main logic to retrieve the test data from Jenkins, returning XML data that I transformed to JSON, as required by Slack’s API
Something became clear very quickly: all my routing logic required MEL and all my transformation logic required DataWeave. But, why? Why was I forced to use and learn two languages when DataWeave is powerful enough to handle all of it?
Well, the answer to that is just timing. When MEL was introduced, Mule was very Java-oriented. Up to that point, there were a variety of evaluators to handle different inputs, like Groovy and JSON. As a result, MEL was created to build a consistent experience when dealing with these expressions.
On the other hand, transformations were considered separate; for the most part, only transformers and DataMapper were used at the time. Yet, our customers still needed to handle more complex transformations with high performance, which is why we introduced DataWeave in 2015.
With a highly performing transformation engine and rich querying capabilities, DataWeave became a hit. So, there we were, with two languages inside our platform, and one so powerful that the other featured a function to call it: dw().
So we asked ourselves: “wouldn’t it make more sense to just have one experience for building expressions, querying, and transforming data?” Using DataWeave would mean leveraging all of its power:
- In Mule 3, you have to transform everything to Java objects to evaluate any expressions (e.g. when routing payloads or logging data). You also have to learn the specifics of each transformer. But, with DataWeave as the expression language, you could simply query the data directly and forget about those transformations.
- With DataWeave, our expressions could be focused on the structure of our data, rather than its format. This is because a Java array is the same as a JSON one in DataWeave – we don’t need different expressions to handle them.
- Access to binary data could be done anywhere you need it and, thanks to some exceptional streaming improvements, you can get larger than memory, random, and repeatable access!
That is why Mule 4 unifies expressions and transformations with a single, modern, consolidated language that has performance at its core, making all of the above a reality.
How DataWeave Revolutionizes Mule 4
DataWeave now provides a service that is used by the Mule Runtime engine to evaluate expressions. The runtime, in turn, gives DataWeave all data regarding the current execution, including payload, variables, expected output, and metadata. This enables DataWeave to know, for example, whether a String or a Map is required, how to handle each variable, whether to coerce a type, etc. Then, one can write expressions as in the example below:
#[payload ++ variables.myStatus] #[attributes.statusCode]
In this example, the payload, variables, and attributes keywords will be interpreted as such.
Now, many may wonder how this one liner DataWeave expression actually works, especially since DataWeave requires users to declare the output format. The output type is inferred, when possible, but you can also add it to that one liner.
In the example below, we use a JSON payload to set an HTTP request’s headers, taking that existing map of headers and adding one to it with the expression:
#[output application/java --- payload ++ { host : 'httpbin.org' }]
The backend will answer with the received headers, which contain the values sent to our HTTP listener as body and the host one we added.
Full Integration
So far we have only talked about DataWeave expressions as one liners for routers and simple attributes. Another simplification we created is for flows. By allowing users to define content “inline,” we decreased the number of transform elements needed. For example, you can build the content of a file inside the File connector’s ‘write’ component, there is no need to use a ‘transform’ component in order to get the payload you need beforehand.
In the example above, you do not need additional steps to iterate the received JSON payload; and the new file path is determined with the expression:
#[payload.name ++ '.' ++ dataType.mimeType.subType]
Also, we add a desired “date” attribute within the write operation, exactly where it’s needed, setting the content to:
#[payload ++ { date : now() }]
That last expression is a great example of the output type being inferred. Since we know the payload is JSON, there’s no need to specify the output and the generated files will be in that format as well.
This works for all new connectors since it’s supported by our new Mule SDK. In the example below, the HTTP body of a request is set with a script, where I could take advantage of all of DataWeave’s features––as in any transform component script:
#[
%dw 2.0
output application/json
---
payload ++ {location : 'LATAM', office : 'BA'}
]
Additionally, we can configure the listener response body with the expression:
#[payload.data]
This is because the backend server will return a JSON, where that attribute represents the payload sent to it. So, the data received by the listener will be modified to include some more attributes, and later forwarded back.
Notice in the above example that the data received in the HTTP listener is actually application/x-www-form-urlencoded. DataWeave is now handling that out-of-the-box––no parsing needed. Even if I had sent a JSON payload, the result would have been the same. This is because DataWeave allows us to focus on the data’s structure, rather than its format.
Compatibility
But enough about DataWeave, let us address the elephant in the post: Mule 3 compatibility and MEL. Well, MEL is still around, it’s just deprecated. Although DataWeave is the primary and default expression language, every expression can feature the “mel:” prefix to indicate that it should be evaluated with MEL.
In the example below, the HTTP listener will answer whether a variable starts with “SUCCESS” or not, using MEL to configure the response body with the following expression:
#[mel:flowVars.receivedMessage.startsWith('SUCCESS').toString()]
This should help Mule 3 users adapt more easily, but, if I were you, I’d make the DataWeave jump as soon as possible to start taking advantage of all its great features!
A little more on DataWeave
Extensive efforts were made by the DataWeave team on the language version 2.0. The team not only supported the runtime integration, but also worked to improve the language itself. I’ll leave it to them to tell you more about that in a future post, but here’s a sneak peek:
- Imports and modules: you can now package and import scripts into others, enabling you to reuse and share your code.
- Java interoperability: static methods can be executed through DataWeave.
- New data formats: we added text/plain, application/x-www-form-urlencoded, and multipart support.
- Type system: you can indicate the type for functions and variables with type inference helping you out.
- Simplified syntax: everything is a function and if/else clauses are now the norm.
And so many other features, we will need another dedicated blog post!
Lastly, I wrote that Slack/Jenkins app in 3.8.4, after I was working on the development of Mule 4 for a while. Not only did I experience the complexity of the process first-hand, but I also found myself wishing Mule 4 was out so I can start building apps there! So I hope these glimpses make you as excited about Mule 4 as we are!
Ready to get started? Download the Mule 4 beta today.
