MuleSoft’s DataWeave, which is now the main expression language in Mule 4 beta, is a powerful technology that can efficiently transform complex structured data between popular data formats, including JSON, XML, CSV, and Java objects. I’m a courseware developer and trainer at MuleSoft, and I’ve been spending a lot of time learning about DataWeave recently.
Update, April 2018
I previously posted the below blog on how to reuse DataWeave code in a Mule flow using the readUrl() function. In the blog, I stated that this was a temporary workaround (or hack) until DataWeave 2 arrived.
With DataWeave 2.0, that support has now been added to allow you to import and export complete DataWeave Modules. This is only supported in the new Mule 4 runtimes, which is the only Mule runtime available in Anypoint Platform Design Center’s flow designer. Here is the product documentation for this new feature.
This is now the preferred way to refactor and reuse your DataWeave code. The main issue with using readUrl() is that it reads the DataWeave code from the file system every time the flow reaches the readUrl() call. With DataWeave Modules, the DataWeave functions are loaded into application memory.
You can still use readUrl() for it’s original purpose, to read files into your DataWeave code, either from the file system, or from a remote URL.
In the latest DataWeave 1.x version – included in the Mule runtime v3.8 –there is limited support for reusing DataWeave functions and variables between Transform Message components (the component that applies DataWeave code).
In this post, you’ll learn a workaround that enables you to separate out reusable DataWeave code, and then reuse that code between multiple Transform Message components and flows––including flows in separate applications and MUnit flows.
Create a Mule project with reusable DataWeave library files
The example Mule project used in this post is available in this GitHub repository. You can import this project into the latest Anypoint Studio to follow along with this post. Alternatively, you can create a new Mule project in Anypoint Studio and copy the examples from this blog post.
Below is a Mule project in Anypoint Studio. It has a folder named dw added to src/main/resources, with a file named myLib.wev. When we export this Mule project as a deployable archive file, any file in src/main/resources will be automatically added to the project’s classes folder; so it will be in the classpath when the Mule application is deployed to Mule Runtime. In this sample project, the HTTP listener is listening for requests on port 8081.
Code the reusable DataWeave file
The myLib.wev file contains some reusable variables and functions.
%dw 1.0
//Reusable constant
%var exchangeRateFromUSDToBritishPounds = 1.35
%function convertPriceFromUSDToBritishPounds ( input )
//Can use the getIntroString() function which is defined further down
"$(getIntroString()) : $(input * exchangeRateFromUSDToBritishPounds)"
%function getIntroString()
"The price converted to British Pounds is"
---
//Provide external names for variables and functions defined in the header
{
exchangeRate : exchangeRateFromUSDToBritishPounds,
convertPrice : convertPriceFromUSDToBritishPounds,
//Use an anonymous lambda to define the function
formatString : (aString, formatter) -> formatter(aString)
}
Read in and use an external DataWeave file
Now ,let us read in this external DataWeave file and use it in a Transform Message component. The first step is to use the new readUrl function to read in the DataWeave function from the classpath. Define a variable myLib as a reference to myLib.wev the DataWeave file.
%dw 1.0
%output application/json
%var myLib = readUrl("classpath://dw/myLib.wev")
In this example we are embedding the DataWeave library file inside the project, so we give the URL relative to the classpath dw/myLib.wev. If you are deploying into a customer-hosted on-prem Mule Runtime, you could also store your DataWeave libraries in a common external location that you add to Mule Runtime’s classpath.
Note that this is a similar technique to the way you can store Mule application properties files or other external libraries (such as JAR files) in an external location. You can learn more about this technique in our Anypoint Platform Operations: Customer-Hosted Runtimes, product documentation, or Tanuki Java Service Wrapper documentation. Depending on the scope of your project, any external assets, including reusable DataWeave files, need to co-exist, be versioned, and evolve according to your project lifecycle’s organization, policies, management systems, repositories, and other tools (such as GIT, Maven, Nexus, and Jenkins). You can learn more about how MuleSoft customers commonly manage and automate MuleSoft project lifecycles (including implementing continuous integration pipelines) in our MuleSoft Development: Advanced training class.
In Anypoint Studio, the file is stored in src/main/resources, but in the deployable archive, all the files from the src/main/resources folder are moved into the classes folder.
Here is a complete Transform Message component in the convertPrice flow:
%dw 1.0
%output application/json
%var myLib = readUrl("classpath://dw/myLib.wev")
//Access functions in the myLib reference
%function printPrice(priceInUSD)
myLib.formatString( myLib.convertPrice(priceInUSD), (price) -> upper price )
---
{
//Format the key
(
myLib.formatString( "result", (aString) -> capitalize aString )
) :
//Format the value - read in the input price as an HTTP query param
printPrice( inboundProperties.'http.query.params'.price )
}
Preview and test DataWeave code, which uses external DataWeave library files
You can preview example data transformations using the Preview pane in the Transform Message component editor. In the Transform Message component, in the left-side Input pane, right click on Inbound Properties > http.query.params, then select Edit Sample Data.
Set an example price:
%dw 1.0
%output application/java
---
{
price: 600
}
In the right-side Output pane, select the Preview button, which opens the Preview pane. In the Preview pane, you should see the result of the body expression.
Change the price from 600 to 500 and verify that the output in the Preview pane changes as well. This shows you that you can preview live changes to DataWeave code, even when you are reading in external DataWeave files.
Here is the entire reuseDataWeaveCode.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking"
xmlns:dw="http://www.mulesoft.org/schema/mule/ee/dw" xmlns:http="http://www.mulesoft.org/schema/mule/http"
xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/ee/dw http://www.mulesoft.org/schema/mule/ee/dw/current/dw.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd">
<http:listener-config name="HTTP_Listener_Configuration"
host="0.0.0.0" port="8081" doc:name="HTTP Listener Configuration" />
<flow name="convertPrice">
<http:listener config-ref="HTTP_Listener_Configuration"
path="/" doc:name="HTTP" />
<dw:transform-message doc:name="Transform Message">
<dw:input-inbound-property doc:sample="sample_data/map_string_string.dwl"
propertyName="http.query.params" />
<dw:set-payload><![CDATA[%dw 1.0
%var myLib = readUrl("classpath://dw/myLib.wev")
%output application/json
%function printPrice(priceInUSD)
myLib.formatString( myLib.convertPrice(priceInUSD), (price) -> upper price )
%function headerString ()
"the price2"
---
{
//format the key
(myLib.formatString( "result", (aString) -> capitalize aString )) :
//Read in the input price as an HTTP query param
printPrice(inboundProperties.'http.query.params'.price)
}]]>
</dw:set-payload>
</dw:transform-message>
<logger level="INFO" doc:name="Logger" message="#[message.payload]" />
</flow>
</mule>
Deploy and test the application
Deploy the reuseDataWeaveCode project to Mule Runtime. For example, here I am deploying the project to an Anypoint Platform account from Anypoint Studio.
In this example, I deployed the application to a public Anypoint Platform URL.
After the application deploys, open a web client (you can use a web browser), and make a GET request to the HTTP listener. If you deployed to Mule Runtime on your local machine, the URL is . For my CloudHub deployment, I’ll make requests from a web client to http://reuse-dataweave-code.cloudhub.io/?price=700 . Simply click this URL now to try it out.
My web browser has a JSON parser extension and shows this response:
Conclusion
As you build up more complex DataWeave transformation for your projects, you’ll want to reuse some of your transformation logic. Today, you can do this using the readUrl() function. Please let us know what you think about this feature, and if there are any additional modularities you’d like to see in future releases.
If you’d like to learn more about how to use DataWeave to perform powerful complex data transformation, you can take the Anypoint Platform Development: Advanced DataWeave course.
To get started, register for our free, self-paced course MuleSoft.U Fundamentals, which includes introductory DataWeave training.
