Part 1: DataWeave 2.0 Syntax Changes in Mule 4 Beta

dataweave mule 4

This post was written by one of the stars in our developer community, Manik Magar 

Mule 4 beta is already out. One of the major change in Mule 4 is making DataWeave a default expression language over Mule 3’s default Mule Expression Language (MEL). The XML namespace of DataWeave is moved from dw to ee (core), and the DataWeave version has changed from 1.0 to 2.0 in Mule 4.

Apart from syntax changes, there are many new features in DataWeave 2.0. In a series of two blog posts, I will focus on the syntax changes in DataWeave 2.0 and compare them to 1.0. 

In this blog post, I will specifically discuss header changes in DataWeave (section 1) as well as iterative function changes within the body (section 2). In my next blog post, I will dive deeper into DataWeave body changes by discussing operator changes as well as other differences.

Changes covered in this post may not be the list of all syntax changes and there may be more changes.
Let’s begin with a quote about change: “Your success in life isn’t based on your ability to simply change. It is based on your ability to change faster than your competition, customers, and business.”

Now let’s get started, a DataWeave script file can be divided into two sections:

  1. Header that defines directives
  2. Body that defines the output

1. DataWeave Header Changes

The DataWeave version has changed from 1.0 to 2.0. Along with that, the list below summarizes changes to the header sections of DataWeave:

  1. dw version changed from %dw 1.0 to %dw 2.0
  2. All directives in DataWeave 1.0 must start with %. This is no longer needed in 2.0 (except for %dw declaration).
  3. Custom functions in DataWeave 1.0 are defined with function directive. In DataWeave 2.0, it has been changed to fun. Also, the function body and function signature must separate by = instead of a space.
 In both dataweave versions, you can define custom functions with either var or function(/fun) directives.

In my honest opinion, all these changes may be trivial but they are good. The removal of %, saves you some characters and makes the header section look more like regular code instead of directives.

The two code listings below show the same script written in both versions:

Listing:1.A – DataWeave 1.0 Headers
Listing:1.B – DataWeave 2.0 Headers
Listing:1.C – Output of Both Scripts

2. DataWeave Body Changes

DataWeave script’s body section is where you write all your transformation logic that defines output structure. Some interesting changes have happened to the body section of the script as well. Let’s look at those changes.

 Below grouping is not an official grouping by MuleSoft but just something I came up with. Makes it easy to write and follow :).

2.1 Iterative Functions

You guessed it right! These are the functions that allow you to iterate and operate on a collection of objects or a single object. This can include functions like map,mapObjectfilterpluckgroupBy, etc. Some interesting changes have been made in DataWeave 2.0 for these functions.

2.1.1 Automatic Coercion from Object to Array

DataWeave 1.0 allowed automatic coercion of object to array. What that means, is in DataWeave 1.0, the map operator which allows you to iterate over array or collection of objects, was also working if the input was an object.

Consider the DataWeave 1.0 script below, where we expected to receive a list of users to map. For some reason, at runtime we received a single user object instead of the collection of users. Why? Because bugs are everywhere!.

Our script should have either failed or generated single user object, but it generates two user objects with some really incorrect data.
Listing:2.1.1.A – DataWeave 1.0 Object to Array Coercion

Listing:2.1.1.B – The Output

Are you wondering why?

Because DataWeave 1.0 treated every property (key-value pair) of an object as a single object and then our input object to be a collection of those objects, like the array in the listing below. Now, if you provide this as an input to map, you are going to get two objects with some null fields.

But was that the expected output from original script? Off course, Not.

Listing:2.1.1.C – Object Coerced to an Array

DataWeave 2.0 has generously corrected this behavior and removed the automatic coercion of objects to array. With, DataWeave 2.0, a similar script will generate an error (in preview mode, as well as at runtime) 

mule 4 dataweave 
Did you notice the error details shown in studio 7? They are so detailed and clear! Yet Another Useful Change!

In my opinion, this change can definitely safeguard users from having unexpected output and data corruption in the downstream systems. 

2.1.2 Overloaded Functions

In DataWeave 2.0, the functions filter and groupBy are overloaded to operate on objects. This could be useful to create sub-objects from a large object.

Consider an object {"firstName": "Manik", "lastName": "Magar", "Address Line1" : "Address line 1"}and we want to create another object with all the properties where value contains ‘Ma’.

With DataWeave 1.0, we will need to use mapObject and then map all key-value pairs conditionally. Something like the listing below:

Listing:2.1.2.A – DataWeave 1.0 Creating Sub-objects
Listing:2.1.1.B – The Output

In DataWeave 2.0, we can leverage the overloaded function of filter which allows us to operate on objects and achieve the same result but with shorter code.

Listing:2.1.1.C – DataWeave 2.0 Creating Sub-objects with a Filter Function

Above one-line and simple code generates exact same output as in <em>Listing:2.1.1.B</em>. To understand the difference, if you write that filter function in DataWeave 1.0, the output will not be an object but an array of values satisfying the given condition.

Listing:2.1.1.D – DataWeave 1.0 Filter Object Output

Similarly, groupByfunction has been overloaded to allow invoking three-argument lambda on object:

Listing:2.1.1.E – DataWeave 2.0 Group by an Object with an Overloaded Function
 <em>Listing 2.1.1.E</em> also gives you a sneak peak at if …​ else …​ that replaces when …​ otherwise …​ conditional syntax of DataWeave 1.0. We will see more about it in a later part.

Running this script against our user object, will partition (group) the object as below –

Listing:2.1.1.F – The Output

 In DataWeave 1.0, the lambda signature is (value, index) so it is not possible to write a similar group by on an object.

If you run groupBy on an object in DataWeave 1.0, it does not output keys which could be another problem.

Listing:2.1.1.G – DataWeave 1.0 Group by an Object
Listing:2.1.1.H – The Output

To achieve an output like <em>Listing:2.1.1.F</em> in DataWeave 1.0, we will need to take an approach similar to that shown in <em>Listing:2.1.2.A</em>.

2.1.3 Adding a Third Parameter

DataWeave 2.0 has added index as a 3rd parameter to mapObjectpluckfilter, and groupBy. Some of these functions also have index in DataWeave 1.0, but as a second parameter.

Consider the two code listings below:

Listing:2.1.3.A – DataWeave 1.0 New Parameter Addition
Listing:2.1.3.B – DataWeave 2.0 New Parameter Addition

Listing 2.1.3 – A vs B

  • The third parameter index has been added to mapObject function. This could be useful if you want to iterate on object properties based on property indices. 
  • There is an addition of a third index parameter to pluck, similar to mapObject
  • The filter function had index parameter in DataWeave 1.0 too, but when an object is passed to filter function, the index moves to the third place and a new second param key is introduced to the lambda. This is a great addition to operate over objects. 
  • There is NO change to filter lambda when input is a collection. 
  • Similar to filter on an object, groupBy for an object also gets key as new second param and index moved to third. 
  • There is NO change to gruopBy when input is a collection.

2.1.4 Allow NULL Input

Things happen, payloads becomes null sometime and when that happens, script starts failing. In DataWeave 1.0, functions will error out due to null input.

In DataWeave 2.0, mapmapObject and filter functions have been modified to accept NULL input and when that happens, they just return NULL.

Consider the DataWeave 1.0 and 2.0 scripts below: 

Listing:2.1.4.A – DataWeave 1.0 Null Input

All of the above mappings in DataWeave 1.0 will fail to compile due to NULL input.

Listing:2.1.4.B – DataWeave 1.0 Null Input

All these mappings in DataWeave 2.0 will not fail but will generate the output below.

 Whether this is acceptable output or not, depends on your use case. You should consider the null input case and handle it appropriately.
Listing:2.1.4.B – Output of DataWeave 2.0
 NULL for map is allowed when lambda is used and returns a single value that can be made null i.e. (T, Number) → Boolean format. For example, consider below mapping that tries to output a key-value pair, but will error out in DataWeave 2.0 too.

And that summarizes DataWeave changes in headers and in the body (regarding iterative functions)! Check out my upcoming post, where I will further discuss DataWeave 2.0 body changes by focusing on operator changes as well as other differences.

This article first appeared on JavaStreets.