At some point, you may have a service that returns a lot of fields and related objects. What if the service consumer doesn’t want all the fields all the time? In other words, they would like to have the response filtered. This is where the Content Filter comes to the rescue. After you read this article, you will learn how to use it in your own scenarios. So let’s roll!
Content Filter
This is the opposite pattern to Enricher Pattern – more about it you can be found here. In this particular scenario, we want to simplify our response. Remove unnecessary information that the receiver doesn’t need.
As you can see in the diagram above, before adding the filter we have a message with three elements and after the filter is applied we have only one element. The magic is happening in the black boxes called Content Filter. However, we as designers should know how this particular element works. So, how to define filter?
We may choose two ways of doing this:
- Positive filter: We specify what we would like to have in the response.
- Negative filter: We specify what should be removed from the original payload.
Filtering in Mule 4 using DataWeave 2.2
To filter out some fields, we have DataWeave language. So for filtering purposes, we will use the Transform Message component. We can filter properties from objects and arrays. So let see how to do the positive and negative filtering in DW.
Negative filtering
In DataWeave we have two operators – and –. The first one allows for removing a key-value pair by providing the key.
This way, I can remove one field at a time. In order to remove two fields, like in the example above, I would need to do it in the following way:
%dw 2.0
output application/json
---
{
"id": "idy4234-2",
"title": "Meetup"
} - "id" - "title"
The double dash operator (–) allows providing keys array. In other words, each key present in the array will be removed from the supplied object.
In the example above, we have provided an array with title and id keys. As a result of applying — operator, we got the empty object.
Positive filtering
This is the opposite scenario in contrast to previously described negative filtering. This time we would like to define which properties should stay, not mentioned ones should be removed from the final outcome.
We don’t have a similar operator to perform that action. filterObject function is the best match for that case. This function iterates over each key and applies lambda expression. If the condition is evaluated to true, include the field in the output. Otherwise, omit it.
Below you can see a code snippet showing how to do a positive filter. We instruct filterObject to include the key-value pair only if the key is present in the array – here “id.”
%dw 2.0
output application/json
---
{
"id": "idy4234-2",
"title": "Meetup"
} filterObject ((value, key) -> ["id"] contains key as String)
Filter fields in your REST service
I have looked for may publicly available services to see some good guidelines to do the filter. However, many of them are too complex or too simple. Here, is some version in between. Below you can see a trait filter definition.
As you can see, I have decided to introduce the default value *all indicating that all fields will be returned. To distinguish negative and positive filtering, I have decided to introduce – (minus) prefix before fields to remove from the final response. Values are separated by the comma without any additional space.
Warning: We can’t mix the positive and negative filters. I have decided on this rule because it makes sense. Either we want to specify what should be returned or skipped.
Here is the link to the trait filterable. You can place it some common traits library or in a specification API directly.
Use trait in your API specification
In order to use an externally defined trait, you need to include it in the API specification. You will do this like in the snippet below:
#%RAML 1.0
title: Consent Experience API
version: v1
baseUri: https://sfdc-consents.profit-online.pl/api/{version}/
mediaType: [ application/json ]
types:
consent: !include types/consent.raml
traits:
filterable: !include traits/filterable.raml
As you can see declared filterable trait in traits section. How do you apply this to the resource?
You need to specify the array of traits using is, like in the second line on the snippet below.
/consents:
is: [filterable]
get:
responses:
200:
body: consent[]
Example usage
The first case is the call to retrieve all the fields.
GET /mongodb/sapi/consents HTTP/1.1 Host: ambassadorpatryk.com
or
GET /mongodb/sapi/consents?filter=*all HTTP/1.1 Host: ambassadorpatryk.com
The second case is to retrieve only specified fields.
GET /mongodb/sapi/consents?filter=id,name,title HTTP/1.1 Host: ambassadorpatryk.com
The third case is to retrieve response without the specified fields.
GET /mongodb/sapi/consents?filter=-details,-id HTTP/1.1 Host: ambassadorpatryk.com
Summary
I think that the presented solution for filtering fields gives some flexibility but isn’t overcomplicated. In many publicly, available APIs providers give the possibility to decide which fields are expected at the response.
If you have any other ideas on how to introduce filtering fill free to share it. Let’s learn from each other.
For more developer resources, visit the tutorials section of our website.
