Content filter pattern for REST service fields filtering

At some point, you may have a service that returns a lot of fields and related objects. What if the service consumer doesn’t want all the fields all the time? In other words, they would like to have the response filtered. This is where the Content Filter comes to the rescue. After you read this article, you will learn how to use it in your own scenarios. So let’s roll!

Content Filter

This is the opposite pattern to Enricher Pattern – more about it you can be found here. In this particular scenario, we want to simplify our response. Remove unnecessary information that the receiver doesn’t need.

Content Filter pattern (source: https://www.enterpriseintegrationpatterns.com/patterns/messaging/ContentFilter.html)
Content Filter pattern.

As you can see in the diagram above, before adding the filter we have a message with three elements and after the filter is applied we have only one element. The magic is happening in the black boxes called Content Filter. However, we as designers should know how this particular element works. So, how to define filter?

We may choose two ways of doing this:

  • Positive filter: We specify what we would like to have in the response.
  • Negative filter: We specify what should be removed from the original payload.

Filtering in Mule 4 using DataWeave 2.2

To filter out some fields, we have DataWeave language. So for filtering purposes, we will use the Transform Message component. We can filter properties from objects and arrays. So let see how to do the positive and negative filtering in DW.

Negative filtering

In DataWeave we have two operators – and –. The first one allows for removing a key-value pair by providing the key.

Filtering using single dash operator in DataWeave
Remove the title property.

This way, I can remove one field at a time. In order to remove two fields, like in the example above, I would need to do it in the following way:

The double dash operator (–) allows providing keys array. In other words, each key present in the array will be removed from the supplied object.

Filtering using double dash operator in DataWeave
Remove both title and id properties.

In the example above, we have provided an array with title and id keys. As a result of applying — operator, we got the empty object.

Positive filtering

This is the opposite scenario in contrast to previously described negative filtering. This time we would like to define which properties should stay, not mentioned ones should be removed from the final outcome.

Positive filtering using filterObject function
filterObject function to filter by custom condition.

We don’t have a similar operator to perform that action. filterObject function is the best match for that case. This function iterates over each key and applies lambda expression. If the condition is evaluated to true, include the field in the output. Otherwise, omit it.

Below you can see a code snippet showing how to do a positive filter. We instruct filterObject to include the key-value pair only if the key is present in the array – here “id.”

Filter fields in your REST service

I have looked for may publicly available services to see some good guidelines to do the filter. However, many of them are too complex or too simple. Here, is some version in between. Below you can see a trait filter definition.

filter trait definition

As you can see, I have decided to introduce the default value *all indicating that all fields will be returned. To distinguish negative and positive filtering, I have decided to introduce – (minus) prefix before fields to remove from the final response. Values are separated by the comma without any additional space.

Warning: We can’t mix the positive and negative filters. I have decided on this rule because it makes sense. Either we want to specify what should be returned or skipped.

Here is the link to the trait filterable. You can place it some common traits library or in a specification API directly.

Use trait in your API specification

In order to use an externally defined trait, you need to include it in the API specification. You will do this like in the snippet below:

As you can see declared filterable trait in traits section. How do you apply this to the resource?

You need to specify the array of traits using is, like in the second line on the snippet below.

Example usage

The first case is the call to retrieve all the fields.

or

The second case is to retrieve only specified fields.

The third case is to retrieve response without the specified fields.

Summary

I think that the presented solution for filtering fields gives some flexibility but isn’t overcomplicated. In many publicly, available APIs providers give the possibility to decide which fields are expected at the response.

If you have any other ideas on how to introduce filtering fill free to share it. Let’s learn from each other.

For more developer resources, visit the tutorials section of our website.



We'd love to hear your opinion on this post


2 Responses to “Content filter pattern for REST service fields filtering”

  1. Nice article. Good to know about this feature. It is applicable to day-to-day dev work. Thanks for sharing this info.

  2. hi when we use positive filtering I am using an array of strings to filter, when there are strings in the array it works fine but when it does not contain any value it will give error. how can i specify if condition for filter Object so that it only works when the array string is not null