Introducing the Validations Module

This all began with a very popular request: “We want to be able to throw an Exception from a flow”. The motivation for this is that it’s fairly common to run into “business ” (errors not related to the handling and transmission of data but the data itself) which should actually be treated in the same way as a connection or system error (things like the remote endpoint is down).

Given the popularity of the request we decided to look into it and started by asking: “which are the use cases in which you would throw an ?”. 100% of the answers were: “because the data was not valid after applying some X criteria”. So, instead of doing a very raw <throw-exception /> message processor we decided to build a validations module, which throws exceptions when a fails.

Do we really need this?

It’s a very common use case when doing integrations to need a mechanism to test some conditions are met and throw an exception if validation failed. Mule currently provides some mechanisms that can be used for validations but none of them completely fulfils the use case:

Filters

Filters can stop execution of a flow if a condition is not met, however they’re not a good fit for this because:

  • Filters are conceptually conceived to just kill event’s processing. They can throw exceptions, but you cannot customise the exception type. So if you have two filters in the same flow you cannot know which one failed. They don’t throw an exception or provide any mechanism to take action on that error.
  • Filters often require you to write a MEL expression for each validation. Not an out of the box solution.
  • At the end of the day, reality if that even though filters can sometimes be used as validators, they’re not conceived for that.

Choice + scripting component

Another option is to use a choice element to evaluate the condition and then throw an exception using groovy or other expression evaluator. This solution is quite verbose and overly complex.

  • You have to evaluate the condition through a custom expression
  • You need custom code to actually throw the exception
  • The choice element forces you to an unnecessary <otherwise> clause

Availability

This feature is available starting with Mule ESB 3.7.0, in both Community and Enterprise editions. However, Studio support only became available starting with the September 2015 release.

Validation Basics

The validations module was designed under the following principles:

If validation fails, a ValidationException will be thrown.

  • By default, this exception will have a meaningful message attached. You can optionally customize this message and change the type of exception thrown, as long as the new exception type has a constructor that overrides Exception(String).
  • For cases in which you want to throw an Exception without such a constructor, or which creation is not trivial, or which is to contain additional information, there will be a factory interface you can implement to create that exception

Ways of accessing a validator

In most cases, there’re two ways of using a validator: through a MessageProcessor or through a MEL function.

Through a MessageProcessor

We provide a message processor for each of the validators. This is what you use when you want to perform the validations as part of your flow and you want an exception to be thrown if a validation fails.

If you like to build your flows using the visual designer in Anypoint Studio, then the validators are exposed as a single item in the components palette:

Validators

 

You can then select the validation type:

Screen Shot 2015-08-28 at 3.26.55 PM

 

The parameters of the validator you selected are then shown:

Validation screen shot 3

 

If you prefer to get your hands dirty and write the XML manually, then each validator is a different message processor in the validation namespace:

Could not embed GitHub Gist 074dde2b5a573e2cd08a: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

A nice thing about the message processors, is that they all accept expressions in all their parameters.

Through MEL

The purpose of exposing validators through MEL is to use them in decision making. Unlike the message processor version, these will be boolean functions without an output message, just true if the validation succeeded, or false otherwise. All the validators are available through a validator variable automatically added into the MEL context.

Could not embed GitHub Gist 0db7bcbedfd1200bde6e: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate Email address

Checks that a given email is valid.

From XML:

Could not embed GitHub Gist 4c22f4bce11724a3c78e: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

It can also be used through MEL:

Could not embed GitHub Gist 4b3c9554b0b36167e3ae: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate using a regular expression

Validates that a given expression matches a Java regular expression.

Could not embed GitHub Gist 0b7aba01c23acdae77a7: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

The caseSensitive parameter is optional and defaults to true.

It can also be used through MEL:

Could not embed GitHub Gist 9b0b9798f6fd3b587a02: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate a string is a valid time according to a pattern

Could not embed GitHub Gist b348c564e127503cc4c6: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Pattern and locale are optional arguments.

  • Locale defaults to the system’s locale
  • Pattern defaults to the locale’s default pattern.

This same validator can also be used to process a timeless date:

Could not embed GitHub Gist 97a0c39a9c563c3a9b12: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Also available through MEL in various overloaded forms:

Could not embed GitHub Gist de1219b88bee651e98aa: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate String, Collection or Map is not empty

In the case of a String, the definition of not empty is that length is greater than zero and it’s not all whitespace characters. In the case of a Collection or Map, it refers to how many items are contained.

Could not embed GitHub Gist e2cc8737e25ce97165c6: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Also available through MEL:

Could not embed GitHub Gist 5e3758352c33a555ff73: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate String, Collection or Map is empty

The exact opposite of the not empty validator

Could not embed GitHub Gist 07b7a3357a8607f7a1d5: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate size

Validates that the input’s size is between a min and max boundaries. This is valid for inputs of type String, Collection, Map and arrays (in the case of the String, the size is actually its length).

Could not embed GitHub Gist b726d5c89e6940085e6f: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens
  • min is optional and defaults to zero, which in practice means that a blank String is accepted. This number must be in the integer range
  • max is also optional and defaults to null, which in practice means that no upper bound is enforced. This number must be in the integer range

Also available through MEL:

Could not embed GitHub Gist 8e793389c1069d416246: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate not null

Fails if value is null or an instance of NullPayload

Could not embed GitHub Gist 66829ae30322b71d1a37: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Also available through MEL:

Could not embed GitHub Gist ae864e8e3409fa7860b1: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate null

Fails if value is not null and not an instance of NullPayload

Could not embed GitHub Gist bb8ea2afe0b758e768ac: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Also available through MEL:

Could not embed GitHub Gist 436423c04ca47d5d30f2: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate a String can be transformed to a number

Could not embed GitHub Gist 2e0821b68c1d0a086f24: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

The processors above validates that a String can be parsed as a number of a certain type. minValue and maxValue are optional and allow to check that, if valid, the parsed number is between certain inclusive boundaries. If not provided, then those bounds are not applied.

The valid options for the numberType attribute are:

  • INTEGER
  • LONG
  • DOUBLE
  • SHORT
  • FLOAT

It is also possible to specify a pattern and a locale to perform the validation.

  • Locale defaults to the system locale.
  • Pattern defaults to the locale’s default pattern.

Full form of this validator would be:

Could not embed GitHub Gist 61a48794729d6a187bd5: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Also available through MEL:

Could not embed GitHub Gist 72effdb9d64ad9e15d38: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate IP address

Checks that a given ip address is valid. It supports both IPV4 and IPV6. In the case of IPV6, both full and collapsed addresses are supported, but addresses containing ports are not.

Also available through MEL: #[validator.validateIp(‘127.0.0.1’)]

Could not embed GitHub Gist e92ed52aa863ccbe2d48: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Validate url

Validates that a given String can be interpreted as an url. This is done by invoking the URL(String) constructor in the java.net.URL class. If that constructor throws exception then the validation fails. Whatever String which that constructor accepts is considered valid.

Could not embed GitHub Gist 9a36ff3bb9ee8446161a: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Also available through MEL with two overloads:

Could not embed GitHub Gist f0a0f7c4de06a01e4c82: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Boolean / Fallback validations

Although the validations above are quite general and cover many use cases, there’s always the possibility that it doesn’t quite matches your use case, that’s why there’re two fallback expressions which simply evaluate that a given expression is true or false:

Could not embed GitHub Gist caee692969ed893335b9: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Notice that:

  • Because conceptually speaking a validator should not modify the message payload or any of its properties, the MEL expression used here is expected to not have any side effects.
  • It makes no sense to expose these through MEL since boolean comparison is something already built into the language.

Custom Validators

If you want to perform complex validation logic, or reuse some code you already have on a separata jar, then the is-true and is-false fallbacks might not be that convenient. You might want to build your own validator.

Leveraging the Validator API, users can create their own validators, and there will be two mechanisms to execute them in a flow. Next we’ll list the steps necessary to do that

Implement the Validator interface

Just like other components like the MessageProcessor, Filter, Router, etc., the validators also have a well defined contract inside Mule’s code:

Could not embed GitHub Gist b02b83f778baf1d174ff: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

And in case you’re wondering, here’s how the ValidationResult looks like:

Could not embed GitHub Gist 69af1dfc71d54e94351a: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Your implementations of the Validator interface are required to:

  • Have a default constructor
  • Be thread-safe
  • Should not have any side effects over the message payload or any of its variables

Once you’ve done that, you can execute the validator through the custom-validator message processor:

Could not embed GitHub Gist d004b5b5c1338bf09df2: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Each instance of custom-validator will create its own instance of the given class and will hold it, thus the requirement to be thread-safe. The recommended way of achieving thread safeness is by being stateless, but that’s up to the developer.

Another option is to reference the custom validator via a registry name:

Could not embed GitHub Gist c11ece36f4d8af6962aa: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Notice that once more, the instance of the validator to be used will always be the same across different executions and thus the validator needs to be thread-safe. Your implementation of Validator can be either in the project or in a jar you add to the classpath. It doesn’t matter as long as it’s instantiable by the application’s class loader

Customizing exception class and message

There’re some cases in which the user might require low level control on how exceptions are created when a validation fails. For that, the following interface will be used:

Could not embed GitHub Gist 2e533b2edfd889f2643c: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

This interface receives the Event which failed the validation and the validator that raised the error. This method is intended to return the exception to be thrown but not to throw it.

Implementations of this interface should never throw exceptions and be thread-safe and have a public default constructor.

Global Level

At a global level, you can only override the default ExceptionFactory. It doesn’t make sense to try to customize a message at this point since you don’t know which validator might fail. You can configure it like this:

Could not embed GitHub Gist d86dd44fd22a24c0ddd6: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

You could also provide a reference to a Spring Bean instead:

Could not embed GitHub Gist 9dfe0833f48b2c48ae75: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Notice that you either provide a classname or a bean reference. You can’t do both things.

In Studio, you can do the same by creating a validation:config global element. You can do that by dropping a validation component in your flow and clicking on the add configuration icon:

Screen Shot 2015-08-31 at 10.56.58 AM

 

Then select the validation configuration:

Screen Shot 2015-08-31 at 11.06.22 AM

You can either provide the classname of an ExceptionFactory or a reference to a Spring Bean.

Screen Shot 2015-08-31 at 11.09.36 AM

Validator Level

On any of these validators you can customize the type of exception thrown by providing the canonical name of an exception type. If that exception type does not override the constructor Exception(String) an IllegalArgumentException will be thrown. You also get the chance to customize the message of the exception thrown.

Could not embed GitHub Gist 1be7e8ba3ef50a4fb13a: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

The above setting overrides the global ExceptionFactory configured in the validation config and creates the exception by those parameter. NotAnAdultException is expected to have a constructor taking one String argument, otherwise it will fail (that will be validated at start time).

NOTE: You don’t have to customize both the exception type and the message, you can also just customize one of them.

In Studio, you can do this clicking on the customize tab that you get on every validator component:

Screen Shot 2015-08-31 at 11.43.14 AM

I18N

Since validators provide a message upon failure, another common concern is how to apply I18N. By default, the common validators provide their message in American English. Those message are not hardcoded but in a resource file. Users wanting to provide their own internationalized messages can do so by specifying their own resources file at a config level:

Could not embed GitHub Gist 7ab0bab7dbf1ceb4565b: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

The i18n is optional, but if you specify it then the bundle Path attribute is mandatory. The locale attribute is optional and defaults to the system locale. However, it is most useful when used with an expression which returns the locale to be applied on the given event:

Could not embed GitHub Gist b23ac567dde79877a1e6: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

The example above assumes that by the time the validator is executed, there will be a flowVar called tenantLocale in which the locale to be used has been set (local is optional, it will default to the current locale).

You can also do this in Studio using the global elements config dialog, just like with the ExceptionFactory:

Screen Shot 2015-08-31 at 12.11.53 PM

Validating many conditions at once

There’re scenarios in which you want to evaluate many conditions with the possibility that many of them fail. In those cases, it’s useful to generate only one error with all of the descriptions. This use case is also supported:

Could not embed GitHub Gist e96f75a5934a2f521cff: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

 

About the all validator:

  • All validations are executed, even if all of them fail
  • If any of the validations above fail, one single exception will be thrown. The exception contains a multiline message in which each line corresponds to each failing validation.
  • Validators are executed sequentially using the flow’s thread
  • Since validators don’t have side effects, the order shouldn’t matter
  • Unlike the other validators, the all validator does not allow directly customizing the exception type or the message through a validation:exception or exception factory elements (you can however customize the message of the inner validators though).

In Studio, you can use this by dropping a validation component into your flow and choosing the “All” validator. You’ll get a table below in which you can add/edit/remove your custom validators:

Screen Shot 2015-08-31 at 12.54.09 PM

Example: Validate json Http Post

Let’s wrap this up with a simple example. Suppose that someone is posting the following json through a http listener:

Could not embed GitHub Gist 8ea36ffdc95b38f9d3d5: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

Now consider the following config:

Could not embed GitHub Gist b94370cbc8103c1768d6: Bad credentials. The API can't be accessed using username/password authentication. Please create a personal access token to access this endpoint: http://github.com/settings/tokens

In the example above we validate that:

  • First and last name are not empty strings
  • That the age is a valid integer number above 18
  • That the email address is valid
  • That the social security number has the correct size and matches a regular expression

Valid feature?

That’s it. We hope you like this feature. Have feedback? Please comment!


We'd love to hear your opinion on this post


12 Responses to “Introducing the Validations Module”

  1. Well done, @MuleSoft!

    This great product just keep getting better.

  2. Well done, @MuleSoft!

    This great product just keeps getting better.

  3. This is great! what about a standard validator for xsd schema validation?

  4. This is a great feature. What about XSD and JSON validators? Do we need to write them as custom validators?

    • Hello, for json there’s a validator which you can even use inside a validation:all block. More information here

  5. Awesome one. This avoid lots of choice routers and helps in reducing a lot of un-necessary scenarios while writing MUnit test cases. Thanks for sharing.

  6. I feel, Mule has to normalize the releases. For customers it is becoming hard to keep up with your releases.

  7. Thanks for the valuable post.

  8. This is a great module, but I feel the killer-feature is the ability to customise the exception-type and exception-message.

    This benefit is unfortunately lost with the validate-all component. I tried to use it in my flow and was ready to submit a bug-report until I read here that it is expected behaviour to simply throw a single exception regardless of the internal exception-types.

    I feel there should be a short-circuit validate-all, in which it evaluates internal validators until one fails, and it then throws that validators declared exception-type.

    Some other validators would be very useful as well, a “validate schema” validator that can validate XML/ JSON would be nice.

    Otherwise extremely good module. The validate-all problem can be solved simply by putting numerous validators in a row as well, so no biggie.

  9. I’ve tried to provide my own customer ExceptionFactory, but when the exception is eventually raised to my apikit:mapping-exception-strategy, it’s always been transmuted to a generic org.mule.api.MessagingException. it kind of defeats the purpose of having custom exceptions if they do not bubble back to the generic exception handler

    • Hello Nathan,

      Thank you for your feedback. It seems like it’s actually apiKit the one taking the exception you threw and wrapping it in a MessagingException. Have you checked the cause of the MessagingException to see if the exception you expected is there?