December 2013 Release – Batch Jobs, Expression auto-completion and more

December 10 2013

2 comments 0
motif

We’re excited to announce the Mule Studio (December 2013) release! It includes a new CloudHub Mule Runtime (December 2013), which introduces new batch capabilities for complex ETL-like data integration tasks such as synchronizing SaaS and on-premise applications, flat file processing, and database reporting and synchronization. Also new in this release are Studio support for Mule expression language auto-completion, cron-expressions for poll scheduling and support for management of job schedules in CloudHub.

 

Batch Jobs: Mule for data integration workloads

Batch jobs help you accomplish complex data integration tasks by expanding Mule’s capabilities. They provide a toolkit of building blocks to do things such as synchronizing SaaS and on-premise applications, flat file processing, and database synchronization or warehousing.

Batch jobs evolved directly out of our experience with building these types of integrations for our customers. This new functionality reduces the need for custom coding, which decreases time-to-production and increases maintainability. Some highlights of the new capabilities include:

  • a load, step and commit model, which ensures reliability and recovery
  • an ability to stream records from large record sets in flat files or connectors such as Salesforce
  • a capability to conduct parallel record processing for fast loading
  • a feature to set maximum number of failed-to-process records to prevent a batch job from continuing to process
  • an ability to manage and control schedules from CloudHub
  • an ability to report on the result of the job
Batch jobs have a very structured way of functioning which is based around the concept of records. This structure makes it possible for things like reporting and parallel processing to happen automatically. Let’s take a look at the processing phases in the context of an example use case: loading a large set of leads from a CSV file into Salesforce.
  1. Input: This phase of processing accepts your serialized message payload for processing as a batch job. In our case, this means watching for a new CSV file, then converting the data into a collection.
  2. Load and Dispatch: Next, the Mule created records of the serialized content, then loads all the records onto a queue for processing. This ensures that even if there is a problem that occurs during the job, the job can recover and continue.
  3. Process: Then, each record gets processed in a series of batch steps – typically this means loading into another target system. For example, you could have one step to check if the lead exists and another to load it into Salesforce. This phase happens in parallel to speed up processing.
  4. On-Complete: Finally, you can use the on-complete section to create a report of the results of processing a batch job, including how many records failed and how many succeeded.
You can find more details in this complete end to end example.

 

Cron Support and CloudHub schedule management

To complement our new batch capabilities, we’ve introducued the ability to create cron-based schedules with the poll element. This makes more complex scheduling or polling – such as Monday/Wednesday/Friday schedules – possible without having to use Quartz.

In addition, you can manage these schedules from CloudHub. You can view, change and dynamically run schedules from a new schedules tab. Managed by a centralized scheduling service, schedules ensure that only one worker is running a schedule at a time, and the load balances between different workers.

Mule Expression Language auto-completion

Also new in the this release is support for auto-completion in Mule Studio. This functionality ties in with Mule DataSense to extend our metadata-driven design experience, which gives you access to the payload information at design time. It also includes support for available functions, such as xpath() and regex(), so you can see the arguments and documentation for each function.

Further refinements

We’ve also introduced several other refinements, including DataMapper performance improvements, a simplified New Project wizard, and improvements to DataSense usability. For a complete guide, please consult the release notes.

 

Happy integrating!

We believe that having a single, unified platform for your integration needs – from data integration to API publishing – is what the New Enterprise needs, and we’re excited that this release delivers on that vision. We hope you’re as eager to test drive these new capabilities as we are to introduce them. For more information, access the the release notes or and download Mule Studio (December 2013) with CloudHub Mule Runtime (December 2013) on our download page. These improved runtime capabilities are available now with CloudHub and will be available in our on-premise tool (Mule ESB) with the 3.5 release early next year. As always, we look forward to your feedback in this post’s comments and in our forum.

 


We'd love to hear your opinion on this post


2 Responses to “December 2013 Release – Batch Jobs, Expression auto-completion and more”

  1. Great news, I’m excited to get my hands on the “batch jobs” feature. When will it be available for us running Mule ESB on-premise?

  2. It’s coming to on-prem early next year. You can already try it out with Studio though and we’re going to get an on-prem beta/milestone release soon too for folks to try.