Accelerate integrating with APIs using Mule and the DevKit

November 1 2011

10 comments 0

We spent a good portion of 2011 working about our initiative. That initiative has brought us the wonderful Cloud Connect DevKit, a brand-new MuleForge and lots of cloud connectors. We are extremely pleased with how all turned out and we think that the Cloud Connect played a huge part into the success that Cloud Connect has become. We felt truly in love with the way you can turn a into a fully feature Cloud Connector.

As a matter of fact we love it so much that we decided to expand that development philosophy to everything Mule. Hence, we are are proud to introduce… the DevKit. The new DevKit is much more flexible than ever before, and we extended its hooks from Cloud Connectors to Mule Modules in general. Let me give you a quick tour of what is it all about.

A Quick Overview

Before we get into the new features of the DevKit I want to talk a little bit about what the DevKit and what it can do for you. The DevKit is a tool to accelerate the development of Mule Modules. Its basic philosophy is to keep your Java development to a minimum while generating some boilerplate code behind the scenes to make everything work.

Imagine that you want to develop a custom Message Processor that will transform the Payload of the Mule Message in some way. That kind of development usually involves writing the following:

  • An schema that extends abstract types on Mule schema
  • A namespace handler for the namespace of your module
  • A bean definition parse that will parse the schema elements
  • A class that extends MessageProcessor

The DevKit converts that process into just the following:

  • Write a POJO that contains a method annotated with @Processor

Look simple, isn’t it? That is the idea behind the DevKit. It will generate the schema, the namespace handler, the bean definition parsers and everything that might be needed to extend Mule.

New Annotation Set

We think that the previous annotation set was excellent for Cloud Connectors, it felt awkward when those concepts where applied to anything other than Cloud Connectors. So we re-aligned the annotation set to match key Mule concepts making them truly intuitive for a Mule developer.

The following are the core anntoations:

  • @Module – Marks your class as Mule Module. This a key annotations as without it the class won’t get parsed. It dictates under which namespace your module will live.
  • @Processor – Marks a method inside your class as being callable as a MessageProcessor.
  • @Source – Marks a method inside your class as being callable as a MessageSource.

Bean Injection

Cloud Connectors used to have @Property to declare fields within the connectors which could be configured from an outside source such as Mule. We have renamed the @Property notation into @Configurable.

@Configurable can be placed on fields and/or setter methods in the module to mark POJO properties that can be configured.

New in this release is the ability to also generate special bean reference fields. Imagine that we want to create a Mule module that searches thru MongoDB documents for keywords. In that case you would want to re-use our MongoDB Connector. Something along the lines of what follows:

Which could potentially be used from Mule as:

New Project Types

As you undoubtedly have guessed by now, Cloud Connectors are no longer the only thing that can be built using the DevKit. We have extended its reach far and beyond and we will continue to extend its reach in the upcoming months. The current list of this release is:

  • Cloud Connector
  • Transformer
  • Generic Mule Module


Streaming APIs

It used to be that the Cloud Connect SDK could only generate Message Processors, which were great for calling third party APIs. But, what happened when that API supported streaming? What if I wanted that stream to generate events that will trigger my flow?

We thought about that, and the result is @Source. [email protected] will generate Message Sources instead of Message Processors. A Message Source can generate events that are later processed by Message Processors. The closes analogy could be that of an inbound endpoint. While an inbound endpoint offers a bunch of functionality at its core it is a Message Source.

Let me show you an example. Imagine that you want to connect to an API that implements Bayeux protocol, and whenever you get an event over the wire you want to send that event over to Mule for processing within a flow.

Notice that in the method signature there is a SourceCallback, which is mandatory for every method implementing @Source. The SourceCallback is your method communication with the outside world (in this case it will be the flow). Every time you call callback.process(message) it will generate a MuleEvent containing message as the payload and send it for processing to the flow.

Here is how you could such method inside your Mule configs:


Improved Collection Support

The Cloud Connect DevKit had collection support for a while now. We have extended that support to include the ability to reference beans from whit in the collection or to replace the entire collection with a reference as well.

We have changed they way we generate the XML schema for Maps as well. Previously for each item of the collection you were able to specify the key value pair using the Key and Value attributes on the item element, now the Key remains the same, but the Value attribute has been deprecated. The Value is now obtain from within the text content of the element.

And last but not least is the free form feature for Maps. This feature is particularly useful for making the XML more readable or as a replacement when the structure of an object is not known in advance. Look at how you will be able to invoke our SalesForce connector:


Mule’s Lifecycle Support

We found that many Cloud Connectors were implementing Initialisable interface to do some sort of initialization. Of course implementing that interface in the connector POJO added a dependency directly to Mule. It is not that we do not encourage that it is that we always thought the DevKit as a bridge between a POJO and Mule, and that we want the DevKit to provide this kind of functionality. Of course that starting today it does.

We added not only the possibility for the Mule module to participate in the initialization phase of its lifecycle but in all four phases of it. Each phase is controlled via an annotation:[email protected], @PreDestroy, @Start and @Stop.


Automatic Connection Management

This one is another heavily requested feature. The DevKit can generate automatic connection management around your POJO. What this means is that if you correctly annotate your POJO telling our DevKit what method inside your POJO is used for connecting, which one is using for disconnecting, etc then the DevKit will generate a whole layer on top of your POJO in which true connection management happens. The connection management adds the following features to your POJO:

  • The ability to extract connection parameters (which used to be at the configurable level before) from the payload or headers of the message.
  • Automatic retry and reconnect. As an example if the third party API fails because the session expired, the DevKit will automatically drop the connection, establish a new one and retry the operation.
  • Connection pool. Automatic pooling of connections. We keep the connections open for as long as the user wants to, we also pool them so multiple operations can use the same connections.

Using such features is not complicated at all, it just involves adding four simple annotations to your POJO:

New JavaDoc Doclet

And almost last… is the new JavaDoc doclet. We want your module to be awesome, and honestly what awesome thing have you come about which didn’t have documentation? The Doclet combines JavaDoc and MuleDoc into a single searchable entity. The users of your module will be right at home. We introduced new doclet annotations (like @sample.xml) that will help you to document your module and add examples on how a user might see it. The best part about it is that you don’t actually have to do anything except document your code and run mvn javadoc:javadoc.

If you want to see what the new documentation will look like you might want to check the documentation of our Salesforce connector.

Never leave your IDE ever again

The Cloud Connect DevKit was based around Maven but the reality is that to this day the integration between Maven and IDEs (like Eclipse or IntelliJ) is not what it should be. It was required for the developer to leave the IDE fantasy if he/she ever wanted to rebuild the glue code/schema that the DevKit generated. We didn’t liked that. So we reworked the idea and ported the Maven plugin into an Annotation Processor. Most moderns IDEs have support for them out of the box. Of course that our Maven plugin is still there but it is just a wrapper around APT.  Now you can code your Mule module and have the code being generated on-the-fly as you compile your code.

There is more?

Of course this is just the tip of the iceberg. We added many more features but listing them all would make this post endless. I truly encourage you to try our DevKit out. This is a list of features that have been added:

Don’t worry if any of these sound strange to you. We will cover them in detail in upcoming blog posts.

More to come…

You can find the repository along with its source code here. Our documentation can be found here.

We'd love to hear your opinion on this post

10 Responses to “Accelerate integrating with APIs using Mule and the DevKit”

  1. Congrats Emiliano and team, this is a major addition to Mule! I expect to see connectors flourish thanks to DevKit 🙂

    We all think of public APIs but I think DevKit will also be of paramount importance for corporate Mule users looking to integrate internal APIs. They’ll be able to create specific and very expressive connectors and share them internally.

  2. Emiliano, this looks like a fantastic leap forward. It validates our decision to move forward to build cloud connectors. I am excited to use this new version. Is it only available in source code form currently?

    • No, we offer binaries as well. Check out the documentation link, it will tell you how to get started.

  3. Congrats – this is huge! I agree David, this lowers the barrier of entry for private / internal extension of Mule. Its also very nice to see the Cloud Connector model of development extended to the rest of the system.

  4. […] this blog post I will show how to extend Mule in a simple way using the recently released Mule DevKit. The goal of the Mule DevKit is to accelerate the development of Mule extensions by abstracting you […]

  5. Do you consider the DevKit [3.0.3] at this point stable enough for production development for Mule 3.2? Will this version or a version in the near future be marked Release instead of Snapshot?

    • 3.0.3 is SNAPSHOT at this point. 3.0.2 is our latest release version.

      We are always improving the DevKit and we have a substantial roadmap ahead of us full of improvements. Almost 90% of our connector base is based on 3.0.2 so yes we consider it to be stable enough. Having said that, the DevKit is always a moving target. We are releasing new minor DevKit versions almost weekly. Having said that, we try to keep our compatibility with previous versions as high as possible.

      Is there any particular feature that you need in 3.0.3 that is not available in 3.0.2?

  6. […] notifications and dispatch updates to one or more Message Processors. With the recently released Mule DevKit you don’t need to care about this as you can simply create Message Sources using just one […]

  7. […] previously demonstrated Mule DevKit is an excellent way to add first-class, configuration-schema aware support for your […]

  8. […] We’ve made it easier than ever for developers to create and use their own cloud connectors, with the Mule DevKit […]