Handling HTTP callbacks using the new Mule DevKit: a Twilio example

motif

When you send a request to an API and it gets processed the API might want to notify you app about the status of the request. In order for your application to handle this callback you would have to set up an endpoint to listen for the notification and then send the url of that endpoint to the API. For example Twilio, one of the most popular public APIs, uses callbacks to tell you whether a SMS message has been successfully delivered or not in addition to a SMS id that you can use later for tracking purposes. Since the goal of the new Mule DevKit is to make things simpler it provides an easy way to handle HTTP callbacks.

Handling HTTP callbacks with the Mule DevKit

The Mule DevKit provides a special data type to facilitate the scenario in which a Mule flow needs to be invoked when a HTTP callback is received. The org.mule.<a href="http://www.mulesoft.com/platform/api" target="_blank" rel="" title="Anypoint Platform for APIs" >api</a>.callback.HttpCallback interface exposes the method getUrl() which tells you the URL to use in order to invoke a particular flow when a callback is received:

The Message Processor that gets generated by the Mule DevKit can be consumed as follows:

where handleSuccessFlow and handleFailureFlow are two flows that don’t have any inbound endpoints whatsoever:

A couple of things to notice:

  • parameters of type HttpCallback are appended a “-flow-ref” suffix to emphasize the fact that you need to pass a flow name in the Mule configuration XML file.
  • a method can take any number of HttpCallback parameters
  • a parameter of type HttpCallback can be annotated with @Optional

in the

As I mentioned before, Twilio API uses callbacks to tell you about the status of your requests. Wether you are making an outbound callback or sending a SMS message, Twilio API allows you to send a URL where you’ll receive information about the phone call once it ends or the status of the outbound SMS message after it’s processed by Twilio. Let’s consider the following two flows:

The first one simply uses the Twilio Cloud Connector to send a SMS message. The second one will get executed when Twilio notifies your app about the SMS message status. Notice that the flow ‘callbackFlow’ has no inbound endpoint but is referenced by the status-callback-flow-ref attribute in send-sms-message.

Conclusion

What I find interesting about this feature is that it not only simplifies the development of Cloud Connectors but users of the Cloud Connector get a nice and intuitive way of saying “hey, invoke this flow when X happens” without knowing too much about callbacks, inbound endpoints, URLs and all that detail.

More information on this new feature can be found here. Another good source of information is the Twilio Cloud Connector source code.

follow: @federecio @mulejockey


We'd love to hear your opinion on this post


16 Responses to “Handling HTTP callbacks using the new Mule DevKit: a Twilio example”

  1. Taking a step-back,
    I can see how the callback flows are referenced by the custom Module, but how would I configure my application to use the custom instance I created to process the callback?
    Would you use the “…flow.xml” file to configure the custom Module/Connector, or the mule-config.xml file?
    And how does that definition look?

    Just coming up to speed –
    Thanks
    Kevin

    Agree(0)Disagree(0)Comment
  2. Hi Kevin,

    I don’t think I quite follow you: “…how would I configure my application to use the custom instance I created to process the callback?” What do you mean by custom instance?

    Would you mind posting the relevant piece of code so that I can guide you through?

    Thanks,
    Fede

    Agree(0)Disagree(0)Comment
  3. Hey Federico,
    Thanks for responding.

    In the above ExampleConnector java code at the top of this page, you have an @Module defined as a class that contains an @Processor. This is the class I am speaking of.

    How do I configure my flow to use this class? I see your example xml just below it “consuming” this code, but when I start my application with this example, I get “not bound” errors from the xml parser.

    So, I take it that you are defining your custom namespace in the mule-config
    file. However, when I set up my own namespace for this custom class, the attributes relevant to my class are not found. Here is a snippet of the stack trace:

    ” cvc-complex-type.2.4.a: Invalid content was found starting with element ‘sendsms:config'”

    So – I am just trying to tell Mule that I have a custom (sendsms) class deployed and to use that.

    Do I need to also define an .xsd file for this class?

    Here is my class:

    @Module(name=”sendsms”, schemaVersion=”1.0″)
    public class sendSMSModule
    {
    /**
    * Configurable
    */
    @Configurable
    private String myProperty;

    /**
    * Set property
    *
    * @param myProperty My property
    */
    public void setMyProperty(String myProperty)
    {
    this.myProperty = myProperty;
    }

    /**
    * Custom processor
    *
    * {@sample.xml ../../../doc/sendSMS-connector.xml.sample sendsms:my-processor}
    *
    * @param onSuccess The HttpCallback for successful transmission
    * @param onFailure The HttpCallback for *un-successful* transmission
    *
    * @return the URL for success or failure accordingly
    */
    @Processor
    public String myProcessor(HttpCallback onSuccess, HttpCallback onFailure) {
    String onSuccessFlowUrl = null;
    String onFailureFlowUrl = null;

    onSuccessFlowUrl = onSuccess.getUrl();
    onFailureFlowUrl = onFailure.getUrl();

    //do something with the outcome
    return onSuccessFlowUrl != null ? onSuccessFlowUrl : onFailureFlowUrl;
    }
    }

    Here is how I am trying to “consume” it:

    Thanks,
    Any help is appreciated.

    Kevin

    Agree(0)Disagree(0)Comment
  4. The xml didn’t post for”how I am trying to “consume” it”: I’ll try again.

    Agree(0)Disagree(0)Comment
  5. Hey Kevin,

    what you are doing looks ok. Are you trying to run a unit test from your IDE when you get this error? If so, you have to make sure your IDE knows where to get the xsd file from because it is not hosted online. The xsd file is under target/generated-sources/mule/META-INF and there is a file called spring.schemas in the same directory which maps the namespace URI with the physical location of the schema file.

    Keep in mind that since the xsd file gets generated by a Maven plugin, you have to run the ‘compile’ goal at least.

    I would suggest you try running the test from the command line using maven and then configure your IDE (if you are using intelliJ is pretty straightforward, right click on the namespace URL -> ‘Manually set up external resource’).

    Hope it helps,
    Federico

    Agree(0)Disagree(0)Comment
  6. Federico,
    Thanks again for responding. I finally think I am on the right path.

    I am using MuleStudio Release Candidate to run my simple flow and I am just trying to stand up the basic functionality.

    Since the RC MuleStudio IDE does not have the “Web and XML” “Editing XML Catalog Settings” portion in Eclipse->Preferences, I can’t use the IDE to configure where the local .xsd resides (for example, see this link: http://wiki.eclipse.org/Using_the_XML_Catalog ).

    Can you show me how to specify the xsd locally – using the mule-config.xml file?

    I am using xsi:schemaLocation like this with no luck.
    (I’m developing on a mac)
    .
    .
    xsi:schemaLocation=”
    file:////Users/wristfracture/MuleStudio/workspace/twilio_send_sms/target/generated-sources/mule/META-INF/mule-sendsms file:////Users/wristfracture/MuleStudio/workspace/twilio_send_sms/target/generated-sources/mule/META-INF/mule-sendsms.xsd”>

    Thanks,
    Kevin

    Agree(0)Disagree(0)Comment
  7. Hey Kevin,

    Oh I didn’t realize you were working on Studio. The version of Studio you are using doesn’t support custom Cloud Connectors, this is targeted after Studio GA.

    Having said that, there is a workaround to use your custom Cloud Connector inside Studio (but with no visual editing). How are you doing that right now?

    The steps are:

    1- build your your project using Maven: mvn clean package

    2- browse your /target folder and locate a zip file, copy that file to your Studio project in src/main/app/plugins (you might need to create the ‘plugins’ folder)

    And that should be it. I just tried this myself and works just fine. I think you were missing step #2.

    Please note that this is just a workaround. Studio will provide support for custom modules after GA version.

    Let me know if you still have problems.
    Federico

    Agree(0)Disagree(0)Comment
  8. Federico,

    I am able to use Studio to stand up an outgoing flow using an Http endpoint -> Twilio Cloud Connector. The Twilio service is receiving fine.

    However, on the return from Twilio is where I am stuck. Twilio should be returning an acknowledgment – and my custom connector (as shown in your code above should receive it.

    By the way – what does copying the zip file into the plugins directory do for me? I can’t tell any thing is different about Eclipse.

    Also – do you know how to specify a local .xsd file in my config file? I can’t seem to get that right.

    Kevin

    Agree(0)Disagree(0)Comment
  9. Federico,

    The Mule-Twilio.xsd schema is no longer online…The MuleSoft online documentation references this everywhere.

    Is there a reason it has been pulled?

    http://www.mulesoft.org/schema/mule/twilio/1.0/mule-twilio.xsd

    Kevin

    Agree(0)Disagree(0)Comment
  10. Kevin,

    Schemas are not hosted online. Instead they are packaged inside the connector’s JAR/ZIP file. There is a file called spring.schemas which handles the mapping between the namespace URL and the physical location of the schema.

    As to why you are not receiving the acknowledgement from Twilio, are you sure your IP address is accessible from outside your network? If it is accessible, can you share your flows?

    Federico

    Agree(0)Disagree(0)Comment
  11. Federico, thanks man.

    I’ll check my IP, Twilio might be throwing ack’s against my gateway address.

    Can I just use an http:inbound-endpoint instead of a “status-callback-flow-ref” to receive Twilio acknowledgements? I need to have a static URL (not a dynamically generated one like the twilio connector produces I assume)…

    Als on an unrelated note –

    The developers here in my shop (along with myself) are having difficulties “packaging” the mule deployable with maven.

    We are running “mvn clean package” with mule in the pom.xml file.

    But, the produced .zip file in the target directory doesn’t contain the entire contents of what is necessary for the app. i.e., there is no lib/ folder and none of the dependent j.jars.

    is there something we are missing in order to produce a valid z.ip deployable with maven?

    Kevin

    Agree(0)Disagree(0)Comment
  12. Hey Kevin,

    You don’t need to have an inbound endpoint, that’s the idea. The part of the URL that is dynamic is the path, the host portion is configurable. You might want to check this docs: http://www.mulesoft.org/documentation/display/DEVKIT/Handling+HTTP+callbacks

    The Maven artifact will be deployable to Mule only if the packaging is set to mule-module. Check that you have this in your pom.xml:

    mule-module

    If you have that and Maven builds successfully then you are ok to deploy to Mule. Keep in mind that Mule jars will not be included in the zip file since they are provided by Mule at runtime.

    If you you follow this steps and you still can’t deploy to Mule please share the stack trace.

    HTH,
    Federico

    Agree(0)Disagree(0)Comment
  13. Well, the reason I say I think we need to have a static URL is because sometimes it takes a long time to acknowledge the delivery of an SMS message. Up to 27 hours… During this time, our server can go down or network outages can occur. Upon rebooting, the dynamically generated URL (path) won’t be recalled/re-bound/ or running. There is no way for our service to figure out how to reconstruct the URL for Twilio to contact us with the acknowledgement.

    Have any ideas about that?

    Agree(0)Disagree(0)Comment
  14. btw, our whole thread is being posted to your blog page you sent.

    Agree(0)Disagree(0)Comment
  15. It would be nice if the HttpCallback interface allowed me to specify the ‘/path’ portion of the URL and have that be a static path. Instead of only having the ‘/path’ portion unspecified and randomly generated. This way, I could have a static URL.

    see: http://www.mulesoft.org/documentation/display/DEVKIT/Handling+HTTP+callbacks

    “How It Works
    Behind the scenes a Mule flow is created dynamically containing a Message Processor that invokes the desired flow. By default, this Message Processor is wrapped in a async message processor chain. This behavior can be overriden using the ‘async’ option.

    The URL of the HTTP inbound endpoint is built using ‘localhost’ as host and ‘http.port’ environment variable or ‘localPort’ value as port. The path of the URL is a random string.

    The URL that is passed to the external system is the same one as the URL of the HTTP inbound endpoint except that the host is replaced by the ‘domain’ setting (or its default value) and the port is replaced by the ‘remotePort’ setting (or its default value).”

    Agree(0)Disagree(0)Comment
  16. Hey Kevin,

    We use GitHub to track issues, would you mind opening a ticket here:

    https://github.com/mulesoft/mule-devkit/issues

    And there we can discuss further what you are proposing.

    Thanks,
    Federico

    Agree(0)Disagree(0)Comment