Its about time the Web became more event-driven. We have had AJAX for many years enabling events between server and browser, but on the backend we are still polling data. With the explosion of public APIs from SaaS, Social Media and Infrastructure Apps, more and more applications are written by composing web APIs. Developers often need to call a API to get data updates, only to find that nothing has changed. Streaming APIs provide a more elegant solution to polling allowing developers to subscribe to changes they are interested in.
Salesforce recently announced their new Streaming API that given a SOQL query can associate it with a topic that applications can subscribe to. This means any application subscribed on that topic will receive updates as they happen in realtime.
Streaming Schreaming. So What?
Realtime data over the web sounds good but how much improvement is there over polling? Well for starters polling isn’t very elegant, asking an application for changes is hugely inefficient. Secondly, polling APIs mean that you have rate limit restrictions, these restrictions exist to reduce load on the APIs infrastructure, and rate limiting on SaaS applications is often restricted to the number of calls a day which means you usually cannot even get close to real time without paying a lot more. Finally, the REST paradigm (the one used by 90% of public APIs) is not well suited for realtime, Streaming APIs work around the limitations to allow data to be pushed to the client in the same way AJAX does i.e. CometD and Web Sockets.
If you think back to days when AJAX term was coined, many people didn’t see the true value of what the technology offered until the first Web 2.0 web sites started to get attention. To the less informed, AJAX seemed to just remove the need for the user to refresh their page in the browser. I believe streaming APIs is a major step in moving towards an event-driven Web.
The SalesForce Streaming API
Salesforce is not the first company to announce a streaming API, Twitter, Facebook and others started to emerge almost a year ago with streaming APIs. What’s different about Salesforce is that they are the shining light for most other SaaS companies (after all they pretty much defined the category) and they’ve started the ball rolling and others will follow suit. Getting our application data at realtime isn’t something that we’ve had before, it will change the dynamics of the applications we build.
Of course, there is a big win for Salesforce with their streaming API, with over 50% of all traffic going through their APIs, that’s a lot of applications nagging their infrastructure asking for updated information. The number of requests that return no data could be as high as 60%. That’s a lot of inefficient processing. The Salesforce platform can be greatly optimised if people start using the streaming APIs since a single query result set could be filtered to serve hundreds or thousands of customers.
Streaming APIs and Mule
Mule is the first integration platform to support the SalesForce streaming API, and to test it out we create a simple demo application with Salesforce Chatter and Twilio running on our cloud platform; Mule iON.
The Demo is pretty simple, when you post to someone’s chatter wall hashtags can be used to perform actions. In this case if you post to the wall and include the #now hashtag the recipient will will be notified of your message through SMS.
To make this possible we created a Salesforce custom object (SMSNotification) that includes the user’s mobile number to send SMS messages on. We created a trigger to update this custom object with FeedItems posted on the users Chatter wall. We then created a topic called ‘/SMSNotificationStream‘ that selects everything in the SMSNotification object. So when a new post on a wall is made the SMSNotification object gets updated with the item and has a mobile number associated with it that is used to send SMSes.
One of the nice things about this demo is that is super easy, there is only a few lines of Mule XML to make it work:
- You’ll see that the flow starts with ‘sfdc:subscribe-topic‘ this initiates a streaming connection to Salesforce and will trigger the flow when new data arrives.
- The filter is used to detect hashtags. Here we only look for #now, but we could look different ones and react to each separately
- The Twilio ‘send-sms-message‘ command extracts the number and message to SMS from the payload received from Salesforce
- This configuration can be dropped in to Mule iON and happily sit there listening and reacting to Chatter; like a message switchboard in the cloud
- This all happens in realtime!
The screenshot below shows what a wall message looks like in Salesforce Chatter and the SMS message being received via Google Voice.
Let me at it!
Right now the Salesforce Streaming API is in beta and the streaming connector for Mule will not be officially released until Mule 3.2 in a few weeks. If you want to try it out before that please let me know (tweet or email ross at mulesoft).
Follow: @rossmason, @mulejockey
