With the explosion of APIs (funny that we have started using the term APIs instead of web services) what have created a monster integration challenge that I talked about in a previous post about cloud silos. However, to get an idea of how this challenge will manifest itself I thought it would be interesting to take a look at some statistics.
Programmable web is a great resource for tracking APIs and mashups and has become the defacto registry for web APIs. The site provides a wealth of statistics tracking back to 2005 when the site was started by John Musser.
You can see that just 5 years ago there was only 100 odd APIs out there, before that eBay, Amazon and SalesForce were the trailblazers. Whats interesting is the acceleration and the types of APIs being released. With the exception of Amazon APIs started out exposing application data, then social media, then digital content, then well everything else, It’s pretty amazing that by the end of this year there will probably be over 5,000 APIs giving you access to everything from making conference calls, to storing files in the cloud, to calculating your carbon footprint with AMEE.
API as a Product
The interesting thing is that for many new companies their API is the product, Amazon Web Services is the grand daddy here, but Twilio is one of the coolest making telephony services available to everyone. Face.com is another which offers facial recognition APIs, and pretty much all of the infrastructure services like Loggly, MongoHQ and SendGrid are all selling developer APIs to do stuff you used need infrastructure for.
The API is everything for the Cloud
This is a big shift in the market and a massive opportunity. Once everything becomes accessible and addressable using APIs we can build more amazing applications in shorter time and combine things in ways that haven’t been thought of before. The cloud becomes a mega-platform for composing applications from existing application services. We’re already seeing the value in APIs, evident by looking at the number of hits these APIs receive. The API billionaires club shows how much we are starting to depend on getting our data through APIs.
The explosion in smart devices such as phones and tablets are driving this usage up and will continue to skyrocket as more smart devices hit the market. Think about cars, smart meters, point of sales systems, appliances (Sun’s original vision for Java was not so crazy after all), toys, clothing, and everything in between will get connected. Some analysts estimate the number of connected devices to reach between 20-50 billion by 2020; that’s a staggering number. Does it seem unrealistic? I don’t think so, lets take a look at Netflix who’s API load has gone through the roof in recent months.
This is a massive jump in requests driven by the fact that Netflix is now getting embedded in TVs and Blu Ray Players. This is Metcalfe’s law in action; as we add more devices to the network, the more value is achieved, both for consumers, who can get thousands of movies on demand and for Netflix that has tapped into a major revenue channel.
What this all boils down to is that each new API creates a new silo of data and/or functionality. These APIs need to be integrated into applications to be useful for people. As we have seen in the past the urge is to integrate each API with the application directly. This way of doing integration is called point-to-point integration. Anyone that has worked on projects that do integration this way often are too afraid to touch the code or decide a rewrite is needed. The term often used for this is spaghetti code because each API is integrated directly from you application, mixing concerns of your application with no service layer to shields the application from changes in the API, or to help with authentication, error handling, monitoring and fault tolerance.
Research from Morgan Stanley in 2010 about Computer Growth drivers identify more conservative growth in connected devices but correctly assert that this exponential growth in devices presents a massive integration challenge. The dynamics of this integration challenge is different from before. The API explosion means there are many more applications to integrate. The APIs themselves differ wildly since most favour REST (~75% according to PW) where this is no contract or schema and there is no conventional wisdom out there for building RESTful APIs. Also, many of the lessons learned in the enterprise will have to be learned again by whole different audience as more app developers build applications through composition rather than traditional coding. This new developer audience will need the right tools to avoid the pitfalls and get the job done without all the hard graft of integration.
Our goal is to build the tools for this new breed of integration development, Mule Flow and Mule Cloud Connect were designed to make cloud APIs easier to consume and compose. Mule iON is the first integration Platform as a Service (iPaaS) dedicated to providing integration between APIs and on premise applications, currently in private beta. I’ll be posting more over the coming months how we are addressing these new integration challenges.