Integration has been an enterprise challenge for a long time. In order to create the new products, applications, and services that businesses need to confront the digital revolution, you have to connect systems, data, and devices. There’s no way around it. Enterprises struggled with point-to-point integrations, which created a spaghetti-like mess of code, and then, for many years, companies found a better integration solution with ESBs (Enterprise Service Bus) and an ESB integration approach.
ESB integration helps clear out the spaghetti mess created by masses of point-to-point integrations by providing a simple, well defined, “pluggable” system that scales well. In addition, ESB integration provides a way to leverage your existing systems and expose them to new applications.
But times change, and technology evolves, and for today’s complex enterprise technological architecture – with its hybrid infrastructure and a rapidly exploding number of endpoints – ESB integration is no longer adequate. ESB integration presents both technical and organizational challenges, reducing organizations’ integration capabilities’ effectiveness.
Organizations should look beyond ESBs and consider an API-led approach to integration. In a future post, we will look at the advantages that an API-led offers over an ESB Integration. For now, here are the top 5 challenges with an ESB-Led approach.
1. ESB Integration presents organizational challenges
ESBs face procedural and political bottlenecks as different parts of the business need to move at different speeds. Gartner refers to this as PACE layering; they recognize that business capabilities – together with their supporting applications – support innovation, differentiation, and operations––all of which need to change at different speeds.
An ESB can often become the bottleneck to implementing changes, while maintaining service to the rest of the business. For example, HR needs to implement new legislation updates, Finance wants to implement new tax measures, and Asset Management needs to acquire and onboard new warehouse capacity. With ESB integration, all the changes are added to the ESB change queue, which moves slowly. It’s hard to explain to the business owners that they have to wait for IT and other priorities are ahead in the queue. While it is possible to implement multiple ESB instances to support multiple business domains (HR, Finance etc.), this is where we hit the second ESB integration challenge: cost.
2. ESB Integration encourages a monolithic architecture with high cost
ESBs can be very complex; they comprise multiple products, require significant hardware resources, have many different tools, and require specialist skills and experience which are hard to source and maintain. The cost of ESB infrastructure, implementation, and ongoing costs is high; so high, in fact, that very few customers will be able to afford multiple ESB instances. In addition, ESBs can be a single point of failure and a single point of outage, especially when upgrades are required.
3. ESBs were not designed for the cloud
With the widespread adoption of cloud and SaaS applications, integration needs to support on-premise, hybrid, and cloud/SaaS applications. This requires integration capabilities that are cloud-friendly, scale horizontally, spin up/down many instances rapidly, and provide unified tooling and a consistent approach to on-premise, hybrid, and cloud integration.
These are all capabilities which ESBs were not designed to deliver and, until today, still struggle to provide. Purchasing additional cloud integration tools is an option, but not a good one. With this approach, you have multiple integration tools and you will need to re-work your integrations as you move applications to/from the cloud. This can prevent your business from taking advantage of cloud and SaaS applications.
4. ESBs treat all integrations as equals, but they should be built with specific purposes in mind
Integrations fall into three broad categories:
- Device integrations – to support multiple channels; for example, mobile, web, and partners.
- Process integration – these integrations provide orchestration logic to tie together integrations that span multiple back-end systems.
- System integration – these provide connections to our applications.
All of these integration categories behave differently and have different requirements. While ESBs will often support process and system capabilities, they rarely provide direct support for device integrations. Customers must, therefore, purchase additional products (such as an API Gateway), increasing costs and complexity and, in turn, introducing monitoring, management, and security gaps as integrations flow backwards and forwards between multiple products.
Mixing integration categories also causes problems with ESB integrations. For example, including channel specific logic with the process integration makes it difficult to introduce support for new channels and devices. Mixing process and system logic impede easy access to backend applications and sharing of orchestration logic.
ESB integration treats all integrations as equals––reducing agility and flexibility, while increasing governance overhead.
5. ESB integration does not deliver agility and innovation
ESBs were designed before the cloud, before the mobile explosion, before social media, and before the Internet of Things. Most importantly, ESBs were designed before the enormous pressure to innovate and move faster arose––triggered by new technologies, rising consumer expectations, and business disruptors.
ESBs, like many other legacy technologies, have added support for CI/CD and DevOps, but the products were simply not designed for agility and innovation. Experience shows you can’t simply add agile and innovation capabilities to legacy products; they may have the “bumper stickers” but they are not able to deliver. And, finally, ESBs were built to deliver technical integration components and this has resulted in low-levels of reuse and poor support for Line of Business (LoB) IT.
ESBs have had their time, but evolving enterprise technology requires a different integration approach. Take a look at how businesses in every industry have met their digital transformation goals by adopting an API-led approach to connectivity.
