Given that transformational programs touch a large portion of an enterprise’s core system, they can serve as a unique opportunity to develop an integration strategy. As is often the case with integrations, it may seem simpler to solve for isolated portions of the work via point-to-point solutions; but to truly support company growth and deliver connectivity between systems, this process needs to be done from a holistic point of view.
At Air Canada we went through two major transformational programs in less than four years:
- The migration of our passenger service system (PSS), the most important IT component of an airline, which connected to about 60% of our overall systems including our reservation system, inventory system, and a departure control system.
- The (re)acquisition of our airline loyalty program Aeroplan, which involved implementing an entire new loyalty ecosystem.
Because these initiatives required a monumental integration effort, we made a strategic decision to source a new integration platform and develop an in-house integration capability. What made our delivery of these transformations so successful was that many of the components could be reused to some extent in other contexts. Here are five steps Air Canada followed to implement this and how your organization can follow along too:
#1 Formalize your objectives in a comprehensive document
Developing an integration strategy can be challenging at the beginning because there are multiple dimensions into it: technology, architecture, organization, governance, delivery, operations, etc. This complexity needs to be formalized as a whole and shared across the company — this facilitates the implementation of an integration platform and furthers its adoption.
When I was first assigned to develop my company’s integration capability, I created a comprehensive document called the “Enterprise Integration Platform Manifesto” which reinstated the core objectives of our team and integration platform. It described each aspect of what we wanted to achieve and our proposed solution for achieving it. It was shared across the different IT departments to ensure alignment with our strategy. This initial document was essential to the success of our integration capability — after that, rather than maintaining it, it was derived into a multitude of documents that are maintained and constitute the essence of our practice.
#2 Determine your delivery model
A fundamental decision that needs to be made at this stage is about the integration delivery model. MuleSoft describes three degrees of decentralization as summarized in the table below:
The best model for your company depends on a variety of factors:
- How many IT teams in your organization have developers that can develop integrations?
- How do you position your integration platform? Is it only for core systems integrations and enterprise services? Does it also include experience APIs and integration by front ends?
- How important is it for your integration delivery to scale up to accomodate for major programs?
- How realistic is it to have a portion of your integrations delivered by lines of business?
In our case, at Air Canada:
- There was limited capacity to develop among other IT teams.
- We had a well-defined digital stack for the digital/UX integration under a different department, so we did not position the platform to deliver for the front ends.
- It was essential to scale up for our major transformational program and to have some elasticity in the long term in case of a surge in project demand.
- Lines of business would only marginally help because of the complexity and criticality of most of the services we had to deliver.
As a consequence, our implementation is a hybrid of the three models, mostly looking like Model 1, with the following distinctions:
- Because these projects involved hundreds of systems and impacted thousands of employees, our major transformational programs were developed with System Integrator partners, using “Decentralized Delivery.” A C4E team ensured our standards were respected, and some members of the C4E team were part of the delivery team to ensure a smooth transition.
- Because of the need for elasticity, we partnered with IBM to build a hybrid organization where Air Canada and IBM deliver together. This made it simpler for Air Canada to augment the team when required, in a performant and cost-effective way with the right mix of onshore and offshore resources.
- We are developing a “LOB delivery” approach in some cases that allow for it. This is usually when there is a desire for it within the LOB and the development flow is not too complex or mission critical.
#3 Implement your target integration platform
Some preliminary work is needed to better understand your organization’s integration needs and find the best integration platform to meet them. Answer the following questions to better assess the options:
- What integration capabilities are already available at your company? Can some of the platforms be reused or enhanced for a portion of the work?
- How specific is your industry in terms of regulations and standards?
- What are your strengths and weaknesses in terms of software development, devops practice, and operations?
We went through an RFP process to determine the best integration platform for our needs. We learned that no single integration platform answered all our needs, so we took a best-of-breed approach and selected a combination of technologies for our enterprise integration platform:
- MulesSoft is our main platform for API integrations, API management, data transformation, and service orchestration. Our MuleSoft-dedicated team represents more than 80% of the overall integration team and this is where we concentrate most of the heavy lifting.
- IBM Integration Bus (IIB) and its embedded MQ solution is our platform for event-driven integrations (queue- and topic-based integrations, with content-based filtering and routing)
- Axway Secure Transport is our managed file transfer solution.
Once the target platform was defined, the next challenge was to build it strong enough to support our mission-critical flows, and fast enough to enable our transformational programs.
Due to the nature of our business, where an integration failure can lead to flight delays, we developed a multi-region solution with an active-warm standby setup that can failover to the secondary region within a minute. We also spent a lot of time “proving” the resilience of the platform, going through all possible partial failure scenarios of each component, then combining them.
The end result was a solid platform that we trusted on paper and in practice, while gaining operational expertise in the process.
#4 Choose a delivery approach for your transformational programs
When developing an integration strategy through a transformational program, you can’t use a product-driven approach where systems and features are progressively connected as new roadmap items come up. This is especially true in the context of a major migration.
We migrated about 30 systems to the new platform, all of which were directly connected to the old passenger system. It was clear from the start that the legacy services could not be migrated since they were too idiosyncratic and their data model had been degenerating over time.
A fair amount of these services were using similar capabilities of the passenger system, related to “flight distribution” that has two standards:
- OTA from OpenTravel Alliance
- NDC (New Distribution Capability) from International Air Transport Association.
We based our new services on the OTA standard because we had in-house expertise in it. This standard offered the different verbs we wanted to implement and a data structure attached. We just need to marginally extend it for some company-specific features. This constituted the “top-down” part of the approach.
There was a “bottom-up” aspect to it as well. Since the project was a migration, we determined the exact interactions required between the application and the passenger system at the level of each “impacted application” and listed all the data elements the application required. This ensured the migration would protect the existing scope of integration for each application.
From that work we built the “converged” services with extended OTA standards. This returned the necessary data and composed the right orchestration in all scenarios.
During our transformation journey we had to adapt our strategy as circumstances changed. Once the program was successfully delivered, the next challenge was to transition into maintenance and evolution.
#5 Maintain your assets and scale up when needed
Large transformational programs generally involve system integrators that help a company scale up with the required number of architects, delivery managers, business analysts, etc. They also have experience running programs that involve thousands of people, which is not necessarily the case in-house.
It may be more strategic for a system integrator to deliver the integrations as well — but with a major transformation, you’ll need to augment the integration team significantly then transition into a smaller maintenance and evolution team.
To ensure a successful delivery and transition when integrations are developed by an external vendor, enforce your integration practice and involve internal resources in the delivery. It’s important to “choose your battles” and to determine which resources should remain internal, given you have limited resources and don’t want to be a bottleneck in delivery.
At Air Canada, we determined that product owners should remain internal resources and ensure continuity between program delivery and steady state. We decided to fully control the DevOps toolchain and platform operations, while architecture work was a joint effort between the program team and internal team. Regarding development, the first program was largely outsourced due to a lack of internal resources. For the second program we embedded internal developers into the delivery team, which proved to be an easier transition.
Down the line, you will transition to a steady-state model where smaller teams can maintain your assets, and some delivery can shift to new projects as the need for integration grows in other lines of business. The main challenge after that is to determine your team augmentation strategy, either by hiring ad hoc consultants or through preferred partners. At Air Canada, we made the strategic choice to build a hybrid integration team that combines internal staff and IBM, which was efficientive for quickly adding resources when there is an urgent project need.
Summary
The key lessons learned from our integration journey are:
- Define and share internally your overall integration plan and strategy from the start.
- Think wisely of your delivery model, there is no approach that fits all situations
- Take time to select your second program, and implement it with its long-term non-functional requirements in mind.
- Team elasticity will be key to a successful delivery model and needs to be part of your strategy, especially when delivering company-wide transformational programs and when facing a temporary surge of integration demand due to new projects
My team at Air Canada will be sharing more of our projects and how-tos over the next few months. In the meantime, see more technical use cases and check out the MuleSoft Developer LinkedIn page.
