During my time at MuleSoft, I’ve had a lot of conversations with people from other companies about their specific challenges and one recurring theme is almost always present: the conversation starts out talking about point-to-point integrations. It’s pretty natural when presented with two systems to want to draw a connection. The data here is needed there and, after that, we’re done because a problem was solved.
In practice, it rarely ends up working well that way as the number of projects grows over time. Regardless of the size of the company, many teams have integration needs. Imagine that the integration need start out like simple computer networks — two computers directly linked by a wire and exchanging data. Now, imagine that every two parties in a company that needs to exchange data with each other do the same thing. Eventually, a tangled mess is inevitable and the mess itself becomes a serious obstacle to moving vital projects forward. This kind of sprawl is a real challenge when the focus is only on integrations.
You wanted this:
You ended up with this:
Focus on composability:
The more demand there is for integrations within a business, the stronger the likelihood that there are common APIs that can be built on top of key business systems that would be reusable to solve multiple needs across projects. Building custom APIs allows you to create simpler integrations because it enables you to present the data to your business in a manner and format that is easily consumed by developers around the business.
Avoiding over-architecting:
On the other end of the spectrum is the threat of over-architecting, which is rightfully a very solid concern. We all know we need to spend energy thinking about how things will need to work down the road, but nobody wants to risk moving too slowly by over thinking the approach. Over-architecting tends to stem from a fear that development work will overlook specific use cases or details. This risk can be mitigated effectively by focusing on three benefits of API-led design: focus on the API consumer, rapid prototyping, and API layering.
You needed this:
You were pitched this:
Focusing on the API consumer means determining the specific data structure needs presented by the party that will be using your API. Using API Designer and RAML, the API can be created and a mock version can be tested in a very short time period (hours not days). Finally, APIs can be layered and combined to meet specific requirements. API layering means taking an API you have already built and tailoring it to the specific needs of a specific consumer (or to your specific security requirements).
So where exactly is the happy medium? What’s the right way to solve your integration needs and – at the same time – provide for flexibility that will scale with your business? The answer is to focus on building APIs that are reusable, while creating integrations with your customized APIs.
A process for architectural success:
In our work here at MuleSoft, the process has 4 steps.
1. Look at data integration needs together
Naturally, there’s a strong tendency to want to solve integration problems one at a time. There is almost always a set of burning problems that need to be solved and the need feels so urgent that the challenges are looked at as separate problems with distinctly separate solutions (even if the tool you use to solve them is the same). It is really easy to get into the mindset of “everything will be all right if I just solve this problem” instead of the mindset of “how can I solve this problem and open the door to solving similar problems faster.”
Getting out of that mindset isn’t hard. Take time with stakeholders in the lines of business and look at the integration wish list. The list should include the systems involved, the specific data (objects) within those systems at a high level, and a quick summary of what the needs actually are. Without doing any deep dives, align the projects based on systems and data that they share in common and group those projects.
Next, call out the highest priority projects and look at what they have in common with lower priority projects. As an example, my team discovered that a quote-to-cash project we were working on had data in common with two other projects that needed to be completed. We then focused on creating APIs that provided the services and data we needed around those key systems that would not only work for these three projects, but would also suit a variety of needs in the future. By taking this approach, we ended up with several reusable APIs that could be used to make future projects move faster.
2. Represent your business data the way the business uses it
Now that you have a basic idea of attributes that projects have in common, spend some time mapping out how the data is best consumed within your teams. Figure out the structure of the data you are presenting and how that will be used by the consumer. Determine if any of your consumers should only be able to see a smaller set of the data you are presenting. This is really the part of the process where you figure out what you need and build the specifications for the data that provides value.
Now think about your business as a set of objects and processes. What are the objects your business runs on and where do they exist? Since you are designing an API around your business needs, you don’t need to worry as much about where the data is, but you do need to think more about the way you want to represent the data.
3. Prove your idea
Create your API specification using RAML in API Designer and create sample data that uses the structure you determined in the previous step. Then use the mocking service to make that API spec “live” and have the consumer or developer test it. This is, without comparison, the most exciting part of the process because – in just an hour or so – you can have something for your API’s consumer to test against. This allows for high levels of collaboration and incredibly fast results to get to a state that will provide the most benefit without spending development time. In this way, rather than discovering nuances as the API is being developed, the nuances are discovered through quick iterations. The end result is an API contract that has been tested.
4. Create, consume, and share your own APIs
Now that you’ve prototyped the API in RAML, it is time to put the live data from the underlying systems. One thing that is of great concern to the business is keeping live data safe; with Anypoint Platform, you also get control over who has access to the API (authentication) and therefore who has access to the data.
In addition, with the latest Anypoint Exchange feature of the platform, you can use this process to build a catalog of APIs that run your business. As you are developing against the API specification, you will discover that you can compose functionalities with your API by building other APIs underneath. As an example, customer data may come from two or three different systems each with their own native interface and each having the potential to present data that would be reusable by another project. This means that you can compose the API for the project by creating other APIs that deliver the data, and then use each of those APIs to compose the complete the picture you developed in step 3 above.
This approach helps bring the focus on the development of meaningful and high-value APIs within the business, while simultaneously reducing the tendency to over-architect. The key is keeping the focus on the reusability.
Want to try it for yourself? Check out the latest release of Anypoint Platform to see how all these features can work for your business.
