Did you know that organizations generate 27% of their revenue from APIs and related implementations, but only 18% of organizations say they can integrate end-user experiences? (Source: MuleSoft 2021 Connectivity Benchmark Report). With a growing number of demands and expectations both internally and externally, connecting applications in a seamless and scalable way continues to be a major challenge for organizations on the path of digital transformation. Simultaneously, new classes of middleware technology emerge at a rapid scale to address application connectivity problems. Service oriented architecture (SOA), enterprise service bus (ESB), ETL, iPaas, virtualization, APIs, Streaming, and robotic process automation (RPA) are few examples of such classes of technologies and architecture styles.
With so many business applications, data types, and middleware technologies, IT architects are often faced with tough design decisions to deliver the most efficient integration solution that meets business requirements in a cost-efficient and timely manner. In this series of blogs, we will provide guidance on decomposing integration problems into their most basic and fundamental integration use cases. We further introduce the technologies and integration patterns that can be used to deliver those use cases.
Common integration technology classes
In the year 2021, technology has been at the forefront for meeting many business demands, including applications connectivity. There are a variety of technology options that fulfill this space. In this blog, we will look at the technologies holistically as a group or class that share similar architecture and behavioral standards.
As we describe the integration use cases subsequently, we will use this list to identify the relevant technology classes.
Foundational integration use cases
While there are many business scenarios for application and data integration, we believe that every business scenario can be broken down into the following unique and fundamental integration use cases. These use cases are building blocks and can be combined to develop more involved and complex use cases.
#1 Aggregation of data
Businesses consolidate data from multiple sources for semantic completeness and contextualization purposes. Within these use cases, data from multiple applications are copied into a central location for further analysis and processing, for example, data warehouses, data lakes, or dashboards.
Some of the common examples of business scenarios are:
- Building a single view of customer
- Migrating data from on-prem applications to a cloud-based data warehouse
- Consolidating sensor data and ERP data for advanced analytics on usage patterns
- Preparation of executive sales dashboards
#2 Streaming data ingestion
Most organizations have modern devices and objects that are capable of instrumenting and generating data to reflect the current or past state of the device. This data is vital for an organization’s operations and needs to be ingested into a platform for downstream consumption and usage. Typically, such devices can generate data either at a slow pace (every minute or hour) or at a fast pace (sub-milli-second).
Some of the common examples of the business scenarios are:
- Processing IoT sensor data at edge locations
- Real-time consolidation of inventory across stores
- Consolidating user click data on retailer websites
- Collecting and analyzing machine vibration data to predict failure
#3 Data sync between multiple applications
For a given piece of data, there is generally a well-defined system of record where its lifecycle is managed. However, that data is often required within other systems to complete the business transactions within those applications. The data changes faster in the system of record and is required near real-time in the consuming applications. Consuming applications may further enrich or update certain information. More often than not, these consuming systems should not act as systems of record.
Some of the common examples of the business scenarios are:
- Upon creation of a new customer account, syncing the account details to ERP, CRM, and Financial applications. Updates to customer accounts can be made in the CRM applications and those updates are sent back to the system of record.
- Synchronizing customer data with the customer support applications so your front-line agents can better serve your customers.
#4 Sharing data with external partners
Any given organization typically works with multiple external business partners, such as suppliers, contractors, government agencies, etc. These business partners are organizations on their own with different systems, processes, and applications. Data exchange between your organization and the partner organization, however, is critical to complete business transactions.
Some of the common examples of the business scenarios are:
- Sharing purchase orders with your suppliers
- Sharing invoices for payment purposes with your suppliers
- Sending tax data with the government
- Sharing machine’s operating parameters with the regulatory organization
#5 Broadcasting events
Events occur all the time within an organization. New hires, machine shutdowns or failures, expense approvals, payment issues, website traffic exceeding a certain threshold – all of these are classic examples of meaningful events that have downstream impacts and demand further actions. Identifying, capturing, and notifying these events to downstream consumers are critical aspects in these use cases.
Some of the common examples of the business scenarios are:
- Tracking a consignment’s shipment status
- Sending notifications for customer order fulfillment status
- Tracking the inventory updates in real-time
- Experiencing a retailer website crash after a surge in user clicks
#6 Bulk or batch data movement
There is an ever-growing need to see data as quickly as possible, however, there are valid scenarios where delays in presenting information is acceptable or presenting information in real-time is not required. Such scenarios optimize and provide better utilization of the resources usage in a way that constant or online availability of processing resources is not required.
Some of the common examples of the business scenarios are:
- One-time migration of customer accounts from an old CRM to Salesforce
- Periodic migration of invoices from ERP to payment systems
- Nightly reconciliation and analysis of financial transactions from multiple applications
- Daily summarization of shipping and receiving transactions
#7 Synchronized data transfers or process triggers
Synchronous data transfers are scenarios where the receiver of the information makes a request for data transfer and waits until the sender has transmitted the requested information. During the time the data is not received, any further processing at the receiver is suspended. Such scenarios are valid when there are strong dependencies between the requested data and subsequent transactions at the receiver.
Some of the common examples of the business scenarios are:
- Retrieving last year’s sales report from the CRM system
- Searching for a customer’s invoice within the ERP system
- Checking a prepaid balance for your cell phone SIM card
- Retrieving a patient’s prescription record before searching for medicine availability
#8 Asynchronous (fire and forget) data transfer or process triggers
Asynchronous data transfer or process triggers are scenarios where the data is sent to a target without an expectation of acknowledgment or confirmation back to the source for success or failure of transfer or process trigger. The source continues subsequent activities are sending the data or command to trigger the process. The consumers of source applications do not experience any wait time as a result of the data transfer of processing.
Some of the common examples of the business scenarios are:
- Sending promotional texts or emails to prospective customers
- Triggering an expense approval workflow
- Processing of your insurance claim receipts
#9 Orchestration and data processing
Significant semantic clarity is built when data from multiple sources are brought together and orchestrated. Orchestration of data provides additional context or clarity. In this use case, a layer of abstraction is developed between the source and target where deeper business logic is embedded. Often the orchestration use cases are heavily tied and specific to the underlying business domains.
Some of the common examples of the business scenarios are:
- Activating a customer account – credit check, background verification, employment validation, etc.
- Processing a new hire employee – salary account setup, IT assets delivery, work locations, etc.
#10 User interface integration and mashups
Integration is often thought of as the transfer of data among applications for machine-level consumption, as described in previous use cases, however, presentation of information in a consolidated format for human consumption is a widely known use case. Information is aggregated in a central location for human consumption.
Some of the common examples of the business scenarios are:
- Portals, Intranets
- Public Websites
- Mobile apps
Delivering end-to-end business processes / use cases
The foundation integration use cases described in the previous section are not exclusive to each other. Rather, we often see that they complement each other and serve as primitive building blocks to deliver more complex business solutions. These use cases will often be chained together to deliver an overall end-to-end business solution or process
For example,
- Sending a purchase order to your external supplier after management approval is complete
- Searching customer record details in CRM and subsequently orchestrating the marketing information that is tailored to the customer
Enabling business use cases with Anypoint Platform
While multiple technology classes are available for meeting your integration use cases, Mulesoft’s Anypoint Platform is a leader within multiple of these technology classes. These technologies work in unison to provide full lifecycle management for the APIs and Integration solutions.
Login to Anypoint Platform or sign up for a free trial to begin enabling reuse of your APIs and integrations.
