I am excited to announce that MuleSoft has released a new Anypoint Connector for Amazon Redshift, allowing easier integration of data warehouses into existing application networks. With this new Anypoint Connector, users can automate common business operations by integrating Amazon Redshift cloud data warehouse with other business applications like ERP systems, analytics services, and BI tools — such as Microsoft Power BI and Tableau.
Over the last few years, we’ve seen the data warehouse cement itself as a cornerstone piece of technology for an organization’s data and analytics strategy. As a central repository for structured and semi-structured data, it has made it easy for organizations to build applications and services that rely on fast and reliable access to large amounts of data. As the provider of the #1 integration and API platform, MuleSoft helps organizations load and extract data into and out of these databases — commonly known as data integration and ETL.
The rise of the cloud data warehouse
Amazon has led the charge in the development of the first widely-used cloud data warehouse, Amazon Redshift. As a cost-effective and more reliable alternative to managing a traditional on-premises data warehouse, Amazon Redshift adoption quickly picked up momentum. And, with new advances in technology driving down the cost of storing large data sets, businesses spun up analytics programs to derive insights using popular BI tools.
Increasingly, IT departments are moving their data into Redshift to easily expose data for analytics use cases and across other business applications. Examples include: consolidating eCommerce order data stored in other database services such as Amazon DynamoDB, prepping and cleaning log data streamed via AWS Kinesis Data Streams, or mirroring customer data in a CRM — kept in-sync by implementing a bi-directional sync. In all of these cases, the challenge becomes integrating the data across these systems in a consistent and reliable manner.
ETL/ELT as an approach to integration
Traditionally, large data sets have been consumed using the ETL approach, which stands for “extract, transform, load.” Data is extracted from a data source, transformed into the necessary format (such as a table with only specific values rather than the full data set) on a staging environment, and then loaded into a data warehouse for further dashboarding/analytics and visualization. ETL is useful when frequent updates to data are not needed, and users would like to save on computing costs by transforming small amounts of data in a staging server before pushing to the target. ETL is also most common when working with on-prem data storage solutions.
With cloud data warehouses like Amazon Redshift, a new trend known as ELT is emerging. This stands for “extract, load, transform” and allows users to directly transfer data from the source data warehouse to the destination tool before computing the transformations needed. This is most common with large amounts of data that may need frequent updates.
Anypoint Connector for Amazon Redshift
Any integration (ETL, ELT, or otherwise) always starts with connectivity to the source and target system. And many times, this connectivity is based on open database connectivity standards, such as JDBC (Java Database Connectivity) or ODBC (Open Database Connectivity). These standards are the foundational building blocks that database connectivity is based upon, but today’s developer doesn’t expect to get down and dirty configuring JDBC drivers. Luckily, AWS provides native connectivity between Redshift and many of their other cloud services. For connectivity outside of the AWS ecosystem, however, Anypoint Connectors are the best option.
The new Anypoint Connector for Amazon Redshift provides the easiest and fastest way to connect and integrate Amazon Redshift to other business applications across the enterprise. Anypoint Connectors provide a simple, centralized configuration experience with built-in support for common integration requirements such as pagination, batch processing, and data transformation. Anypoint users can also manage and standardize versions and authentication schemes across applications.
Now, with this release, AWS users can get more data into Redshift and expose the results of their data warehousing analytics in other parts of the business using the new Anypoint Connector for Amazon Redshift.
Getting started
The Anypoint Connector for Redshift is available now on Anypoint Exchange. To get started with the Amazon Redshift Connector, watch our demo or check out our latest developer tutorial.
For other resources on how to integrate AWS cloud services with MuleSoft, check out our AWS resource page
