Unleash the power of your application network and Serverless Computing to transform, innovate, and stay ahead of your competition.
In the previous post, we covered how a serverless-first mantra offers a mindset focused on “don’t lift and shift — transform” and how serverless and SaaS are growing into the de-facto standard for delivering business software. Further, I highlight why we should focus on low Total Cost of Evolution (TCE) rather than only low Total Cost of Ownership (TCO) for our solutions. This is where we find business value by combining serverless (AWS Lambda and the like) and MuleSoft’s Anypoint Platform.
What about a real-world example?
In Australia, the electricity supply is regulated and delivered by licensed market participants with various roles that include retailers (that citizens purchase electricity from), network service providers (that maintain the poles and wires), generators (physical power plants and eventually “virtual” ones), meter data providers (metering services), and a number of other market roles.
Since the introduction of Advanced Meter Infrastructure (AMI or Smart Meters) sharing and processing of smart interval meter data has become a key part of the market and participant systems. Interval Meter Data is collected, validated, substituted, aggregated, and finally transformed to/from the Australian aseXML format to share between market and participants via the B2B e-Hub.
Let’s look at how we tackled this interval meter data processing 10 years ago with the available state-of-the-art integration platform and server-based technologies, as well as how that can be tackled today with the latest generation of iPaaS and serverless technology. The example centers around process production metering data for 1 million smart meters, with four registers each and 15min interval data, in a benchmarking exercise as a technology proof of concept.
Meter data processing (MDP) example comparison:
| 10 years ago: | Today: | |
| MDP for | 1 million smart meters | 1 million smart meters |
| Technology | Server-based and SOA | Serverless and iPaaS |
| Future Reuse | Low (design focused to meet required performance at some expense to reuse and flexibility) | High (design focused on performance and re-use via API-led approach) |
| Hardware | Sun Enterprise M9000 Server with Enterprise SAN | Not required |
| Software | Sun JCAPS Integration Platform Oracle Database utilisingOracle External Table, Oracle Table Function, Oracle Merge, Partitioning, and Oracle Stored Procedures | MuleSoft Anypoint PlatformAWS Lambda and DynamoDB WCU |
| Time to deliver | 9 weeks (not including infrastructure set-up) | 5 weeks (no infrastructure to setup) |
| Size of team | 6 (+ part-time delivery manager) | 2 (+ part-time delivery manager) |
| Performance | Processed in 60 minutes | Processed in 10 minutes |
| Costs | $$$$$ (high upfront software licensing and upfront infrastructure costs, and on-going maintenance costs for both) | $ (On-going subscription and pay for use costs, much lower implementation cost and duration) |
One of the reasons this is a good use case for an API-led and AWS Lambda solution is that meter data tends to be processed around the same time each day with the corresponding spike of processing and then used throughout the organization for many use cases, including sharing the meter data consumption detail with consumers. There is in fact a market obligation that meter reads for the previous day are sent and received by 6 am between market participants. This tends to lead to a spike of processing around 6 am, which is suited well for AWS-Lambda/serverless as users pay for what they consume and avoid any idle compute time that is common in server-based technologies. Furthermore, Anypoint API Community Manager (ACM) can be utilized to share that meter data via APIs with various internal and external developers and consumers, to foster, support, and grow a community around them aligned to your business outcomes.
Both benchmark designs are highly performant and were innovative for their time, especially in light of the original comparison legacy system that was only able to handle the processing of 15,000 meters in one hour and constraint to scaling to a maximum of around 50,000 meters. The daily AWS cost is around $25 for AWS Lambda and DynamoDB WCU for the meter data processing for one run for 1 million meters, and storage costs depend on how long the data is retained in Dynamo. For 1 million meters the cost is around $1.5K per month for one year of data. If we compare for 15,000 meters this would equate to around $1 for one day meter data processing and less than $25 per month for storage in DynamoDB for one year of data!
Although both solutions provide for the required meter data processing the overall total cost of ownership (TCO) and Total Cost of Evolution (TCE) is much lower and performance, ease of change, support for on-going innovation and reuse is much higher with the serverless and iPaaS approach. There are other good use cases for the combination of MuleSoft Anypoint Platform and AWS Lambda that would have different flavors and if you have any scenarios you would like to highlight please do share in comments below.
For more information on MuleSoft Anypoint Platform, Cloudhub, API Community Manager, and API-led connectivity download our white paper.
