Last month the massive Heartbleed security vulnerability was exposed. Three weeks later a security flaw in Microsoft Internet Explorer was revealed. It seems as though every few months there is news of a security breach or vulnerability. As more and more business is done online, in the cloud and through SaaS providers, how can you be sure the applications you and your business use are safe? Using the Heartbleed vulnerability as a case study, this article will examine what went wrong, as well as what you should expect from a SaaS provider, before, during and after a security event.
What is Heartbleed?
Heartbleed is the commonly recognized name of an exposure identified in a critical Internet security software package called OpenSSL, the most common transmission encryption software package used by Internet servers worldwide. This vulnerability allows an attacker to craft keepalive messages in such a way as to force a server disclose its short-term memory space. Since a server's memory often contains personal, confidential information, such as user passwords or credit card numbers, the attacker could obtain that information. More severely, the server may also inadvertently disclose to the attacker its own private encryption key, which then allows the attacker to subsequently listen to all communications with that server, even without using the Heartbleed vulnerability. The nature of this attack is such that it leaves no traces, and is practically invisible to common detection mechanisms (although now that it has been exposed, signatures for it are becoming available for popular intrusion detection software).
Who was affected by Heartbleed?
Unfortunately, OpenSSL is used by about two thirds of Internet servers, and this exposure impacted the majority of them. Even companies that may themselves not be vulnerable – maybe because they were not using OpenSSL at all – were impacted because their overall deployment was impacted somewhere. A common example might involve the use of a Content Delivery Network (CDN) which terminates SSL sessions and is itself vulnerable.
Before a vulnerability is exposed, what preventative measures should a cloud company take?
When you use third parties entities to handle sensitive data on your or your company's behalf, you must ensure that they follow best security practices in order to reduce the risk of exposure. It is important to validate that such third parties have a formal and comprehensive security program in place assigned to an executive stakeholder. The program should include policies to address the protection of sensitive data throughout its life cycle using a risk-based approach, include a broad range of security controls, incorporate thorough internal and external testing (e.g. penetration tests) of products and service delivery environments, and utilize external third party audits to validate control effectiveness and the strength of the overall program.
I just read about another Heartbleed-like issues, what can I expect from SaaS companies?
In the case of a potential exposure like Heartbleed, you should expect all vendors you work with communicate early and often. Almost immediately after a vendor is aware of an issue, they should communicate to customers, via email and updates to their trust site. This communication should make you aware of the issue, the extent of the issue, and their plan (even rudimentary) to remedy the situation. This should let you know of any immediate action you may need to take, including not using an impacted system or shutting systems down as a precaution. From the vendor's perspective, we want to make sure customers know we are aware and are on it. We never want a customer to read about something on TechCrunch before they hear it from us.
During the course of the event, the vendors you work with should be working 24/7 to resolve or patch the issue and you should continue to receive regular updates, again, via email and updates to their trust site. Vendors should let you know the progress they are making, any action you should take, and provide estimated time to resolution. You should be told if it is a vulnerability, meaning there is the potential for a malicious attack, or if there was a breach, meaning a malicious attack occurred, and whether or not your information was affected. In these cases, the detail is reassuring. It shows you that the vendor has depth of knowledge relating to the event and is prepared to take action. Be aware of anything dismissive or overly vague.
When the vendors you work with have adequately addressed the breach or vulnerability, expect a resolution notice. The resolution notice should include an explanation for why they were vulnerable, whether the negative event is due to negligence or oversight on their part, or is a broad, industry-wide event. It should also detail the steps taken to remedy the issue and provide reassurance that you and your company's information is once again secure and detail any action you need to take, such as resetting passwords. Lastly, the company should be continually analyzing areas of improvement. Simply addressing the issue at hand is not sufficient; companies should be performing a post-incident analysis to ensure that their incident response process is constantly evolving to effectively deal with new and emerging threats.
Let's get back to Heartbleed…
Heartbleed was arguably “the worst vulnerability found (at least in terms of its potential impact) since commercial traffic began to flow on the internet,” at least according to Joseph Steinberg of Forbes. Any company providing cloud services through Amazon Web Services, like MuleSoft, was impacted. With that in mind, let's take a look at the timeline of events in the MuleSoft response:
- April 7th, afternoon: Story breaks and the MuleSoft product, operations and marketing teams works through the night to determine the scope of the problem and draft responses.
- April 8th, early morning: MuleSoft notifies potentially impacted customers via email and posts an update to our trust site, while the operations team continues to patch the vulnerability.
- April 8th, mid-morning: MuleSoft sends an update to potentially impacted customers, letting them know that Amazon had released a fix, and we had resolved the issue, but would be rotating all certificates.
- April 8th, early evening: Heartbleed issue is resolved and all certificates have been rotated.
- April 8th, late evening: A series of emails were sent to all potentially impacted customers, letting them know that there was complete resolution and no MuleSoft cloud services remained vulnerable, as well as any action that needed to be taken on their part, such as resetting passwords.
Within 30 hours of the first article being published, not only were all potentially impacted customers notified, but the problem had been solved. Contrast that with vendors who are still struggling to grasp the depth of the vulnerability and the straggling communications you see (or have yet to see) in your inbox. Based on our quick and complete response, we've actually received thank you emails and tweets from customers. Here at MuleSoft, we don't think this type of response should be considered out of the ordinary–this is what you as a customer should expect from all your SaaS vendors.
