How to: Measure agility for enterprise IT

agility enterprise

After much hand wringing and collaborative sessions with teammates (most notably fellow Muleys Andrew Latham and Alison Jarris) and customers, a framework for how to think about agility has emerged that suggests a path about how to measure it (while not falling victim to Goodhart’s law).

For well over a decade, becoming more agile has been a top priority for businesses. Digital-born leaders like Amazon, Netflix, and Google have driven customer expectations sky high, putting the pressure on for enterprises to deliver. As of yet, here is still no industry standard definition for how to measure and track agility. While most professionals would agree that agility is primarily about speed, once people start attempting to put some robustness and quantification around it, the consensus starts to fragment.

Enterprise agility: What is it and how should we think about it?

Despite the lack of industry consensus on how — or even if — agility can be measured, my collaborative team generated at a simple to understand definition that has yet to fail any offered pressure test. While many enterprises reduce agility to being relevant in a technology systems context, one specific concept that drove alignment from both our collaborative team and every executive we shared it with was that agility for enterprises can be seen and felt outside of technology, such as processes, offerings, labor sourcing, and business and financial models. Based on insights from our work helping more than 1,400 customers digitally transform their organizations, we’ve come up with the following definition of Enterprise agility.

Enterprise agility (n): the ability to affect change within the enterprise quickly, easily, and without impactful error.

Let’s unpack each of the concepts in this definition:

  • Affect change: Agility isn’t specific to code changes. Change in this context can can be thought of in multiple ways (e.g., feature change, new product development, content change, culture change, go-to-market strategy change, business model change).
  • Quickly and easily: Quick and easy represent the amount of time and effort from concept to implementation of the change. Improvements here show up in ROI calculations given that there is a lower expenditure of labor and time with a corresponding faster payback period for investments.
  • Without impactful error: Failing fast is a virtue in an agile enterprise. Agility is in part measured by an enterprise’s ability to prevent errors from negatively impacting business KPIs.
  • Demonstrating agility: If an enterprise can affect change in ways that meet at least one of these four criteria, then the enterprise can be thought of as demonstrating agility: across multiple scopes, in shorter time frames, with lower labor effort needed, and with lower risk to business KPIs.

Agility through the loop: What is the OODA Loop?

OODA loop

Every so often, in operational circles, you are likely to hear the term “OODA Loop” which refers to an information processing cycle that crossed over from military usage to a host of other domains and is often viewed as a foundational concept for many of today’s common methods of achieving system level improvement including agile, LEAN, DevOps, and others.

The OODA loop concept and term were developed by US Air Force Colonel, John Boyd. Boyd modeled the information processing cycle for fighter pilots in the throes of combat by breaking it down to four interrelated and overlapping processes through which pilots continuously cycle:

Observation: the collection of data by means of the senses

Orientation: the analysis and synthesis of data to form one’s current mental perspective

Decision: the determination of a course of action based on one’s current mental perspective

Action: the physical playing-out of decisions

This decision cycle of observe, orient, decide, and act, drove extraordinary results in the military context and is now also often applied to understand commercial operations and learning processes across multiple industries. The OODA Loop approach favors agility over brute force when facing off against opponents in any endeavor where time is a critical factor.

An entity, whether an individual or an organization, that can process this cycle quickly, observing and reacting to unfolding events more rapidly than an opponent can thereby “get inside” the opponent’s decision cycle and gain the advantage.

Decision cycle

How does the OODA Loop relate to agility?

“The only sustainable competitive advantage is an organization’s ability to learn faster than the competition.” – Peter Senge, Society for Organizational Learning.

The notion of the OODA loop is embedded and referenced in a wide array of industry publications on lean, agile and DevOps including but not limited to the feedback to synthesis cycle of the OODA Loop is often compared to “Build-Measure-Learn” from Eric Ries, and PDCA from John Deming. Notable referencers of the OODA Loop as a basis for understanding enterprise agility include Eric Ries, Adrian Cockroft, and Microsoft.

The common thread in these oft-cited tools is the learning/change cycle experienced by people in a business/team context. The ability to complete the OODA Loop is akin to completing the learning/change cycle in an business setting and is widely considered to be the foundation for understanding and shortening cycle time to improve agility.

When we take a step back from agile as a methodology and look a the circumstances that made it popular (aside from everybody being frustrated with the myriad of shortcomings of waterfall), we can recognize that one of the core motivations was to help enterprises of all sizes become more responsive to changes in their environment. Aligning around a model that prized smaller units of work in a rapidly changing context, rather than “big bang” delivery where in-project change events are seen painful conflicts to be avoided at all costs, spurred transformational efforts whose effects are still being processed by teams and companies across the globe.

Reframing agility for the modern enterprise at scale

Scaling agility for modern enterprise

When we combine these two ideas we can start to visualize and measure agility as follows:

Agility, in the context of enterprise business, can be thought of as the ability to intentionally complete the OODA Loop and enact change for systems, processes, offerings, business models, labor, and more.

Within this conceptual frame, agility can be stratified into layers with each upper layer leveraging the operationalized capabilities in the layer beneath it.

The foundational layer “systems and infrastructure” is where teams typically apply Agile and DevOps methodologies. Sustained mastery of agility at this layer, enables organizations to focus efforts on smoothing and driving changes in enterprise “processes and offerings” which are often more difficult given the everyday impacts on both people and technology.

Just like the layer beneath it, sustained mastery of agility at the “process and offerings” layer, enables organizations to focus efforts on smoothing and driving changes in enterprise “business and financial models” which are amongst the hardest to achieve given the deep financial implications, business constraints and the ultimate test of market acceptance with a customer base that doesn’t take well to changes in their relationships with providers.

Industry-leading agility examples

As with any model, we can test its applicability through examples in the real world.

Agile business and financial models

  • Amazon – Original offering of online bookstore organically grew into other retail categories and then spawned successful highly scalable offerings across industries regardless of brand elasticity or specific expertise (e.g., retail, consumer electronics, streaming media, subscription services, voice assistants, cloud PaaS, advertising, healthcare. etc.).
  • Google – Launched as a search engine advertising platform targeted at consumers, Google has leveraged its expertise in scaling systems and infrastructure into multiple lines of business with wholly different operating models (e.g., enterprise SaaS, consumer electronics, telecommunications, navigation, streaming media, home security, etc.)

Agile process and offerings

  • Salesforce – One of the first SaaS companies, Salesforce has leveraged their mastery of a decoupled infrastructure to constantly evolve its processes and offerings to customers. This agility has enabled Salesforce to remain compelling and relevant to businesses of all sizes despite the hyperspeed cycles of change across the software industry.
  • Netflix – Practically owns their category as they have one of the most prolific infrastructure capabilities to scale product features/offerings to a very personalized set of capabilities to their customers. Netflix built a world class capability of systems and infrastructure management capability on top of AWS (Netflix can quickly deploy thousands of servers and terabytes of storage within minutes. Users can stream Netflix shows and movies from anywhere in the world, including on the web, on tablets, or on mobile devices) to enable their self-driven cannibalization of their mail-order DVD rental business. Netflix’s burgeoning mastery of the “Process & Offerings” layer of agility, is enabling their next level of transformation from media delivery into content development and has now become one of the largest and most successful media businesses in the world (evidenced by multiple global media companies selling off content development capabilities rather than compete with Netflix).

Agile systems and infrastructure

  • Etsy – Ability to on board and scale sellers, buyers and products – 2017, close to 33.4 million buyers had purchased goods through the Etsy ecosystem.
  • Facebook – Ability to scale users, capabilities, production offerings, analytics etc. to drive a very personalized experience – Approaching 2 billions users with ~1.4 billion daily users.
  • Walmart – Ability to add digital offering enabled through an extremely fast and nimble logistics supply chain. – They’re part of one of the largest and safest fleets, and every year they drive 700 million miles to make millions of deliveries to our stores and clubs.

From model to measurement: Specific KPIs for each layer of the pyramid

When viewed as separate conceptual layers, each segment of the new framework has direct, observable KPIs that can be tracked and measured as elaborated below:

Business and financial model KPIs:

  • New LOB time to market
  • Consumer perception of brand elasticity
  • Success rates of cross industry penetration
  • New product scaling time and cost

Process and offering KPIs:

  • New product time to market
  • Market test cycle time
  • Acquisition integration time and cost

Systems and infrastructure KPIs:

  • MTTR (Mean Time to Resolution)
  • Throughput (Story-points/Time, Function-points/time
  • QA Test cycle time
  • Batch size per deployment
  • Lead time for changes (commit to deploy)
  • Deployment frequency
  • Deployment success rate
  • System availability
  • Defect escape rate
  • MTBF (Mean Time Between Failure)

Making metrics work for you

Muleys across the globe work with enterprises at all stages of transformation and our customer success organization is constantly collecting and integrating learnings from each of these encounters. If you are working on driving agility or transformation efforts within your enterprise we’d love to hear from you regarding your efforts and how you are thinking about agility. We are always integrating and iterating and the larger community continues to play a key part as we all work towards our vision of the ubiquitous application network.

These learnings inform both the enterprise agility model described above, available in the hands-on guide to digital transformation.


We'd love to hear your opinion on this post