DevOps leaders deliver software 200 times more frequently than their peers, study shows

Source: zdnet.com

While it seems everyone is looking to instill DevOps within their software delivery cycles these days, those that are leading the way are seeing exponentially higher performance than their peers.

That’s the word from a survey released by DevOps Research and Assessment (DORA) and Google Cloud, which identified 20 percent of enterprises that could be considered “elite performers” with DevOps, to the point where it is delivering impressive, and tangible, results. The report, authored by a team led by Dr. Nicole Forsgren, represents six years of research and data from over 31,000 professionals worldwide.  

Forsgren’s team finds these elite performers have 208 times more frequent software deployments, the study shows. The elite group routinely deploys on-demand and performs multiple deployments on a daily basis. By comparison, low performers reported deploying between only once a month at best. The range of software deployments went from 1,460 deploys per year (calculated as four deploys per day x 365 days) for the highest performers to seven deploys per year for low performers (average of 12 deploys and two deploys). 

“It’s worth noting that four deploys per day is a conservative estimate when comparing against companies such as CapitalOne that report deploying up to 50 times per day for a product, or companies such as Amazon, Google, and Netflix that deploy thousands of times per day, aggregated over the hundreds of services that comprise their production environments,” Forsgren’s team adds.

Elite performers report change lead times 106 times faster than low-end performers. The elites show lead times of less than one day — with change lead time measured as the time from code committed to having that code successfully deployed in production. In contrast, low performers required lead times between one month and six months, the study shows. 

Smaller organizations seem to be leading the pace of DevOps. For the first time, we found evidence that larger organizations (those with more than 5,000 employees) are lower performers than those with fewer than 5,000 employees. “Heavyweight process and controls, as well as tightly coupled architectures, are some of the reasons that result in slower speed and the associated instability,” the DORA/Google team states. 

The researchers outline how elite DevOps enterprises differ from lower achievers, and offer guidance on how to join this elite group. “Start with foundations: Basic automation (such as version control and automated testing), monitoring, clear change approval processes, and a healthy culture. Then identify your constraints to plan your path forward. This strategy works for those just beginning transformations as well as those who have been  optimizing for years. Focus resources on what is currently holding you back, then iterate: Identify constraints and choose the next target.” 

In addition, elite DevOps sites emphasize autonomy with their teams. “Empowered teams who make their own decisions about tools and implementations contribute to better software delivery performance,” the researchers find. “In this year’s research, we see that given the opportunity, high performers choose useful and usable tools, and these kinds of tools improve productivity.”

Cloud also plays a role in DevOps success. The highest performing DevOps teams were 24 times more likely than low performers to execute on the full capabilities of cloud. Elite performers also”In our survey, 80 percent of respondents said the primary application or service they supported was hosted on some kind of cloud platform. Using the NIST framework, we investigated the impact of essential practices on software delivery performance and, for the second year in a row, found that what really matters is how teams implement their cloud services, not just that they are using a cloud technology.