Hype hunters: Researchers put DevOps tools and tricks under microscope

Source – siliconangle.com

Can an academic approach to analytics predict the best efficiencies for developer operations?

Fact: Seventy-one percent of businesses have adopted some form of DevOps to streamline software development and deployment, according to VersionOne Inc.’s 2017 “State of Agile Report.” The most popular DevOps tool is container technology from Docker Inc., according to a separate report from RightScale Inc., with 35 percent of respondents employing the virtualized method for running distributed software applications.

Do statistics like these give businesses a roadmap to an effective DevOps strategy? No, they don’t, according to Nicole Forsgren (pictured), chief executive officer and chief scientist of DevOps Research and Assessment (DORA). A peek into companies’ playbooks can be “interesting,” Forsgren said; but who knows if they’re scoring or not? DORA’s academically rigorous research is different, because it predicts which DevOps methods will yield business outcomes.

“We don’t say things like, ’27 percent of the industry is using configuration management,’” Forsgren said. Instead, DORA studies finer details of how companies perform DevOps, and most importantly, it measures their success rates. “Those other reports cannot say what is predictive of improvement,” she added.

Forsgren spoke with Jeff Frick (@JeffFrick), host of the theCUBE, SiliconANGLE Media’s mobile livestreaming studio, during the recent PagerDuty Summit in San Francisco, California. (* Disclosure below.)

This week, theCUBE spotlights Nicole Forsgren in our Women in Tech feature.

Blind lead blind through software spree

“For years, everyone’s been like, ‘DevOps — what is it? Does it matter?’” Forsgren said. With iterative software development and failing fast now key competitive advantages for digital businesses, DevOps is now a buzzword.

“It’s about using technology and automation in conjunction with core values and practices and processes that we’ve adopted from the lean and agile movements,” Forsgren said. DORA teamed up with Puppet Labs Inc. several years ago to produce the State of DevOps Report. Together, they differentiate the DevOps tools and methods that produce results from those that don’t.

Examples of the shortfall of satisfactory results compared with adoption exist in many technology realms. According to Gartner Research Inc., 70 percent of big data initiatives fail. If a company were to simply research what others are using to monetize big data, then went out and bought the software, it would have a whopping 30 percent chance of success, according to Forsgren.

Likewise, despite DevOps’ high adoption rate, many organizations still struggle to realize its intended friction-free effects. Sixty-eight percent of DevOps practitioners in a Shippable. Inc. survey said poor connectivity in the DevOps toolchain was a major snafu. On the cultural end, 27 percent said that siloed departments hindered the streamlined collaboration DevOps requires.

DORA researchers cut through product hype with stringent standards of measuring DevOps success, according to Forsgren. The data it collects and incorporates must pass rigorous review; compelling anecdotes about a technology or an operational method don’t make the cut. The goal, always, is to accurately predict outcomes.

“It lets us say, ‘If we focus on core capabilities, they will predict organizations’ ability to develop and deliver quality software with both speed and stability,” Forsgren stated.

This scientific approach is wise, since the keys to productivity may lie where companies least expect them. Speaking to the theCUBE in June about the failure of big data initiatives, David Hseih, senior vice president of marketing at Qubole Inc., explained how DataOps — basically DevOps for big data — sometimes upends the tried-and-true: “There’s a bunch of sort of slightly counterintuitive behaviors — like decoupling your IT team from the end-users and treating the data team as sort of a platform building team — that can unleash amazing efficiencies,” he said.

For years, companies took for granted that the price of speed is a loss of stability, Forsgren pointed out, but DORA’s research debunked this. “We thought that in order to get stability, you had to slow down. It doesn’t show up anywhere in the data,” she said. “High performers get both.”

Better business through science

DORA does not exit solely for the advancement of knowledge — it also applies knowledge through assessments and consultations with businesses interested in DevOps.

“We offer an assessment using these strong, scientifically based measures that we have prepared and refined over four years of rigorous academic research,” Forsgren said. DORA collects data from all of the “doers” in an organization. These include, test and development teams, operations, quality assurance, information security, vendors, contractors, and consultants. DORA then takes these measurements and benchmarks them across an industry.

“I can — algorithmically —  tell you what your bottleneck is, what your constraint is, where you should start to accelerate your performance,” Forsgren said.

Companies who’ve consulted with DORA report “outsized” results, according to Forsgren. Capital One Financial Corp. is among DORA’s clients. “They did the assessment across over a dozen lines of business. We focused them on the right two capabilities [out of 20]. They saw a 20x improvement in deploy frequency in only two months,” she said.