The need for data-driven DevOps and how to start

Source – itwire.com

DevOps is, at heart, the application of programming techniques to infrastructure issues. Using tools like Puppet, virtual servers and other environments can be scripted, built, removed, and generally maintained with full version control and automation.

DevOps brings together IT departments, renewing focus on measurement, sharing and continuous feedback loops.

However, as systems become more complex, DevOps workflow becomes complex and opaque. The business wants solutions and features implemented. Programmers want to plan, code, test and release their applications, and along the way, ideally hear back from the business how the new code was received. Infrastructure teams want to manage capacity, ensure uptime, and deliver reliable, predictable and robust systems to users.

Perversely, the more advanced a business is on its DevOps journey, the greater the risk of failure, suffering potential pitfalls like impaired collaboration, increased waste, harder to scale, reduced agility, longer mean time to release and other related problems.

Here’s where data-driven DevOps comes in, says Mann.

Mann is an Australian living in Colorado, and responsible for spreading Splunk’s message globally, and learning from customers. He describes his role as a 50/50 split between incubating Splunk’s DevOps strategy and building products, and time spent on the road working with customers and learning what people really want.

Mann speaks with great enthusiasm about the vast amount of customer insight he has gained and how he has distilled it down to three key metrics, across developers, engineers, QA and management.

1. Cycle time: what is the velocity of delivery? How fast can the business go from an idea to having it in front of its customers, be they external or internal.

2. Quality. It’s possible to release quickly but have low quality and see your customers complain. To combat this, typically Scrum teams come together to make go/no-go decisions whether to push a release out. Yet, anecdotally, this is often based on guesswork. Mann says he knows a company with a six-page PowerPoint slide deck which includes 40 data points, filled in by e-mailed responses by 21 people. This was their release process.

However, much of this was guesswork. The data may exist, but it’s difficult to find. Some of the respondents are guessing, and some don’t have time to look into properly so say “yes” to avoid holding up the process.

“This isn’t the right way to release software,” Mann says. “It’s important to make data-driven decisions on code quality, the number of tests, code coverage, pass/fail rate on tests, whether security has been checked, if the libraries are authorised or unauthorised, and so on.”

3. Impact. What is the business impact on the customer? Once the software is released it’s important to get information back to the developers so they know if they’re doing the right thing. What is the business impact on the code you’ve written? You may be writing code and delivering fast, but if it is bad code with no business impact then what did you actually achieve? “Even if it’s high quality but has no business impact then why do it?” Mann asks.

“These metrics are like a three-legged chair,” Mann says. “App delivery falls over if you don’t have all three legs.”

Yet, knowing you should use metrics — cycle-time, quality and impact — is one thing. How do you put it into practice in your existing environment?

Fortunately, it’s easy, Mann says, and it doesn’t even have to cost you a cent to get started.

His own product, Splunk, plays a key role in DevOps, he says. “It’s the ultimate feedback loop, with systems thinking connected to the customer, helping understand your role in the entire system whether dev or QA security. Splunk provides a common feedback loop for everyone, with lots of people using the same data in different ways. Ops are interested in response time and performance as they keep uptime. Devs are interested in the same numbers to make their app more efficient and use resources more efficiently. There are so many data points that so many people in that data chain will share.”

Mann gives this advice for companies who would like to start with data-driven DevOps but don’t know where to begin:

1. Download the free trial of Splunk and sign up for the cloud service. You don’t need any on-site infrastructure.
2. Apply some straightforward techniques to get data out of systems. If you want to measure speed, for example, get data out of your repository of stories. If you use Jira there’s a Splunk add-in in the Atlassian marketplace.
3. Next, get data out of your release system like Jenkins. This gives you an endpoint.

Immediately, with little effort and no cost, you can begin using Splunk to see who in your team is working on new code and when it is being released. You gain insight into your velocity and cycle time right away. This is an important KPI.

You can take it from here, Mann explains, by ingesting from your QA tools like AppScan or Selenium. “Bring that in and see the quality of your releases. Track this over time and see who is doing good work and who might need training.”

A more mature statistic is business impact metrics and customer engagement, but this is also achievable, Mann says.

“You can use the Splunk HTTPS Event Collector, or HEC, to see what your users are doing on a web front-end whether they are using mobile or otherwise. Bring in business impact metrics by using Social Sentiment to measure Twitter and Facebook feedback,” he says.

“Put this in the dev room and show the current social sentiment on your releases. When people have a problem with a system they don’t typically phone you to log a ticket; they turn to others, then complain on Twitter. Measuring your social sentiment gives instant feedback to developers.”

“Trying these things out with the free Splunk trial will give your team value very quickly,” Mann says. “If you want to get into something deeper and more complex, anyone can look me up and make contact,” Mann says.

“With Splunk, you can make decisions on real-time data. Organisations iterating quickly can really benefit. Your developers and engineers can see the customer experience and business impact and customer engagement. This type of data-driven DevOps is really unique in this space,” Mann says, “but you get lots of data points. As a QA professional you want to know class quality and open incidents. This is information Ops wants to hear as well so they know what to prepare for, and the quality of the next release.”

Mann predicts machine learning will be the next big thing in data-driven DevOps. “The number one question from Operations is ‘when will this release come?’ followed by ‘and how many bugs does it have?’,” he says.

“With machine learning, you will know this in advance. You will know how long a certain team takes, the code complexity, the file complexity, and more. It’s not a use case I’ve seen yet, but I’m excited to see who wants to try it out.”