Continuous Testing: Solving the Biggest DevOps Bottleneck

Source – devops.com

Earlier this year, Computing Magazine conducted a fascinating study around the adoption and benefits of DevOps in its DevOps Review 2017 publication. The findings that stood out most fascinating to me where around quality assurance and testing.

In this survey, the question was asked, “Where are the main holdups in the software production process?” Of all the self-proclaimed DevOps practitioners, a whopping 63 percent said the biggest delays occurring in the QA/Test phase. In fact, this cited bottleneck was more than 2 times greater than the next proclaimed bottleneck, which they felt was in planning (29 percent).  In this same survey, another question asked, “What tends to lose out more in DevOps environments?” While 43 percent said, “Nothing,” nearly 30 percent of these DevOps practitioners said that quality was being sacrificed. When asked which area they wanted to focus on improving in the next year, over 40 percent opted for quality.

Why is it that quality has taken such a hit?  The study cited two issues: 1) Testing and QA is often siloed from the other parts of the organization, and 2) Testing tools haven’t matured much in the past five years.

Testing must become a cross-functional operation across the software development lifecycle (SDLC) spectrum, one that involves all teams. Agile developers and testing teams don’t have weeks to test, but hours. Testing hasn’t kept up because it hasn’t applied the methods and concepts of agile development have not been applied to testing and QA processes. By removing the constraints that force testing to the end of a development cycle, developers and QA teams can achieve accurate, automated testing from day one. They can create an integrated pipeline, capable of taking an idea from design to deployment, at speed, without compromising quality.

Open-source, cloud-based solutions can be the catalyst that brings the speed of agile to testing.  Here are a few examples.

Better In-Sprint API Testing: Since APIs have become the major component of applications, organizations are shifting their approach from starting testing when the user interface is complete to shifting the focus of testing at the API level.  But the hard thing is finding enough time in a development sprint to test them sufficiently.

Today’s applications are a collection of microservices and API calls. UI testing alone just won’t cut it. UI testing is very brittle, and whenever it changes, testing breaks. Testing needs to shift left into the development cycle—into sprints. But if you shift testing left to be a focus of the sprints, what kind of API testing can you really do? How comprehensive can they really be?  Creating those API tests has to be quick, and simple.

Dev and Test teams must utilize automated tools that not only generate API tests, but create an exhaustive set of test cases from the API, then execute those tests. These tools also need to support the most popular open-source testing frameworks allowing developers to run functional API tests as performance tests without any proprietary conversion issues. And, vice versa, developers must be able to take open-source performance tests and be able to run them as functional API tests.

Orchestrating Testing: A Continuous Delivery Pipeline Manager: “Faster” is becoming table stakes with automation.  The facts show that companies who have widespread agile adoption and are deploying more frequently with significantly reduced lead times. But, this too often leads to release failures, quality issues and disorder with multiple—if not hundreds—of applications rolling through the SDLC. Plus, things slow down significantly with time-consuming handoffs and rework in the release process. DevOps teams, who release features and fixes frequently while significantly reducing errors and manual processes, need toolchain-agnostic pipeline planning, orchestration and optimization solutions that enable them to eliminate spreadsheets, automate test processes and feedback, and continually improve the delivery of revenue-generating features to end-customers. New CD SaaS tools help make this orchestration process easy.

Performance and Functional Testing Using Open Source Frameworks: More and more organizations are trying to move from legacy testing tools. It’s not so much that these don’t adequately test, but they were never designed to meet the needs of agile development approaches. Rather, they were designed for releases that occurred two to three times per year. But that won’t cut it anymore.

And as developers begin to incorporate testing activities in the coding process, they need simple-to-use, open-source tools. Luckily, there are such tools like JMeter, Selenium, Gatling, Locust and others. There are even tools that help you convert from legacy platforms to open source technology with ease, and guides to transform existing functional tests created with other tools over to load tests.

These are just a few examples of how continuous testing eliminates the testing hurdle. With continuous testing, organizations can:

• Automatically convert user stories from requirements management tools like CA Agile Central or Atlassian JIRA into the minimum number of test cases required for complete code coverage, and being able to adapt automatically to changing business requirements.
• Reduce test cycle prep time by requesting test data from within requirements.
• Request, reserve, and create test data at any time, even from requirements phase.
• Virtualize APIs for better testing, while managing API access to developers.
• Institute easy API testing that can generate comprehensive test cases (positive, negative, edge case, and SQL injection) and execute those tests automatically.
• Democratize testing with SaaS-based testing tools so that developers can test early and test often.
• Speed testing by feeding test data directly and securely to virtual services.
• Use open-source testing tools such as JMeter, Selenium, Jenkins, Jenkins, Appium, Taurus, Gatling, Locust and more.
• Orchestrate the application delivery pipeline to promote releases from one environment to the next as soon as all the tests pass and enable faster feedback loops.

This is testing unleashed, going hand in hand with agile development to accelerate quality into production.