To reap the full benefits of DevOps — quicker releases of high-quality software — successful organizations know they must introduce continuous testing throughout their software delivery process. The test phase cannot exist as an independent step at the end of a development cycle.
Continuous integration and continuous delivery often spur this realization — these practices serve as the foundation for a successful DevOps approach. What IT leaders and their teams often don’t realize, however, is that the key to the proper introduction of continuous testing requires significant cultural and process changes.
Testing needs to happen in multiple places throughout development
The most important aspect of continuous testing is that automated testing happens throughout the development process. There’s real value in testing as you build or update an app — not after.
When your DevOps team tests code at each stage of the development process, it can address issues more efficiently, since the changes are fresh in the developer’s mind. This fast feedback loop increases the velocity of releases and improves the quality of the software you ship. On a business level, DevOps teams can pitch that continuous testing will result in higher revenues and more satisfied customers, because you’ll deliver new, high-quality features to your customers faster than your competitors.
Let’s break down continuous testing
In some advanced organizations, developers write all the tests.
While it’s important to test throughout the development pipeline, it’s equally essential for teams to maximize their velocity with the right test at the right time. Choose smaller sets of tests early in the pipeline — often called smoke tests — and a more comprehensive test suite later in the pipeline — often called a regression test. Continuous testing doesn’t mean testing 24/7 year-round. It just means you have a test for every step of the way.
A smoke test lets organizations find major bugs and regressions early and avoids the cost of a full regression test. The tests should check that key functionality works as expected across a few key platforms and should execute only a few of them to not affect developer productivity. One best practice to develop mobile applications is to run smoke tests against iOS simulators and Android emulators to speed execution time and minimize cost.
GROWINGAGILE
To get the most out of DevOps, organizations need to overhaul their testing process.
DevOps teams should run these more comprehensive tests less frequently. These tests should provide full regression and compatibility testing against a wide range of browsers and real iOS and Android devices to ensure a quality end-user experience. The combination of early smoke tests and later full regression tests lets organization cost-effectively speed application development.
Development needs to evolve
Development teams must stay close to the testing process. It’s the biggest cultural change that needs to occur, especially if an organization is early on its journey to continuous testing.
Developers often think that testing doesn’t fall within their job description and that all testing responsibilities fall to QA. To fix this mentality, organizations need to break down silos and adopt a team commitment to quality. In addition, developers must learn new skills — like understanding how to build apps that are easier to test via automation.
For example, if developers consistently include unique identifiers for objects, it eases both test authoring and test maintenance. Teams also need to embrace new development methodologies that integrate testing into the actual development process. Many advanced teams introduce behavior-driven development and test-driven development approaches where teams write their tests before they even get to application code. And in some advanced organizations, developers write all the tests.
Looking forward
The benefits of continuous testing have proven substantial, but to get there, both developers and QA professionals need to fundamentally change their approach to automated testing. While these changes are neither easy nor quick, they will drive faster growth and higher profitability at a business level.
GROWINGAGILE