The Critical Difference: Continuous Quality versus Continuous Testing

Source – cio.com

The rapid adoption of the Agile method and the software development and operations (DevOps) approach isn’t surprising. As anyone in our industry knows, the pressure to conduct system testing and quality with both speed and accuracy is a daily struggle. Employing Agile methodologies allows us to better address business and customer needs, while breaking through the challenges and limitations of traditional waterfall development. As an extension of Agile, DevOps has emerged as a solution to a fundamental necessity, based on a simple philosophy: business works best when coordination and collaboration are present. It would seem the picture of perfection is complete — but, of course, we know it isn’t.

Now, we see more efficient IT teams who can accommodate complex business scenarios. Organizations respond more rapidly to market changes and more swiftly to the constant change in customer expectations and demand for services. Nearly 10 years since DevOps was introduced, products are getting to market faster. But while Agile and DevOps has sped up, industry leaders are only at the tip of the iceberg when it comes to addressing the issue of continuous quality. Many customers have found they can speed up getting software out, but at what cost?  Defects in production and security problems with services have made front page news.

How Did We Get Here?

We all know the history, but let’s take a minute to recap. We designed better technologies, automated manual systems and — as an evolution of Agile — we turned our focus to continuous integration and continuous delivery. With continuous integration, we identify defects early in the development phase when issues are usually less complex and easy to resolve. Continuous delivery takes continuous integration to the next level by extending Agile practice beyond development in the DevOps cycle. Continuous delivery makes it possible for all teams – testing, support and development – to work collaboratively to streamline the build, test and release cycles. Successful continuous delivery solves for many of the pre-DevOps pitfalls that occur in the software development life cycle (SDLC), and enables shortened iterations, mitigated risks, reduced costs, and accelerated time to market by frequent releases.

Continuous Testing is Important But Not Enough

As demand is placed on superior customer experiences, we’ve introduced continuous testing as part of the software delivery pipeline. Our ability to obtain immediate, real-time feedback on business risks while providing continuity across business units is now a critical phase of the continuous delivery process. We recognize all software has defects, bugs or glitches. So, continuous testing is fundamentally necessary. It’s something we must do. But I would argue that it’s to the detriment of our industry that continuous testing, manual, automated or otherwise, still doesn’t alone ensure quality.

Higher Standards Start with Continuous Quality

The original design of DevOps called for high-quality, updated software that is placed in the hands of the user more quickly. If we expect the delivery of highly functional systems that consider the user experience not as simply an aspirational goal, but as an industry standard, we either need a better definition of continuous testing, or we need to raise our game and make a collective shift to continuous quality.

I believe this will be one of the most important factors in the next phase of the DevOps evolution. The scope of continuous quality is higher reaching than continuous testing. Any definition we adopt must take the following factors into consideration:

• a deep understanding of business processes in production
• upfront promotion of the need for objective data from the business
• recognition of both internal and external customers for quality metrics
• prioritization of continuous quality from day one
• including quality into the agile development methodology (don’t “throw it over the wall”)
• tracking of project health should include quality at each phase of the SDLC
• end-to-end business process validation not just single service / component quality
• performance monitoring and optimization throughout the development and production for feedback

Without continuous quality development, speed and testing may fail to still meet the overall expectations of the business. Continuous testing does not translate into continuous quality. I have seen too many customers who develop in agile but still “throw the application over the wall” to test the same way as waterfall.  This causes problems in the deliverable to business users. Quality needs to be understood at the beginning and measured throughout the SDLC for business outcomes to be fully realized.  If the core business needs are not represented in the validation of the delivered application, it does not matter if the software works or not.