Change is the Focus: QA in the DevOps Era

Source:- capgemini.com

There is a popular business adage that states, “Fast, Good, Cheap: pick any two”. But what if you want it all?  I definitely want to ensure good quality, in little time and at the lowest cost. Therein lies the challenge. Throughout the product development lifecycle, we face bottlenecks that often prevent us from having it all. Today, Agile and DevOps introduce timeboxed development and continuous integration, which means shorter development cycles and more frequent releases. This results in continually changing software that poses unique challenges in quality validation.

What are these challenges?

Regression testing for incremental product release

In periodic release cycles, the development time and effort for software modification is relatively small compared to that spent on regression testing. Selecting regression test cases for incremental code changes or last-minute production defect fixes can be a daunting task. The QA manager generally selects the test cases based on guidelines and best practices. But this selection of a subset is not scientifically backed; it can be less reliable and prone to human error. To ensure there is no regression introduced, QA generally maximizes the test execution. This takes a while, may not be relevant, and in most cases is non-productive.

Broken code during development process

A development engineer working on a module is often so consumed in the work at hand that he may overlook the impact of changes on the larger product software and the completeness of unit test automation and testing. In short, the developer is not aware of which additional tests to execute based on the impact of the changes on the product software. There is lack of mechanism to ensure the effectiveness of test automation prior to code check-in. The result?  Defects landing in regression.

Inadequate shift left testing

In Agile, the focus is on reducing delivery time while still improving quality. But in the real arena, test teams are subjected to work on a stringent timeline to meet the delivery schedules without compromising on quality. It is difficult to optimize both effort and cost, and this strain impacts the adoption of shift left testing.

Lack of test effectiveness assessment

In a complex project with a large code base and test footprints from multiple test frameworks, it is hard to manage test assets and assess the test effectiveness of the entire test landscape. It is nearly impossible to assess the overall test quality by considering the lines of code or functions tested, the errors detected, redundant test cases, and gaps discovered in the test coverage across multiple languages and frameworks.

It’s not about best practices

Though there are a handful of models and best practices to help guide the testing process and overcome certain challenges, they can be intellectually challenging and time consuming. What if we had an intelligent tool that helped us make better decisions based on software change impact analysis?

I would like to introduce QTRON, a test optimization solution that allows one to achieve advanced automation in quality verification at a previously unachievable rate.

QTRON empowers:

§  Developers to uncover defects early in the development cycle

§  Quality engineers to test only the appropriate scenarios based on code changes

§  Management teams to plan product development and delivery based on impact of changes and test efforts

Change is the focus

QTRON’s intelligent data driven algorithm identifies and selects the most relevant test cases for execution based on code changes and the effectiveness of test cases. The architecture is based on three principles:

1.     High automation percentage for product testing

2.     Granularity of code coverage

3.     Mapping of lines of code to test cases in a repeatable manner

The QTRON engine initializes by parsing the code coverage output data to create a mapping of test cases and impacted lines of code. The data carries the intelligence to determine test case effectiveness based on a proprietary algorithm. The user interface presents various features and dashboards related to test case advice and effectiveness.

Figure 1 High Level Solution Block Diagram

QTRON is designed as a plugin-based architecture to ensure future ecosystem support. It is integrated with bug tracking tools, source control management systems, code coverage tools, test management frameworks, and CI tools.

DevOps takes center stage

With DevOps taking center stage and blurring the lines between development and QA, one thing is for sure: there is a clear gap in how we drive quality verification and this gap must be bridged. QTRON is a uniquely positioned solution with clear purpose: to use a repeatable, algorithmic approach to assess test effectiveness and applicable tests based on a single line of code change. The result is a precise and accelerated quality verification process for developers and testers alike.