Shift-Left Testing, Continuous Integration and the Mainframe

Source: devops.com

You may not think, then, that mainframes and shift-left testing have much in common. But you’d be wrong. Shifting testing to the left can deliver tremendous benefits, even in the context of mainframe applications.

Let us explain.

What Is Shift-Left Testing?

Shift-left testing refers to the practice of performing more tests earlier in the software development lifecycle.

Traditionally, most testing occurred just before software was deployed into production environments. Testing code at this stage is useful for ensuring it will behave as expected in production, but it comes with drawbacks.

The biggest drawback of traditional testing is problems are difficult to fix if they are not discovered until just before software is ready to be deployed. At that point, addressing a problem within the code requires determining where specifically the problem exists within the code, removing that code from the application and replacing it with updated code—all without breaking other parts of the application that depend on the original code.

Shift-left testing helps to address this problem by encouraging software delivery teams to perform tests earlier—typically, starting right after individual units of code are written. Being able to test earlier and more often is especially important in software delivery pipelines based on continuous integration/continuous delivery (CI/CD), where the goal is to roll out application changes on a nearly continuous basis.

Since bugs that aren’t detected until just before release can take a long time to fix, a lack of shift-left testing puts organizations at risk of not being able to achieve the main goal of CI/CD due to constant delays caused by software quality problems. In addition, shift-left testing reduces the amount of overall end-to-end testing you have to perform.

You Still Test on the Right

It’s important to note tests performed earlier in the pipeline don’t replace the testing done just before deployment. They complement them and reduce the chance of having problems go undetected until right before deployment. In this sense, the term shift-left can be a bit misleading; it’s not really about shifting all testing to the left of the delivery cycle, as much as it is starting tests earlier, while continuing to test on the right of the pipeline as well.

Shift-Left Testing and Continuous Integration on the Mainframe

Shifting testing to the left makes it easier to fix code problems within virtually any type of application delivery pipeline based on CI/CD. But it can deliver especially critical benefits for software delivery teams striving to bring mainframe applications up to speed with CI/CD.

There are a few primary reasons why. One is mainframe applications are usually written using older programming languages that were not designed to help developers avoid mistakes. This isn’t to say mainframe languages are especially prone to errors; they’re not. But compared to a language such as Python, which is designed to enforce good coding habits, a language like COBOL doesn’t exactly require programmers to be highly disciplined when they’re typing out their code. Thus, it’s especially important to be able to catch bugs earlier within code written in a mainframe language.

Another factor is mainframe applications are often part of legacy codebases. When you’re dealing with code and dependencies that are decades old in some cases, you’re at a greater risk of ending up with software quality problems, despite your best efforts. Here again, detecting those problems earlier rather than later means they can be fixed without interrupting the entire CI/CD process.

Finally—and most important—is the fact that in some cases, there is a large disconnect in terms of skill set and familiarity with mainframe code between software quality engineers and mainframe programmers.

Your typical QA engineer today didn’t grow up with z/OS and may have limited experience troubleshooting software problems. Given the QA team is typically responsible for running pre-deployment tests, this can pose a problem in the event that QA detects issues with a mainframe application that it has no idea how to troubleshoot, let alone fix. By shifting testing to the left, it’s easier to give developers faster feedback so that they can fix bugs more quickly.

How to Shift Mainframe Application Testing to the Left

If you thought mainframes had nothing to gain from the shift-left testing movement, you might also think there aren’t many tools out there to help you do shift-left testing for mainframes.

Fortunately, that’s not true at all. For unit testing, IBM offers Z Unit, a unit-testing tool for z/OS programs. Z Unit is included in Z Open Unit Test and IBM Developer for z Systems. Using these tools, Z developers can automatically generate unit tests for batch and CICS programs as part of their daily work. These tests can be stored in any standard SCM and can be executed independent of their environment.

You can even test mainframe apps on non-mainframe hardware using Z Development and Test Environment. The platform creates virtual Z environments on x86 systems so developers can write, build and test their mainframe applications on any standard commodity server or development box.

Conclusion

Whether your mainframe applications are decades old or have been written from scratch in just the past few years, there’s nothing stopping you from taking advantage of shift-left testing to help validate them. Doing so will make it easier to achieve CI/CD effectively for your mainframe application, which keeps your software delivery team and your end-users alike happier by reducing bugs and delays.