The Role of QA Testing in Continuous Delivery

Source – rainforestqa.com

With more traditional develop-first-then-test models, the rate at which you can merge or ship codes depends on how fast you can test it. Every new or updated feature must go through the cycle. As a result, you can only move as fast as your QA process allows. That’s why in order to do continuous delivery, you need to adopt an equally continuous QA process.

How QA Impacts Continuous Delivery

It’s much cheaper and faster to fix bugs that are recently introduced, since the developers will still have the context of that particular bit of code in their memory. Faster recall > faster fix > less dev time > less expensive! The consequences of not doing QA is that your customer becomes your QA team, and what you save in time and headcount during development you will “pay” with customer churn.

The role of QA in continuous delivery is to accelerate deployment, rather than hold it back. Aside from the fact that it’s cheaper to do QA continuously, it’s also much faster. Continuous testing creates a “safety net” that increases dev confidence in making changes and shipping updates. There’s a much less prominent risk of shipping code, because if something breaks they can rely on QA to catch it.

How to Think about QA for Continuous Delivery

Don’t try to test everything. Don’t try to cover everything up front, just make sure critical paths are covered (shopping cart checkout, landing page, etc.). Focus on having a solid base of unit tests, functional tests, and anything automatable or deterministic to keep your testing as fast and efficient as possible. At Rainforest, we run unit tests in feature branches, and use Rainforest tests in develop which serves as our functional/end-to-end/QA tests.

Build regression tests incrementally. When making a bug fix, add a test for that bug. Adding this to your process will build up the regression suite over time in the areas you need it most.

Don’t rely on manual testing for anything that you don’t have to. Use automation and crowdtesting to offload things that don’t have to be tested manually. These are significantly faster than testing manually, which will speed things up and help you recover time from QA activities.

Continuous Testing in Action: The Role of QA for Rainforest’s Releases

Here at Rainforest, we have a backend and a frontend component, and each has their own release pipeline. Both have the same git workflow:

  1. Create feature branch
  2. Merge feature branch to develop → triggers deploy to staging and QA
    1. For backend, this also triggers a RF run on a small (~10 tests) test suite. Serves as a smoke test to make sure critical functionalities are not broken
    2. For frontend, we only trigger when we intend to ship to prod. The test suite here is much bigger, more coverage.
  3. If tests pass, merge to master → triggers deploy to production

Our co-founder and CTO Russ recently published the Continuous Testing Manifesto, which outlines how to structure a QA strategy to keep up with development. Check it out if you’d like to dive deeper into how you can apply a continuous testing strategy to your workflow.