How CircleCI speeds DevOps cycle times with CI/CD automation

Source:-diginomica.com

As enterprises go digital, they need to speed up DevOps cycle times – here’s how CI/CD automation toolmaker CircleCI is helping

The speed of software development is constantly accelerating — and the faster it goes, the more automation it needs. Modern DevOps teams make hundreds or even thousands of changes to their software code every day, and with the shift toward microservices architectures, there are more and more moving parts to co-ordinate. This simply isn’t sustainable without a highly automated Continuous Integration (CI) pipeline to build and test code and make sure it’s ready for delivery — which is where tools like CircleCI come in.

I recently caught up with Matt Wyman, then VP of Products at CircleCI, to find out more about how its customers use its tools, and where the rapidly rising demand for these tools is coming from. [Since we spoke, Wyman has started a new job, becoming VP of Product at Sauce Labs this month].

A big focus is on using automation to take “a lot of the toil” out of DevOps, he told me. The point of agile development in a DevOps environment is to deliver new software incrementally and get rapid feedback from users. What is known as the CI/CD pipeline has evolved to help speed this process:

Continuous integration (CI) is the practice of doing frequent builds that merge changes back into the main codeline, and running tests each time to catch any problems straight away.
Continuous delivery (CD) automates the release process so that those builds are then released into production on a regular cadence.
Continuous deployment (also CD) takes this a step further and just releases code to production as soon as it’s passed all the tests in CI.
In the past, the testing stage of CI has often been quite laborious, with user acceptance testing, integration testing and performance test labs each adding an extra burden. The automation tools were focused more on the deployment stage, making it possible to deliver code at the touch of a button. That might have been OK at a time when CD meant releasing code to production every couple of weeks, or at most every few days. But now that more and more organizations are on a roadmap towards continuous deployment, the whole pipeline has to run as fast as possible. That’s where CircleCI comes in, automating the testing of code and the workflows around delivery and deployment. “We’re the conveyor belt of the factory,” Wyman tells me.

CI/CD automation at SaaS construction vendor Procore
A good example of how this makes a difference is CircleCI customer Procore, a fast-growing unicorn that has become a leading SaaS provider of project management and productivity software for the US construction industry. Its team of now more than 300 software engineers was being held back by a test suite that took an hour to run, slowing issue resolution times and leaving people waiting while their code sat in queues. Using CircleCI’s cloud offering provided the speedy automation they needed to reduce the testing cycle to minutes and move to continuous deployment. Wyman explains:

CircleCI automatically splits up the test suite and what would be a linear hour plus, we can just do in a few minutes. If one [build] is going to fail, we get that back to the developer instantly.

One of Procore’s key selling points is the ability to replace paper and give people on construction sites immediate mobile access to the information they need. That means ensuring that the app is always available and therefore it can’t afford to be down for more than a few minutes at time. Wyman says:

If you’re going to replace paper, you need to be available in the way paper always is.

Therefore automating the pipeline is not just about time savings. A fast turnaround improves product stability and reduces the vendor’s risk exposure, says Lyle Espinola, Senior Software Engineer Team Lead at Procore:

The longer we take to ship things and the more process we put in place there, the more risky things become. We try to deploy as often as possible, while making it as safe as possible. Once we started moving to automatic deploys, we realized how much safer and faster it is.

Procore now closely monitors metrics such as how long jobs are queuing, and how much time they take to build and then test, so that the company can minimize how long it takes to deploy new code to production. This is a common characteristic among CircleCI customers, says Wyman:

If they start talking about measuring cycle time or cycle flow, we know for sure this is a customer that’s going to be very successful with us.

Across the CircleCI customer base
Not all customers have gone all the way to continuous deployment. CircleCI offers a workflow approval feature that pauses single or multiple jobs until an API call or human approver gives the go-ahead. Some organizations still have meetings to make the final go/no-go decision, says Wyman.

In the best cases, you automate away the decisioning and build test-deploy so that it can constantly be moving … The best organizations find ways to automate away those manual boundaries, but we have organizations that need them.

Customers come from three main avenues, he says. The biggest group simply sign up on line and start using the product, typically because they’re looking to speed up their CI/CD pipeline. A second category come through open source projects such as Angular, React and PyTorch. The third group are enterprises going through some form of transformation exercise that want to reliably accelerate their software development. There are more than 600,000 developers using the service and 30,000 organizations. Customers range from individual hobbyists to global brands such as Capital One, Condé Nast and Samsung. Whatever the industry, there’s one common theme, says Wyman.

These are organizations that need to empower their developers to move rapidly with confidence … They need to ensure that whatever their business application is, they can keep it stable and up at all times.

CircleCI aims to support three core principles that underpin good software practice, he says. The first is being able to deploy code changes with confidence. The second is maximizing the value that can move through the CI/CD pipeline, by reducing the time taken to conceive, build and then iterate. The third, particularly for SaaS vendors, is about delivering applications as a utility.

For a customer like Condé Nast, another benefit is being able to standardize methodologies across a global organization with 200 different engineering operations, each publishing content on their own stack, says Wyman. CircleCI has template configuration files that developers can use to easily provision new resources while working within policy guidelines set by the organization. When everyone has the tools in place to be able to publish and update code in minutes, each team is working at the same level.

Founded in 2011, CircleCI has raised $115.5 million in venture finance, $56 million of that in a D round closed last July. As a measure of the company’s growth, Wyman told me that when he joined CircleCI in mid 2017, the platform was running 120,000 builds a day. By late 2019 that number had ballooned to 1.2 million. With the proliferation of microservices and serverless functions, that need for automation is only going to carry on growing.

My take
The practice of DevOps — and its close sibling, software reliability engineering (SRE) ؏ is continuing to mature. A few years ago, infrastructure-as-code toolmakers such as Chef and Puppet were the big deal. Now the focus is shifting to automation of the entire build-test-deploy process. There’s a growing library of tools that enterprises need to harness as they continue their transformation into connected digital organizations, increasingly reliant on fast-moving software development to deliver business value.