CI/CD pipelines are expanding

Source:-sdtimes.com

CI/CD pipelines are evolving as organizations identify opportunities to improve release velocity and as the industry considers what CI/CD pipelines should look like in the first place.

Amalgam Insights recently released “The 2020 Guide to Continuous Integration and Continuous Delivery: Process, Projects and Products.” In it, report author and research fellow Tom Petrocelli explains what basic and extended CI/CD pipelines involve.

The “basic” CI/CD pipeline includes five processes, which are: merge, build, test, package and deploy. All of these are individually defined so readers have a common reference point. The basic pipeline includes sub-pipelines associated with each step, such as moving artifacts from a build into a repository.

RELATED CONTENT:
The Continuous Delivery Foundation advances CI/CD
CI/CD success requires a sound approach
How some organizations are implementing CI/CD

An extended pipeline has twice as many processes (10 total). It includes plan, code, merge, build, test, security scan, package, artifact repository, deploy, and monitor. Although the flow in both diagrams appears linear (and the report notes that the term “pipeline” suggests a linear process), loopbacks to earlier points in the workflow can occur when failures happen or a security scan indicates a threat.

“The standard pipeline is just the basic stuff. Once my code has been written, how do I get it to release? There are other non-tooling stops along the way because some pipelines will deposit you in UAT or some kind of production staging area but overall the idea is to get from point A to point B,” said Petrocelli. “When you think about an extended tool chain – which is where a lot of the vendors are going – you start to bring in other things like monitoring.”

By extending the pipeline, teams are able to connect and automate more of the total software development process, though there may still be manual steps along the way.

“Right now, CI/CD really addresses one small piece of the total software development process, but there’s other stuff that needs to be integrated into it,” said Petrocelli.

Hasan Yasar has a similar view. He’s technical manager of the Secure Lifecycle Solutions group in the CERT Division of the Software Engineering Institute at Carnegie Mellon University.

“The point of a pipeline is basically to help build an application, test the application, deliver the application, and deploy it into production. A pipeline is not just having continuous integration servers or deployment tools and techniques, but covering the full life cycle,” said Yasar.

By full life cycle he means from the inception phase through build, test, delivery and deployment, as well as monitoring the production environment.

As far as tooling, Yasar said a pipeline should include an issue tracking system, a build system, an integration environment, a communications system such as chat, and monitoring systems that can monitor the progress of various development environments. In the center of all that should be a repository management system. That system should be connected to all the other systems and include all artifacts including infrastructure as code. The central repository helps facilitate communication and collaboration and ensures that all stakeholders (not just developers) have access to the artifacts. All the tools should be integrated, of course.

Whose tools?
End-to-end solutions are emerging alongside the point solutions. The Amalgam Insights report includes a representative sample of some of the vendors moving toward end-to-end solutions. They include Anaxi, Atlassian, AWS CodePipeline, CircleCI, CloudBees, GitHub, GitLab, Google, Oracle, Pivotal (now officially part of VMware) and Red Hat.

The end-to-end solutions are integrated, which saves customers the hassle of cobbling together point solutions.

“Complexity works against you when you want to achieve higher velocity,” said Amalgam Insights’ Petrocelli. “But the broader effect is without these tools, it’s going to be hard to have a microservices architecture. You’re not going to get what you want to get out of it, which is faster and more frequent updates. You’re not going to be able to do those continuous deployments without automated tools.”

“Microservices” is the operative word driving the need for CI/CD. Without that, high levels of automation may not be necessary, particularly if the application doesn’t require frequent changes.

The age-old choice between point solutions and end-to-end solutions is the same for CI/CD as it has been for other types of tools: depth of features versus breadth of features. The latter is easier to implement because all of the pieces are designed to work together. However, depending on the complexity of the application and the environment, it may be preferable to implement “best of breed” point solutions. Point solutions may also make sense if a shop isn’t tied into a particular cloud provider or platform. Cloud providers and platform providers provide CI/CD tool chains free because it helps facilitate vendor lock-in, Petrocelli said.

Whatever the application mix is in an organization, it’s wise to procure CI/CD tooling with the future in mind so it can support the breadth of what exists, including traditional and new types of applications.

“What you don’t want to do is create tensions in your own organization because some people are getting the advantages of automated tools and others aren’t,” said Petrocelli.

DevOps and CI/CD: Real or not?

“DevOps” and “CI/CD” mean different things to different people. Many organizations claim to be doing one, the other, or both, which is not always the case.

“A lot of teams call themselves ‘DevOps’ when they’re really doing a tool chain. Many of them aren’t running integrated DevOps teams,” said Petrocelli. “If you’re not doing the management piece, you’re not doing DevOps.”

The Amalgam Insights report includes a section entitled “DevOps vs CI/CD” to help alleviate some of the confusion between the two. DevOps is deemed a project and organizational strategy. Since it’s conceptual, there’s no fixed implementation. CI/CD is defined as process automation, specifically “the process of taking raw developer code and other build artifacts and turning them into running applications and services.”

“When it comes down to implementation, DevOps is management and organization. It’s not tools. There’s no such thing as a DevOps tool chain,” said Petrocelli. “You can do DevOps and not have CI/CD tool chains in place and you can have CI/CD tool chains in place and not be doing real DevOps or DevOps at all.”

The report also explains the individual elements of CI/CD: continuous integration, continuous delivery and continuous deployment so readers have a common reference point. Quite often, “continuous delivery” and “continuous deployment,” are used interchangeably or one term may be used in place of the other.

Most people associate continuous deployment with the digital native disrupters that are doing hundreds or thousands of releases per day. Continuous delivery is what’s practical for everyone else. Two things that distinguish continuous delivery from continuous deployment are release velocity and the level of automation, which go hand-in-hand. According to the Amalgam Insights report, continuous delivery involves manual delivery while continuous deployment automates deployment.

What organizations call their practices matters less than the results they’re deriving from it. However, it may be harder to achieve the results desired when it’s unclear what the organization is trying to achieve in the first place.