Why we always start with Continuous Integration and Continuous Delivery

Source:- 126kr.com

Ten years ago, researchers and cutting-edge developers had to argue that software testing was part of the process of writing software, not a nice to have depending on the budget. Nowadays, it is almost always accepted that shipping an application (or a feature) is an inherent bundle including code, tests and documentation. Open Source software have been written that way for decades and thanks to GitHub, which generalized Open Source good practices, things got a lot better.

In parallel, other services like Travis CI or CircleCI made automated testing more accessible by providing nice GitHub integration to perform continuous testing , also known as Continuous Integration (CI). The idea behind this concept is to use a build server to, periodically, (i) build the software from its sources, (ii) run the test suite and (iii) report the results to the development team. By linking CircleCI to a project hosted on GitHub or using GitLab (which has built-in CI ), the developers get instant feedback on the health of the software. As the name suggests, the principle of CI also eases the collaboration between engineers, allowing them to “integrate” their changes in a unique shared codebase as soon as the test suite passes. This practice leads to the notion of feedback loop , which has different meanings depending on the context.

The feedback loop in the context of continuous integration is mainly intended for the developers. Its purpose is to tell them whether any code change can be integrated with the current (stable) version of the software, reflected by the test suite being “green” (as opposed to “red” when the test suite does not pass). Experienced readers will probably notice a slightly different vision of CI in the previous sentence: in strict CI, developers push their work to the mainline at least daily .

We, as most people do, use feature branches and do not necessarily merge these branches into the mainline within a day. Feature branches have emerged with the rise of Distributed Version Control Systems (DVCS) like Git; GitHub and GitLab being two popular proponents of the use of feature branches. In fact, both tools are framed around the notion of branches. A feature branch is a short-lived branch that carries a change, e.g. , a bug fix or a new feature.

                                                          The status panel of a Pull Request on GitHub.

The traditional workflow is to (i) create a new branch from the last state of the mainline, (ii) develop and commit, (iii) share the changes by pushing the branch to the shared repository, and (iv) open a Pull Request (GitHub) or Merge Request (GitLab). The last step will then trigger a build on the CI server, which will update the status of the feature branch accordingly (that is where this “green / red” thing comes from). Note that GitHub/GitLab can also indicate whether a branch conflicts with the mainline. Once everything is “green” (as depicted in the figure above), merging the feature branch into the mainline is usually a safe operation. This fifth step closes a typical Git flow , no matter the one you choose. At TailorDev, we use and promote the Simple Git branching model , which uses the master branch as mainline and rebase to avoid conflicts.

                                                                           As simple as that.

The developer experience is greatly improved and most of the annoying things related to testing or software quality in general are automated. The simple Git workflow combined to continuous integration also brings a few organization-related rules to the team. For instance, the developer who creates a feature branch becomes its owner and must take care of it until this branch gets merged into the mainline. Other team members can review his work by performing code reviews , which is also considered a good software engineering practice to improve the overall quality of a software. Both GitHub and GitLab simplify code reviews . Another important rule is: the developer who breaks the build has to fix it, which avoids endless discussions about responsibilities. Everyone contributes to make the software as healthy as possible.

It is therefore not unusual to consider the mainline as the current stable version of the software, which should be deployable or built at any time and fully functional. This leads to the two following concepts: Continuous Delivery (CD) and Continuous Deployment. Both are related: the former allowing the latter. Continuous Delivery is about having a set of methods and tools to be able to release a software to production at any time. While CI and CD are two different concepts, it is not possible to achieve CD without CI. Technically, it is doable but… baby steps please ). On the other hand, Continuous Deployment is deploying every change in production, automatically.

                                                          By Kharnagy – Own work , CC BY-SA 4.0

Continuous Delivery also has a feedback loop, which is not only intended for the developers but for the ops, and even the business and the users. This feedback loop does not have a single purpose but rather different goals for the different people involved. The developers will be interested in knowing whether a change behaves as expected. The ops will be interested in being sure whether the software is correctly deployed and the infrastructure is not overloaded. The business will be interested in shipping more features more often to the users, and the latter will benefit from frequent updates. That is somehow related to the term DevOps , which is a cultural shift and collaboration between the developers, the ops and the testers. Yet, this movement goes beyond the scope of this article and you can read more about it here .

                                     A GitLab pipeline with the different steps to unlock the “deploy buttons”.

At TailorDev, we rely on CD a lot because it eases the communication between our customers and us. We usually set up different environments (such as dev , staging and production ) and write simple scripts to build and deploy the software in an automated fashion . We insist on making these scripts as simple as possible and to rely on boring and stress-less deployment processes. Most of the time we enable Continuous Deployment on the non-production environments to test our deployment processes. The benefits of such a practice are manifold:

• the deployment process is automated, reproducible and documented: a shell script describes the commands to run in order to build and deploy the software. This is the source of truth. The whole deployment process is also documented in a README file;
• anyone can deploy the software at any time: we leverage the GitLab interface to display buttons to deploy to the different environments ( cf. figure above). On GitHub, we use the Deployments API and our beloved hubot . And no, there is no rollback button ;
• we can ship smaller chunks in production more often: release early. Release often. And listen to your customers said Eric S. Raymond. Having different environments also eases the discussions with our customers to be sure that what they want is what they get;
• we can fix bugs introduced in production more rapidly: even though a lot of automated tests usually helps, software testing is about revealing the presence, not the absence of bugs, as Edsger Wybe Dijkstra used to say . Real production data often reveal edge cases and users are perfect monkey testers anyway.

By making sure we get quicker feedback during the development and deployment phases thanks to CI and CD, we can forget these quality controls and focus on adding value to the project. We will be warned if something goes wrong anyway. Requiring these modi operandi at the beginning also enforces that all team members work in a similar manner and in complete collaboration.

As you may have noticed already, CI/CD often leads to the adoption of best practices and standardization (feature branches, code reviews, etc.). That is very important to us too since we usually have short-term contracts with our customers and they must be self-sufficient when we leave the projects. It will become much easier to add new people to the project later. That is why we always start our collaborations with a focus on CI/CD. It takes a couple days to setup everything but it is worth doing it for the health of the project and a great communication.