How devops changes both dev and ops

Source – techworld.com.au

Devops is the literal and figurative fusion of development and operations. For years, these two groups have been separated by cultural and knowledge boundaries, particularly inside larger enterprise IT organizations.

This separation was straightforward: Developers focused on nothing more than code and operations focused on taking that code and making sure it stayed running. The complete disconnect between these two groups led to long QA cycles and infrequent production deployments for fear of downtime or breaking something.

A combination of organizational divisions, risk aversion, and sequential, “waterfall” methods of software delivery meant it could be a year or more between major software updates. In many large organizations, these practices remain in place today.

Yet a major shift has been reshaping the IT landscape during the last decade. Developers, tired of waiting for production to get around
to updating their code, have started writing software that automates operational tasks. At the same time, operations personnel have been contributing their deep knowledge and expertise back into software being written by the developers.

The once clear boundaries between development and operations are blurring. This is leading to increasingly faster application lifecycles, shorter QA sessions, and many more deployments. We even have new processes like continuous integration and continuous deployment to manage. Many large-scale applications built with a devops methodology are being deployed to production multiple times per day rather than a few times per year.

This isn’t just some newfangled fashion that will soon pass. This trend toward automation has progressed gradually during the last 10 years, and the software and processes have become very mature. The consequences of devops inside enterprise IT are going to have to be thoroughly considered by everybody involved.

If you do not understand why devops is so important and you do not prepare for all of the changes that devops is bringing, your organization will be left behind.

How devops is impacting developers

The origins of devops can be tracked all the way back to tools such as Puppet, which arrived back in 2005, before the devops term emerged. Luke Kanies, a Ruby developer at the time, was tired of manually configuring Linux and modifying configuration files by hand. He dreamt of a way to provision and configure Unix-like systems in a more programmatic and reproducible way. So he wrote a Ruby script that did it for him, and he called it Puppet.

Later, similar tools were developed, including Chef, Ansible, SaltStack, and others. Furthermore, communities coalesced around each of these tools. Developers and operations experts would contribute their knowledge into packaged “cookbooks” or “recipes” that would let you configure software in a similar way regardless of the underlying Unix distribution.

With these tools, a developer could create a self-contained, programmatic description of how to run an application. It would include all of the dependencies and be able to configure and start them on various Unix distributions simply by running a script. What used to take weeks of manual setup and tuning by highly skilled professionals could now be done in mere hours with the help of a script.

Although developers could now deploy
their code faster and easier than before, a new problem was introduced. Because developers were no longer as dependent on their operational counterparts, developers became responsible for maintaining their own running applications.

That problem led to the development of yet another tool: platform as a service, or PaaS for short. Though originally conceived by Salesforce and Google, the first iterations of PaaS required developers to write special code that locked applications onto their platform. PaaS wasn’t popularized until Heroku showed developers a way to run their code with very few, if any, major modifications.

PaaS systems were built on top of devops automation principles. In fact, most PaaS systems to this day use devops tools to set up and run them. The difference is that with a PaaS, the applications that run inside them are fully managed. You can start, stop, scale, and monitor an application within a PaaS through APIs. In devops, you can build a toolset to manage your applications, but with PaaS the tools are pre-built for you.

Finally, no discussion of devops is complete without mentioning Docker and containers. The biggest shortcoming of PaaS is that it predefines the application architecture very strictly. If a developer wants more control over the environment, containers provide
it while retaining speed and agility. Using Chef or Puppet to set up an environment from scratch can take a few hours, and you might not end up with a totally exact copy. Using a container, a developer can reproduce a Linux or Windows environment within a fraction of a second and be certain that it is an exact copy.

Developers have been able to see huge
gains in productivity by adopting and refining these devops tools over the years. The days of developers being oblivious to operations and scaling problems is coming to an end as the art of running applications becomes distilled into fully automated computer code.

How devops is impacting operations

Although developers started the devops revolution, sysadmins and operations experts have been key to its ultimate adoption. After all, the tools help them create efficiencies in their work as well. In fact, devops tools have dramatically shifted the landscape of the kind of work a modern, agile operations team is responsible for.

Before devops, the sysadmins and operations teams were charged with keeping individual applications running at large scale. This would include such diverse activities as tuning databases and web servers, setting up load balancers, managing security, managing caching systems, and much more.

Devops has enabled a high degree of standardization, creating efficient ways of deploying, configuring, and running many servers with just a few tools instead of relying on human intervention.

Increasingly, it is the role of the operations team to deploy and maintain an automated, on-demand application service such as a PaaS or a Linux or Windows container cluster. Developers then deploy and scale the individual applications into the application mesh, leaving the operations teams to run and scale the mesh.

Devops platforms and tools have created
a buffer, which has allowed developers and operations teams to work independently of one another, rather than in lockstep. Before these tools, there was a sequential application pipeline. Individual applications would need to have individual requests made to operations, procurements processed, and servers configured before an application could finally be deployed.

With the advent of devops tools, developers can have quotas, where, within certain limits, they would be able to deploy on demand and in real time. The operations teams no longer need to worry about deploying individual applications. They still procure hardware and set up and manage servers, but they do so at a larger scale than the application unit. Their responsibility lies in managing the automated devops mesh to which developers have more ready access.

This technology buffer has decoupled the sequential application lifecycle, enabling developers and operations teams to work more closely together. It might seem counterintuitive that decoupling teams would bring them closer, but the fastest way to start a fight is to put two cooks in the same kitchen. Letting developers work within a well-defined system (e.g., building the code inside the mesh), while
the operations team works outside the container, ensures that the cooks have separate responsibilities.

But the middle ground between the developers and operations teams—the devops
tools themselves—has become a collaborative ecosystem where responsibilities blur. Operations teams have the deep knowledge of best practices for running and maintaining complex software systems. Developers have the ability to teach computers how to implement that deep operations knowledge without human intervention.

Increasingly, employers are looking for programmers with operations experience and sysadmins with programming experience. In general, however, the biggest impact that devops is having on operations teams
is that they are increasingly put in charge of running the application mesh, which provides automated, on-demand deployment options for developers.

A virtuous circle

Marc Andreessen’s famous words, “software is eating the world,” are as true for building and running applications as they are for calling a cab or renting a hotel. For decades, the task of running applications has been the province of an elite group of highly trained sysadmin geniuses. Now it’s going mainstream.

Most of the knowledge was undocumented and passed on verbally, like folklore, from generation to generation. Configuration files all had their own eccentric formats. Sometimes these formats were so strange that there would be configuration files that would compile into further configuration files. The knowledgeof exactly how and why to tweak these files remained a mystery to all but a few.

Devops has become a way to document all that esoteric operations knowledge into open standards that can be documented and tracked over time. The knowledge can be programmed into the logic used to build application platform meshes like PaaS and Linux container clusters—and it can be iterated and shared with others.

As developers become better at operations and sysadmins become better at programming, their responsibilities will eventually fuse. The lines are already blurring, and that is opening the door for new tools for collaboration between groups that used to keep collaboration to a minimum.

Docker, the leading container management platform, is yet another example of this evolution. GitHub gave developers an open environment to share code with one another and collaborate more easily than ever before. The Docker Hub is opening up the same sorts of collaboration for sysadmins, ushering in a new renaissance in infrastructure deployment and management.

Devops will continue to evolve. There will always be a new tool, a different framework, and a cool trend, but the underlying thread that ties them all together is true software automation. Just as automation brought continuous integration and continuous delivery to software development, it is bringing programmable infrastructure to operations.