The Ultimate DevOps Toolkit

Source:- xenonstack.com

DevOps isn’t a tool or a product. DevOps is a process and balanced organization approach for improving collaboration, communication among development and operation.

Redesigning and find new ways for faster and Reliable Delivery for accelerated time to market, improved manageability, better operational efficiency, and more time to focus on your core business goals.

DevOps Toolchain

During Transformation Towards Agile & DevOps, DevOps needs a platform where we can define workflow with different Integrations. Implementing DevOps Culture into your workflow requires using of specialized tools.

Below is an outline of each key category of tools that need to be in your toolkit, and the leading technologies to consider as you build the toolkit that best supports your team and your organization.

So let’s develop our DevOps Toolkit:

1. Source Code Management (SCM) System

Everything we build can be expressed through code. But when everything is the code you need to be sure that you can control and perform branching on it – otherwise things could get chaotic. So to avoid that chaos we use SCM system that includes:

• GitHub: It is a web-based Git or Version Control Repository.
• Gitlab: It provides Git Repository Management, Code Reviews, Issue Tracking, Activity Feeds and Wikis.

2. Build and Continuous Integration(CI)

Continuous Integration is a fundamental best practice of modern Software Development. By Setting up an effective Continuous Integration environment, we can

• Reduce Integration Issues
• Improve Code Quality
• Improve Communication and Collaboration between Team Members
• Faster Releases
• Less Bugs

Jenkins is used as Continuous Integration Platform to merge code from individual developers into a single project, multiple times per day and test continuously to avoid any downstream problems.

 Continuous Integration Platform Features Includes:

• Integration with SCM System
• Secret Management
• SSH-Based Access Management
• Scheduling and Chaining of Build Jobs
• Source Code Change Based Triggers
• Worker/Slave Nodes
• Rest API Support
• Notification Management

3. Building tools

While building our organization, we have invested much of our time in research as which tools we need to include in our DevOps toolkit and which not to. These decisions are based on our years of experience in IT industry. We’ve taken great care in selecting, benchmarking and constantly improving our tools selection.

By sharing our Tools, we hope to foster a discussion within the DevOps community so that we can further improve.

• Apache Maven: It is a Software Project Management and Comprehension Tool. Based on the concept of a Project Object Model (POM), Maven can manage a project’s build, reporting, and documentation from a central piece of information.
• Apache Ant: It is a Java library and command-line tool whose mission is to drive processes described in build files as targets and extension points dependent upon each other.
• Gradle: It is a build tool with a focus on build automation and support for multi-language development.
• Grunt: It is a JavaScript task runner, a tool used to automatically perform frequently used tasks such as Minification, Compilation, Unit Testing, Linting, etc.
• Make: It is a build automation tool that automatically builds executable.
• Packer: It is a free and open source tool for creating golden images for multiple platforms from a single source configuration.

4. Testing

In order to achieve the desired business goals of DevOps, you need to have an accurate, real-time measure of the risks and quality assurance of the features in your delivery pipeline and this can only be achieved through extensive and accurate testing.

Following are the testing tools being used by us to automate and streamline our DevOps Processes

• JUnit: It is a simple framework to write repeatable tests.
• Mocha: It is a simple, flexible, fun JavaScript test framework for Node.js.

5. Artifacts Management

Now that your build pipeline consistently versions your Maven project, you need a place to store your artifacts which are being produced at the end of this pipeline. These artifacts need to be stored much the same way your source code is stored in your SCM.

This ensures access to previously released versions of your product. An Artifact Repository is designed to store your war/jar/ear/etc, and distribute it to fellow developers via Maven, Ivy, or the like, share your artifact with you deployment tools, and generally ensure an immutable history of your released products.

• Using a Standard Artifacts Management System such as Artifactory
• Caching Third-Party Tools

6. Configuration Management

Configuration management is the process of standardizing resource configurations and enforcing their state across IT infrastructure in an automated yet agile manner.

• Ansible: It is an agentless configuration management system which relies on ssh protocol.
• Chef and Puppet: These are agent-based configuration management system.

7. Deployment

Continuous Deployment is a software development practice in which every code change goes through the entire pipeline and is put into production, automatically, resulting in many production deployments every day.

Process Management

• Supervisor: It is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.
• PM2: Advanced, production process manager for Node.js.
• Forever: It is simple CLI tool for ensuring that a given script runs continuously.

8. Orchestration

Software systems that facilitate the automated management, scaling, discovery, and/or deployment of container-based applications or workloads.

• Kubernetes: It is an orchestration system for Docker containers. It handles scheduling and manages workloads based on user-defined parameters.
• Docker Swarm: It provides native clustering functionality for Docker containers, which lets you turn a group of Docker engines into a single, virtual Docker engine.

9. Monitoring

The end goal for your monitoring is to consolidate tools, reduce the total cost of ownership, and automate the configuration via machine learning.

• Monitoring is defined on different levels like a system, platform, application etc. Data-driven based monitoring is done with the help of Zabbix.
• ELK(Elasticsearch, Logstash & Kibana) Stack provides actionable insights in real time from almost any type of structured and unstructured data source.
• Grafana is most commonly used for visualizing time series data from infrastructure and application analytics but many use it in other domains including industrial sensors, home automation, weather, and process control.

10. Collaboration & Notification

• Rocket Chat: Rocket Chat is an open-source messaging app built with Meteor. It supports video conferencing, file sharing, voice messages, has a fully-featured API, and more. Rocket Chat is great for those who prefer to have full control over their communications.
• Slack: It is basically a messaging app on steroids. It’s meant for teams and workplaces can be used across multiple devices and platforms, and is equipped with robust features that allow you to not only chat one-on-one with associates but also in groups. You’re able to upload and share files with them too, as well as integrate with other apps and services, such as Skype for video calls, and you can granularly control almost every setting, including the ability to create custom emoji.

11. Bug Management System

Bugs – Whether they be in the code or in the usability of the program, can be debilitating to any project, potentially causing the entire initiative to fail. For this reason, we use Bug Management System which enables us to identify those bugs and assure the quality of our code. For Bug Management System we use,

• Use of systems such as GitLab and Taiga.
• Workflow related to Bug Filing and Resolution Process
• Integrating SCM System with Bug Resolution Process using Triggers or REST APIs.

12. Collaborative Documentation System

It includes:

• Wiki basics
• Using Media Wiki, Confluence etc.
• Documenting DevOps Projects
• Organizing Documents on a Wiki-Based System for the past DevOps Projects.

13. Security

Access Management

• Users and groups
• Access control lists
• OpenSSH server
• Lightweight Directory Access Protocol (LDAP)
• Active directory (AD)

14. Basic troubleshooting

• nc – Simple TCP proxies, shell-script based HTTP clients and servers, network daemon testing
• netstat- Displaying the Routing Table
• traceroute- Provides information about the number of routes present between source to destination
• vmstat- Summary information on Memory, Processes, Paging etc.
• iostat – Gives Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions
• lsof- List open files(Linux/Unix considers everything as files)
  top- Provides a dynamic real-time view of a running system
• nslookup- Query Internet domain name servers for information
• ping – Packet Internet Groper
• tcpdump- Dumps traffic on a network
• dig – Domain Information Groper, for DNS lookup and to query DNS name servers
• sar – Collect, report, or save system activity information
• uptime – To get a snapshot of info about the servers time since the last reboot, and load
• ifconfig – Configures or displays network interface parameters for a network by using TCP/IP
• route – To show/manipulate the IP routing table

Summing-Up

There are definitely more tools that facilitate various aspects of DevOps environment, but DevOps isn’t just about the tools.Make sure tools are being used right way and for right reasons. It’s essential that the tools and applications used to develop, test, deploy, monitor and manage applications all work seamlessly together.