DevOps and Storage
DevOps represents a cultural shift in how applications are developed and deployed. It can also be used in server deployment. But can it be used with storage?
DevOps (development and operations) refers to a software development method that includes the roles of software developers (dev) and other IT personnel with an emphasis on IT operations (Ops). DevOps is generally an enterprise software development phrase that emphasizes communication, collaboration, integration and automation. The goal of DevOps is to help an organization rapidly produce software products and services and to improve operations performance.
In the classic organization, the various teams, such as software development, quality assurance (QA) and IT operations are separated with very formal communications between them. DevOps creates a multidisciplinary approach so that the teams work together to develop products and services more rapidly. This involves a change in the processes and methods used for communication and collaboration between the various teams.
Generally DevOps is applied to developing products and services from an IT organization. It has bled into data center operations and system administration in a general sense. By some estimates it is used by 33 to 55 percent of organizations and is quickly gaining dominance as the development and deployment method.
Example of DevOps
In the classic development pattern, there are separate teams that handle specific tasks. A team of developers writes the application, doing limited testing to make sure it works to some degree. Then they pass it along to testers (QA) who will run unit tests and more comprehensive tests against the application. The hardware and development used by these two groups is likely to be different which each group owning their own. Hopefully, there has been some coordination between the two groups so that they at least use the same OS and set of tools.
There may be a few releases candidates between the developers and QA, but once they have achieved their requirements, the product is turned over to release managers who then coordinate "alpha" or "beta" testing, first with internal customers and then possibly with external customers. There may be some feedback to the developers, who make bug fixes, changes and updates. Once that is finished, the QA team does another review, and the code is labeled as ready.
Next the code is released to the sysadmins who are going to deploy it. Ideally, they served as testers in the previous cycle, but that's not always true. Moreover, the production hardware and software may be different than what the development and QA team used. Obviously, you want them to be as similar as possible, but there are almost always differences.
DevOps takes this process and creates a single organization that is responsible for all of the necessary tasks—development, QA, test, and sysadmins—with the focus on quickly delivering a better product. Rather than a small group of people focused solely on development, you have people who do development, test and QA, and who are also sysadmins all at the same time. A phrase you commonly hear for people in DevOps teams is "sysadmin coders." This "combined team" may divide up the necessary tasks, but they communicate with each other frequently to coordinate the entire process.
Using DevOps allows the pace of development to increase, which in turn results in applications being deployed faster and usually with better quality. It can also mean that the same number of people can produce more releases or updates.
At its core, DevOps is really a cultural change in how teams work together. This requires a change in the organization, a change in the behavior from team members, and a change in how management approaches the merge into DevOps. Everyone has to be part of the change and agree to it. As with any good cultural change in IT, there are some tools that can help with the transition.
When you start talking about DevOps there are a number of tools that come up in the conversation. While not an exhaustive list, tools such as Chef, Puppet, Ansible, and Salt are some of the top-mentioned tools. There are others, but these are the ones you encounter in polite conversation. It is definitely worthwhile briefly reviewing these tools.
Chef is written in Ruby and Erlang and streamlines the task of configuring and maintaining servers. It works with on-premise servers as well as cloud platforms such as those from Rackspace, Amazon EC2, Openstack, Microsoft Azure, IBM Softlayer, and Google Cloud Platform. The tools are open source, and there is a "Chef Company" that provides support to people using Chef.
You write "recipes" in Chef (written in what is basically Ruby) that describe how Chef manages server applications and utilities and how they are configured. They describe a series of resources that should be in a particular state. This means what packages are installed, what services should be running, or which files should be written. You can specify the specific versions of packages and the order in which packages are installed based on package dependencies.
When used with servers, Chef runs in a client/server model. The Chef client sends various attributes about the particular server to the Chef server. The server uses Solr to index these attributes and provides an API for clients to query this information. These queries can be done from Chef clients, and the resulting data is used to help configure the server.
Puppet runs along the lines of Chef in helping with the configuration and maintenance of servers. It is open source and focused on configuration management, particularly the management of servers. It too is written in Ruby and has its own language for creating node configurations (you can also use Ruby). These are called "manifests" and are similar to Chef's recipes.
The system information is discovered via utility named Facter. This tool compiles the Puppet manifests into a system-specific catalog that contains the resources and resource dependencies (very similar to Chef) which are applied against the target systems. The manifests contain a high-level description of the system aspects such as users, groups, services, and packages without the having to use system specific commands such as yum or apt-get or rpm.
While Puppet is open source, there is a company named Puppet Labs that can provide support for people using Puppet.
Ansible is an open source tool like Chef and Puppet. It is classically used for configuring and managing compute systems. It uses SSH or PowerShell and requires Python 2.4 or later to be installed on the servers. The various modules that make up Ansible use JSON and standard output. The module is installed on the target node (the system to be controlled) and communicates with the Ansible server node using JSON over standard output. The modules can be written in virtually any programming language that is scriptable, such as Python, Perl, Ruby, and Bash.
The system uses YAML to express reusable descriptions of systems. These descriptions are called Playbooks, and each Playbook maps a group of hosts to a set of roles. Each role is represented by calls to Ansible call tasks. As with Chef, in addition to on-premise systems, Ansible can be used with various private and public cloud providers such as VMware, OpenStack, AWS, Rackspace Cloud Servers, Eucalyptus Cloud, KVM, XenServer, IBM SoftLayer, Microsoft Azure and Cloud