DevOps


DevOps is a set of practices that combines software development and IT operations. It aims to shorten the systems development life cycle and provide continuous delivery with high software quality. DevOps is complementary with Agile software development; several DevOps aspects came from Agile methodology.

Definition

Academics and practitioners have not developed a unique definition for the term "DevOps".
From an academic perspective, Len Bass, Ingo Weber, and Liming Zhu—three computer science researchers from the CSIRO and the Software Engineering Institute—suggested defining DevOps as "a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality".
The term DevOps, however, has been used in multiple contexts.

History

In 2009, the first conference named devopsdays was held in Ghent, Belgium. The conference was founded by Belgian consultant, project manager and agile practitioner Patrick Debois. The conference has now spread to other countries.
In 2012, the State of DevOps report was conceived and launched by Alanna Brown at Puppet.
As of 2014, the annual State of DevOps report was published by Nicole Forsgren, Gene Kim, Jez Humble and others.
In 2014, they found that DevOps adoption was accelerating.
Also in 2014, Lisa Crispin and Janet Gregory wrote the book More Agile Testing, containing a chapter on testing and DevOps.

Toolchains

As DevOps is intended to be a cross-functional mode of working, those who practice the methodology use different sets of tools—referred to as "toolchains"—rather than a single one. These toolchains are expected to fit into one or more of the following categories, reflective of key aspects of the development and delivery process:
  1. Coding – code development and review, source code management tools, code merging.
  2. Building – continuous integration tools, build status.
  3. Testing – continuous testing tools that provide quick and timely feedback on business risks.
  4. Packaging – artifact repository, application pre-deployment staging.
  5. Releasing – change management, release approvals, release automation.
  6. Configuring – infrastructure configuration and management, infrastructure as code tools.
  7. Monitoring – applications performance monitoring, end-user experience.
Some categories are more essential in a DevOps toolchain than others; especially continuous integration and infrastructure as code.
Forsgren et al. found that IT performance is strongly correlated with DevOps practices like source code management and continuous delivery.

Relationship to other approaches

Agile

Agile and DevOps serve complementary roles: several standard DevOps practices such as automated build and test, continuous integration, and continuous delivery originated in the Agile world, which dates to the 1990s, and formally to 2001. Agile can be viewed as addressing communication gaps between customers and developers, while DevOps addresses gaps between developers and IT operations / infrastructure. Also, DevOps has focus on the deployment of developed software, whether it is developed via Agile or other methodologies..

ArchOps

ArchOps presents an extension for DevOps practice, starting from software architecture artifacts, instead of source code, for operation deployment.. ArchOps states that architectural models are first-class entities in software development, deployment, and operations.

TestOps

TestOps is to hardware development what DevOps is to software development. The idea is a toolchain that links design and test operations together. In the case of hardware, design means EDA tools and the CAD department, and test means electronic measurement equipment like oscilloscopes and so on.

Continuous delivery

Continuous delivery and DevOps have common goals and are often used in conjunction, but there are subtle differences.
While continuous delivery is focused on automating the processes in software delivery, DevOps also focuses on the organizational change to support great collaboration between the many functions involved.
DevOps and continuous delivery share a common background in agile methods and lean thinking: small and frequent changes with focused value to the end customer.
Lean management and continuous delivery are fundamental to delivering value faster, in a sustainable way.
Continuous delivery focuses on making sure the software is always in a releasable state throughout its lifecycle. This makes it cheaper and less risky to deliver the software.
Improved collaboration and communication both between and within organizational teams can help achieve faster time to market, with reduced risks.

DataOps

The application of continuous delivery and DevOps to data analytics has been termed DataOps. DataOps seeks to integrate data engineering, data integration, data quality, data security, and data privacy with operations. It applies principles from DevOps, Agile Development and the statistical process control, used in lean manufacturing, to improve the cycle time of extracting value from data analytics.

Site-reliability engineering

In 2003, Google developed site reliability engineering, an approach for releasing new features continuously into large-scale high-availability systems while maintaining high-quality end-user experience. While SRE predates the development of DevOps, they are generally viewed as being related to each other.

Systems administration

DevOps is often viewed as an approach to applying systems administration work to cloud technology.

WinOps

is the term used for DevOps practices for a Microsoft-centric view.

Toyota production system, lean thinking, kaizen

Toyota production system, also known under the acronym TPS, was the inspiration for lean thinking with its focus on continuous improvement, kaizen, flow and small batches. The Andon cord principle to create fast feedback, swarm and solve problems stems from TPS.

DevSecOps, Shifting Security Left

is an augmentation of DevOps to allow for security practices to be integrated into the DevOps approach. The traditional centralised security team model must adopt a federated model allowing each delivery team the ability to factor in the correct security controls into their DevOps practices.

Goals

IT performance can be measured in terms of throughput and stability.
Throughput can be measured by deployment frequency and lead time for changes; stability can be measured by mean time to recover. The State of DevOps Reports found that investing in practices that increase these throughput and stability measures increase IT performance.
The goals of DevOps span the entire delivery pipeline. They include:
Simple processes become increasingly programmable and dynamic, using a DevOps approach. DevOps aims to maximize the predictability, efficiency, security, and maintainability of operational processes. Very often, automation supports this objective.
DevOps integration targets product delivery, continuous testing, quality testing, feature development, and maintenance releases in order to improve reliability and security and provide faster development and deployment cycles. Many of the ideas involved in DevOps came from the enterprise systems management and agile software development movements.
Practices that correlate with deployment frequency are:
Practices that correlate with a lead time for change are:
Practices that correlate with a mean time to recovery for change are:
Companies have reported significant benefits, including: significantly shorter time to market, improved customer satisfaction, better product quality, more reliable releases, improved productivity and efficiency, and the increased ability to build the right product by fast experimentation.
The 2014 State of DevOps Report found that "IT performance strongly correlates with well-known DevOps practices such as the use of version
control and continuous delivery."

Criticism

There is a lack of evidence in academic literature on the effectiveness of DevOps.

Cultural change

DevOps initiatives can create cultural changes in companies by transforming the way operations, developers, and testers collaborate during the development and delivery processes. Getting these groups to work cohesively is a critical challenge in enterprise DevOps adoption. DevOps is as much about culture, as it is about the toolchain.

DevOps as a job title

While DevOps describes an approach to work rather than a distinct role, job advertisements are increasingly using terms like "DevOps Engineer".
While DevOps reflects complex topics, the DevOps community uses analogies to communicate important concepts, much like "The Cathedral and the Bazaar" from the open-source community.
Organizational culture is a strong predictor of IT and organizational performance. Cultural practices such as information flow, collaboration, shared responsibilities, learning from failures and new ideas are central to DevOps. Team-building and other employee engagement activities are often used to create an environment that fosters this communication and cultural change within an organization. Team-building activities can include board games, trust activities, and employee engagement seminars.
The 2015 State of DevOps Report discovered that the top seven measures with the strongest correlation to organizational culture are:
1. Organizational investment in DevOps:
2. Team leaders' experience and effectiveness.
3. Continuous delivery.
4. The ability of different disciplines to achieve
win-win outcomes.
5. Organizational performance.
6. Deployment pain.
7. Lean management practices.

Deployment

Companies with very frequent releases may require knowledge on DevOps. For example, the company that operates image hosting website Flickr developed a DevOps approach to support ten deployments a day. Daily deployment cycles would be much higher at organizations producing multi-focus or multi-function applications. Daily deployment is referred to as continuous deployment or continuous delivery and has been associated with the lean startup methodology. Professional associations and blogs posts have formed on the topic since 2009.

Architecturally significant requirements

To practice DevOps effectively, software applications have to meet a set of architecturally significant requirements, such as: deployability, modifiability, testability, and monitorability. These ASRs require a high priority and cannot be traded off lightly.

Microservices

Although in principle it is possible to practice DevOps with any architectural style, the microservices architectural style is becoming the standard for building continuously deployed systems. Small size service allows the architecture of an individual service to emerge through continuous refactoring, hence reducing the need for a big upfront design, allows for releasing the software early and continuously.

DevOps automation

DevOps automation can be achieved by repackaging platforms, systems, and applications into reusable building blocks through the use of technologies such as virtual machines and containerization.
Implementation of DevOps automation in the IT-organization is heavily dependent on tools, which are to cover different areas of the systems development lifecycle :
  1. Infrastructure as code
  2. CI/CD
  3. Test automation
  4. Containerization
  5. Orchestration
  6. Software deployment
  7. Software measurement

    Adoption

DevOps practices and adoption

Jabbari et al. identified DevOps practices and their dependencies. They developed a benefits dependency network which connects potential benefits to an ordered chain of practices. Using this network organizations can choose a path that enables fulfillment of their goals.
Some articles in the DevOps literature assume or recommend significant participation in DevOps initiatives from outside an organization's IT department, e.g.: "DevOps is just the agile principle, taken to the full enterprise."
In a survey published in January 2016 by the SaaS cloud-computing company RightScale, DevOps adoption increased from 66 percent in 2015 to 74 percent in 2016. And among larger enterprise organizations, DevOps adoption is even higher – 81 percent.
Adoption of DevOps is being driven by many factors – including:
  1. Use of agile and other development processes and methods;
  2. Demand for an increased rate of production releases – from application and business unit stakeholders;
  3. Wide availability of virtualized and cloud infrastructure – from internal and external providers;
  4. Increased usage of data center automation and configuration management tools;
  5. Increased focus on test automation and continuous integration methods;
  6. A critical mass of publicly available best practices.