Select Page
 

Benchmarking Release Management

29
March, 2018

by Niall Crawford

“It goes without saying, that the backbone of all organizations today is digital and independent of the end-products you provide”.

 

Digital the Backbone of Business

It goes without saying, that the backbone of all organizations today is digital and independent of the end-products you provide, there is an inherent need to drive your features and solutions from conception (the initial idea) through to production (the end customer) as quickly, safely and efficiently as possible.

  • That is, every organization requires effective IT Release Management.

So, what is IT Release Management? Let us start with a definition (or two):

From Wikipedia:

Release management is the process of managing, planning, scheduling and controlling a software build through different stages and environments; including testing and deploying software releases.

Enterprise release management is a multi-disciplinary IT governance framework for managing software delivery and software change across multiple departments in a large organization.

The former is what a DevOps engineer might gravitate to i.e. someone with the task of building and deploying code to their individual system or services.

And the latter something that might be handled by an Enterprise Release Manager (or Portfolio Manager), that is someone interested in coordinating a family of system/project releases using methods like SAFe (Scaled Agiile Framework) “Agile Release Trains” to maintain collective velocity, phase alignment, and a unified implementation.

A Visual Definition of Release Management

Alternatively, consider the following diagram.

Holistic Release Management is a combination of Vertical Practices responsible for supporting delivery through the system lifecycle (i.e. Analysis, Design, Development, Test, and Implementation), a set of Horizontal Practices responsible for driving transformation across the enterprise (or divisions).

Measuring Release Management

Independent of whether your perspective is vertical (system focused) and/or horizontal (enterprise focused), there is little argument in the importance of getting both activities completed correctly and having the various teams working together as effectively as possible.

However, agreeing on benchmarking may not be trivial as the two tasks, although obviously related, are quite different.  With the above in mind, we decided to put together two sets of release metrics that we thought would be useful in providing insight with respect to current release behaviour and capability.

  • The ten best metrics for Enterprise Release Management (ERM)
  • The ten best metrics for Release Management (aka System Deployment)

Note: Special thanks to some of our enov8 customers for sharing some of these.

Enterprise Release Management (ERM) Metrics

Whom: Typically, the responsibility of an IT Enterprise Release Manager the main goals are to ensure that a family of project releases (changes), which are often related and contain dependencies, can move through the software development lifecycle, from conception to production, in a controlled and aligned fashion.

Key data & trends, to be captured would be:

  1. Size of Enterprise Release by Projects (Project Releases Registered)
  2. Size of Enterprise Release by System Deployments
  3. Size of Enterprise Release by Features (or Stories)
  4. Project Releases Status (Success/Failure)
  5. Project Releases Delivered on Time
  6. Project Release Delay
  7. Number of Project De-Scoped*
  8. Number of Systems De-Scoped*
  9. Number of Features De-Scoped*
  10. Root Cause of Project Release Delay

*Note: In ERM, de-scoping is most typically caused by a project failing to meet its release gate obligations and being removed from the enterprise release as quickly as possible and thus avoid an impending “train crash’.

Release Management (System Deployment) Metrics

Whom – Typically, the responsibility of a DevOps “System Release” Engineer, the main goals are reliability and velocity i.e. the engineer wants to get the job done as safely and quickly as possible.

Key system deployment data & trends would be:

  1. System Deployment Frequency
  2. System Deployment Feature (Story) Size
  3. System Deployment Status (Success/Failure)
  4. System Deployment Time (ideally versus SLA)
  5. System Deployment Outage Time (outage time for deployment itself)
  6. System Post-Deployment Outage Time (outage caused by unsuccessful deployment)
  7. System Post-Deployment Incidents (Tickets) due to a deployment
  8. Mean Time to issue detection
  9. Root Cause of System Deployment Issue (Delay, Incident or Outage)
  10. Mean Time to remediation

Note: In the case of DevOps you would expect these metrics to be of use across the lifecycle. Therefore, the word ‘System’ could quite logically be replaced with ‘System Instance’, where you would have Instances in Integration, UAT, Staging & Production.

Food for thought

Hopefully, you found some of the above release metrics useful & maybe thought-provoking.

Ultimately metrics need to be aligned with your own operating model.

However, in closing we’d say this:

Good release metrics should always do the following:

  • Clarify release strategy i.e. direct good practice
  • Understand existing release capability i.e. strengths & weaknesses and
  • Drive release enhancement i.e. continuous improvement

What else would you add to the above “Release Management” Metrics?

Further Reading

  • Understand the association between ERM & DevOps:

Ref: ERM – The bridge between corporate strategy & DevOps

  • Releases need Environments, Non-Production Environment Metrics:

Ref: Test Environment Management Metrics

Learn More or Share Ideas

If you’d like to learn more about Release Management or perhaps just share your own ideas then feel free to contact the enov8 team. Enov8 provides a complete platform for addressing organizations “DevOps at Scale” requirements. Providing advanced “out of the box” IT & Test Environment ManagementRelease Management and Holistic Data Management capabilities.

Niall Crawford

Niall is the Co-Founder and CIO of Enov8. He has 25 years of experience working across the IT industry from Software Engineering, Architecture, IT & Test Environment Management and Executive Leadership. Niall has worked with, and advised, many global organisations covering verticals like Banking, Defence, Telecom and Information Technology Services.

Relevant Articles

What Is Your Attack Surface?

What Is Your Attack Surface?

15JULY, 2021 by Justin ReynoldsCompanies go to great lengths to protect their physical environments, using deterrents like locks, fences, and cameras to ward off intruders. Yet this same logic doesn’t always translate to digital security. Corporate networks — which...

Data: What Is DevSecOps?

Data: What Is DevSecOps?

06JULY, 2021 by Justin ReynoldsCompanies today face increasing challenges around reducing the time and cost of software development. Many are thus using DevOps methodologies, which combine software development and IT operations to achieve continuous delivery and...

Data: The ROI of Data Security

Data: The ROI of Data Security

24JUNE, 2021 by Omkar HiremathInformation technology and the digital world don’t exist without data. The data of an organization can contain a lot of unclassified, as well as classified information. Irrespective of that, only authorized personnel should have access to...

Enterprise Release Management Best Practices

Enterprise Release Management Best Practices

14JUNE, 2021 by Kathrin PaschenManaging releases at scale is daunting. It involves juggling dependencies, timelines, and requirements. The stakes can be pretty high, too. Not all failures are as spectacular as crashing a lunar lander or losing $440 million. Even so,...

What is Data Leakage?

What is Data Leakage?

28MAY, 2021 by Sasmito AdibowoThe benefits of using cloud environments to store and access data over the Internet has been highly beneficial for many businesses. Cloud environments help both start-ups and enterprises scale up conveniently. However, as with other major...

Serverless for Dummies

Serverless for Dummies

10MAY, 2021 by Eric GoebelbeckerImagine a technology that lets you focus on your business logic and that takes care of issues like reliability and scaling for you. What would it be like if you only had to pay for the computing time you use rather than pay by the day,...