Tag Archives: Measure what matters

Agile Metrics: Output Vs Outcome

Posted on by

Before we dive into Output Vs Outcome-Based Agile Metrics, let us understand what is meant by output-based and outcome-based metrics. Output-Based metrics measure the amount of work completed whereas Outcome-Based metrics measure the value delivered by the completed work.

Output-Based Agile Metrics

Metrics that measure how much work was completed in a given duration are said to be output-based. For example, the number of user stories completed in a Program Increment (PI) or an iteration measures the number of tickets completed in a specified duration. Other examples are:

  • Number of features completed in a Program Increment
  • Number of story points completed in a Program Increment
  • Number of story points completed in a sprint or an iteration

Such metrics can be easily gamed and do not provide any value-add. For example, Team A delivers 6 user stories whereas Team B delivers 12 user stories. This metric does not signify that Team B has better productivity than Team A.

Let’s assume Team A delivers 40 story points and Team B delivers 20 story points. Do you think Team A is a better performing team than Team B? Well, a team’s velocity does not provide insight into a team’s performance. Team A might have inflated their story point estimates or they might be working on user stories that do not deliver a potentially shippable product increment or bring any business value. 

Measuring the amount of work done is an Agile anti-pattern and should be avoided. It does not provide any value and may encourage Agile anti-patterns such that teams write tasks as user stories, create unnecessary user stories that do not bring any business value, and inflate story point estimates.

Outcome-Based Agile Metrics

Metrics that measure the outcome or the end-value delivered by the completed work are said to be outcome-based. For example, the business value delivered is an outcome-based metric that measures the amount of business value delivered by a unit of work during a specific duration. Let’s discuss how to measure the business value for an EPIC, Feature, or User Story. 

How to Measure the Business Value

Measure the delivered business value at an EPIC or a Feature level, rather than at a User Story level. Think in terms of:

  • Will this EPIC or Feature enable you to sell more units?
  • Will this EPIC or Feature allow you or your organization to make more profit?
  • Will this EPIC or Feature enable reduced operating costs for your business?
  • Will this EPIC or Feature enable a new capability for your end-users?
  • Will this EPIC or Feature reduce the manual effort for your team or end-users?
  • Does this EPIC or Feature enable your organization to compete in the market?
  • Does this EPIC or Feature allow to refine hypotheses about the market? 
  • Is this EPIC or Feature required to stay compliant with regulations?

Many EPICs or Features provide little to no value or aren’t actually desired by end-users. It’s important to prioritize the work effectively to deliver more business value. There are many ways to measure the business value like comparing the cost of delay, calculating Return On Investment (ROI), calculating revenues and expenses each month (Cash-Flow analysis), calculating Net Present Value (NPV), Planning Poker, and more. I suggest working with your business sponsors and:

  • Find Key Value Indicators (KVIs) for your product that you can measure at each Product Increment (PI) boundary.
  • Think about both qualitative and quantitative value indicators.
  • Review your KVIs regularly and revise as needed.

One of the simpler methods is to leverage the Planning Poker game and relatively estimate the business value for an EPIC or a Feature with your product stakeholders.

If you are interested to learn about other Outcome-Based Agile Metrics, enroll in my Udemy Course, Agile Metrics: Elevate Team Agility with Scrum Reporting.

You may be interested in my published courses and books as outlined below: 

Also, check out these articles…

Do You Measure Kanban Throughput?

Posted on by

What is Kanban throughput? What's the best way to measure it? This article reflects on these questions and provides a perspective to track a Kanban team's performance in an effective way.

 

What Is Kanban Throughput? 

Kanban Throughput is defined as the average number of items or cards passing through the flow within a specific time duration provided that the work load stays uniform during that period. It is generally used to track team's performance. Throughput variability reflects the difference in size, complexity, and team skills.

 

According to the Little's Law:

Throughput = Total WIP / Average Lead Time

 

Measuring What Matters: Kanban Throughput

The best way to measure throughput in Kanban is with the CFD (Commulative Flow Diagram). The Cumulative Flow Diagram is the visual representation of the cards as they move from one column or state to another on a Kanban board. The CFD plots the number of cards at each stage at a given time.

 

Below is a sample CFD for your reference:

 

Sample CFD Diagram

 

The different colors on this diagram represent the various states in the flow. The height of each color band indicates the number of cards in that state at that point in time. 

 

The CFD provides you with an insight on how many cards moved from one state to another in a specific time duration. Generally, the CFD is plotted for each day, however, if there are too many moving cards in a day, it can be plotted on an hourly basis as well. Below is a sample CFD when plotted for every hour in a working day.

 

Sample CFD - Plotted Hourly

 

Moreover, the CFD provides valuable data on lead time and cycle time trends. Both lead time and cycle time denote the time a work item spends in the workflow until they are complete. Lead time is the time that a card takes from start to finish. Cycle time is the time an engineer spends to actively work on it. In a CFD, both lead time and cycle time metrics are measured along the horizontal axis.

 

The Cumulative Flow Diagram (CFD) also displays total cards across different columns i.e. total WIP. This data is measured along the vertical axis of the CFD diagram.

 

Below is a sample CFD that depicts lead time, average cycle time, and the total WIP.

 

Sample CFD Diagram With Lead Time, Cycle Time, and Total WIP

 

If you are interested about other Kanban charts such as Average Lead Time, Average Cycle Time, Flow Efficiency Chart, or the Blocker Clustering Chart, read my book, The Basics Of Kanban - A Popular Lean Framework.

 

You may also be interested in these articles...

...and below books on Agile and Lean:

What is a Sprint Burndown Chart?

Posted on by

The Sprint Burndown or the Iteration Burndown chart is a powerful tool to communicate daily progress to the stakeholders. It tracks the completion of work for a given sprint or an iteration. The horizontal axis represents the days within a Sprint. The vertical axis represents the hours remaining to complete the committed work. The purpose of a sprint burndown chart is to show the total amount of work remaining.

Let's take an example. The below table shows the actual number of hours remaining at the end of each day within a Sprint to create a sample Sprint burndown chart. The table also captures data for ideal remaining hours. The ideal remaining effort is calculated by assuming a uniform rate of task completion each day. In the below example, you can see that the ideal and actual remaining hours were same at the beginning of the sprint, however, as the sprint progresses, the actual remaining hours vary from the ideal remaining hours. 

Sprint Burndown Chart Table

Next, we will plot a sample Sprint Burndown chart using the above table data. The below diagram depicts the sample chart with ‘Date’ represented on the horizontal axis and ‘Remaining Effort (Hours)’ represented on the vertical axis. 

Sample Sprint Burndown Chart

From the above chart, you can conclude that:

  • There were no spillover stories since the actual remaining hours at the end of the sprint were zero.
  • The team started a bit slow in the first week of the sprint, but then caught-up during the second week.

Other articles that you may be interested in:

Learn all about Agile Scrum with my book, The Basics Of SCRUM. Other Books: