Agile Metrics: Output Vs Outcome

Before we dive into Output Vs Outcome-Based Agile Metrics, let us understand what is meant by output-based and outcome-based metrics. Output-Based metrics measure the amount of work completed whereas Outcome-Based metrics measure the value delivered by the completed work.

Output-Based Agile Metrics

Metrics that measure how much work was completed in a given duration are said to be output-based. For example, the number of user stories completed in a Program Increment (PI) or an iteration measures the number of tickets completed in a specified duration. Other examples are:

  • Number of features completed in a Program Increment
  • Number of story points completed in a Program Increment
  • Number of story points completed in a sprint or an iteration

Such metrics can be easily gamed and do not provide any value-add. For example, Team A delivers 6 user stories whereas Team B delivers 12 user stories. This metric does not signify that Team B has better productivity than Team A.

Let’s assume Team A delivers 40 story points and Team B delivers 20 story points. Do you think Team A is a better performing team than Team B? Well, a team’s velocity does not provide insight into a team’s performance. Team A might have inflated their story point estimates or they might be working on user stories that do not deliver a potentially shippable product increment or bring any business value. 

Measuring the amount of work done is an Agile anti-pattern and should be avoided. It does not provide any value and may encourage Agile anti-patterns such that teams write tasks as user stories, create unnecessary user stories that do not bring any business value, and inflate story point estimates.

Outcome-Based Agile Metrics

Metrics that measure the outcome or the end-value delivered by the completed work are said to be outcome-based. For example, the business value delivered is an outcome-based metric that measures the amount of business value delivered by a unit of work during a specific duration. Let’s discuss how to measure the business value for an EPIC, Feature, or User Story. 

How to Measure the Business Value

Measure the delivered business value at an EPIC or a Feature level, rather than at a User Story level. Think in terms of:

  • Will this EPIC or Feature enable you to sell more units?
  • Will this EPIC or Feature allow you or your organization to make more profit?
  • Will this EPIC or Feature enable reduced operating costs for your business?
  • Will this EPIC or Feature enable a new capability for your end-users?
  • Will this EPIC or Feature reduce the manual effort for your team or end-users?
  • Does this EPIC or Feature enable your organization to compete in the market?
  • Does this EPIC or Feature allow to refine hypotheses about the market? 
  • Is this EPIC or Feature required to stay compliant with regulations?

Many EPICs or Features provide little to no value or aren’t actually desired by end-users. It’s important to prioritize the work effectively to deliver more business value. There are many ways to measure the business value like comparing the cost of delay, calculating Return On Investment (ROI), calculating revenues and expenses each month (Cash-Flow analysis), calculating Net Present Value (NPV), Planning Poker, and more. I suggest working with your business sponsors and:

  • Find Key Value Indicators (KVIs) for your product that you can measure at each Product Increment (PI) boundary.
  • Think about both qualitative and quantitative value indicators.
  • Review your KVIs regularly and revise as needed.

One of the simpler methods is to leverage the Planning Poker game and relatively estimate the business value for an EPIC or a Feature with your product stakeholders.

If you are interested to learn about other Outcome-Based Agile Metrics, enroll in my Udemy Course, Agile Metrics: Elevate Team Agility with Scrum Reporting.

You may be interested in my published courses and books as outlined below: 

Also, check out these articles…

Do You Measure Kanban Throughput?

What is Kanban throughput? What's the best way to measure it? This article reflects on these questions and provides a perspective to track a Kanban team's performance in an effective way.

 

What Is Kanban Throughput? 

Kanban Throughput is defined as the average number of items or cards passing through the flow within a specific time duration provided that the work load stays uniform during that period. It is generally used to track team's performance. Throughput variability reflects the difference in size, complexity, and team skills.

 

According to the Little's Law:

Throughput = Total WIP / Average Lead Time

 

Measuring What Matters: Kanban Throughput

The best way to measure throughput in Kanban is with the CFD (Commulative Flow Diagram). The Cumulative Flow Diagram is the visual representation of the cards as they move from one column or state to another on a Kanban board. The CFD plots the number of cards at each stage at a given time.

 

Below is a sample CFD for your reference:

 

Sample CFD Diagram

 

The different colors on this diagram represent the various states in the flow. The height of each color band indicates the number of cards in that state at that point in time. 

 

The CFD provides you with an insight on how many cards moved from one state to another in a specific time duration. Generally, the CFD is plotted for each day, however, if there are too many moving cards in a day, it can be plotted on an hourly basis as well. Below is a sample CFD when plotted for every hour in a working day.

 

Sample CFD - Plotted Hourly

 

Moreover, the CFD provides valuable data on lead time and cycle time trends. Both lead time and cycle time denote the time a work item spends in the workflow until they are complete. Lead time is the time that a card takes from start to finish. Cycle time is the time an engineer spends to actively work on it. In a CFD, both lead time and cycle time metrics are measured along the horizontal axis.

 

The Cumulative Flow Diagram (CFD) also displays total cards across different columns i.e. total WIP. This data is measured along the vertical axis of the CFD diagram.

 

Below is a sample CFD that depicts lead time, average cycle time, and the total WIP.

 

Sample CFD Diagram With Lead Time, Cycle Time, and Total WIP

 

If you are interested about other Kanban charts such as Average Lead Time, Average Cycle Time, Flow Efficiency Chart, or the Blocker Clustering Chart, read my book, The Basics Of Kanban - A Popular Lean Framework.

 

You may also be interested in these articles...

...and below books on Agile and Lean:

4 Things that Impact Productivity at Work

What are the main reasons that impact your productivity at work? Let's take a look at a typical work day. How many times do you get interrupted by your colleagues who want to pick your brain on something. Or how many times does your leader interrupt you for status updates? How many times do you check your phone or your smart watch? How often do you receive a text, a video, or a mime? This article will list the 4 habits that impact your productivity at work.

 

Accepting Interruptions

If your colleagues ask for your help, do you feel obliged to help them? Or do you schedule time in your calendar to respond to their queries? Does your boss interrupt you while you are in the middle of something important? Do you leave everything to listen to his concerns?

Interruptions are external stimuli that we don't plan for. Things such as unplanned discussions and ad-hoc work requests are the primary interruptions that shift our focus during the day. As a result, it takes longer to get things done. We need to try our best to limit these interruptions as much as possible, even if it means, saying 'No' to your peers. 

 

Encouraging Distractions

How many hours or minutes do you spend on your phone every day on personal things at work ? 

Distractions are actions that we knowingly take to delay or avoid work. Things such as texting on your phone, long phone calls, reading or replying to social media posts, tracking stock prices, checking your fitbit or smartwatch, browsing for deals or coupons etc. are some of the main distractions that we have at work. These distractions reduce our focus on the task at hand and must be minimized. 

 

Multi-tasking

A common myth is that people who can multi-task are high-performers. However, it is just the opposite. The fact is that we are not good at executing multiple complex tasks at the same time. When we say we are multi-tasking, we are actually switch-tasking which means that we switch between one task to another. Switch-tasking slows you down as you spend more time and effort to re-focus on the first task once you have switched to the second one.

Instead, focus on completing the task that you are currently working on, before starting a new one.

 

Procrastination

Spending your time on an unimportant task to delay the start of the most important task is procrastination. The main causes of procrastination are lack of focus, fear of failure, or excessive perfectionism. The best way to overcome procrastination is to break down your goal into smaller tasks, know the bigger purpose or the outcome, overcome your fears, and get started. Stop procrastinating today to achieve higher productivity at work. 

 

More articles:

 

Other Books: