Agile Metrics: Output Vs Outcome

Before we dive into Output Vs Outcome-Based Agile Metrics, let us understand what is meant by output-based and outcome-based metrics. Output-Based metrics measure the amount of work completed whereas Outcome-Based metrics measure the value delivered by the completed work.

Output-Based Agile Metrics

Metrics that measure how much work was completed in a given duration are said to be output-based. For example, the number of user stories completed in a Program Increment (PI) or an iteration measures the number of tickets completed in a specified duration. Other examples are:

  • Number of features completed in a Program Increment
  • Number of story points completed in a Program Increment
  • Number of story points completed in a sprint or an iteration

Such metrics can be easily gamed and do not provide any value-add. For example, Team A delivers 6 user stories whereas Team B delivers 12 user stories. This metric does not signify that Team B has better productivity than Team A.

Let’s assume Team A delivers 40 story points and Team B delivers 20 story points. Do you think Team A is a better performing team than Team B? Well, a team’s velocity does not provide insight into a team’s performance. Team A might have inflated their story point estimates or they might be working on user stories that do not deliver a potentially shippable product increment or bring any business value. 

Measuring the amount of work done is an Agile anti-pattern and should be avoided. It does not provide any value and may encourage Agile anti-patterns such that teams write tasks as user stories, create unnecessary user stories that do not bring any business value, and inflate story point estimates.

Outcome-Based Agile Metrics

Metrics that measure the outcome or the end-value delivered by the completed work are said to be outcome-based. For example, the business value delivered is an outcome-based metric that measures the amount of business value delivered by a unit of work during a specific duration. Let’s discuss how to measure the business value for an EPIC, Feature, or User Story. 

How to Measure the Business Value

Measure the delivered business value at an EPIC or a Feature level, rather than at a User Story level. Think in terms of:

  • Will this EPIC or Feature enable you to sell more units?
  • Will this EPIC or Feature allow you or your organization to make more profit?
  • Will this EPIC or Feature enable reduced operating costs for your business?
  • Will this EPIC or Feature enable a new capability for your end-users?
  • Will this EPIC or Feature reduce the manual effort for your team or end-users?
  • Does this EPIC or Feature enable your organization to compete in the market?
  • Does this EPIC or Feature allow to refine hypotheses about the market? 
  • Is this EPIC or Feature required to stay compliant with regulations?

Many EPICs or Features provide little to no value or aren’t actually desired by end-users. It’s important to prioritize the work effectively to deliver more business value. There are many ways to measure the business value like comparing the cost of delay, calculating Return On Investment (ROI), calculating revenues and expenses each month (Cash-Flow analysis), calculating Net Present Value (NPV), Planning Poker, and more. I suggest working with your business sponsors and:

  • Find Key Value Indicators (KVIs) for your product that you can measure at each Product Increment (PI) boundary.
  • Think about both qualitative and quantitative value indicators.
  • Review your KVIs regularly and revise as needed.

One of the simpler methods is to leverage the Planning Poker game and relatively estimate the business value for an EPIC or a Feature with your product stakeholders.

If you are interested to learn about other Outcome-Based Agile Metrics, enroll in my Udemy Course, Agile Metrics: Elevate Team Agility with Scrum Reporting.

You may be interested in my published courses and books as outlined below: 

Also, check out these articles…

Scrum and Kanban

Scrum and Kanban

What is the Agile Scrum Framework? How is it different from the Kanban approach. If you are applying the Scrum framework and leveraging the Kanban board, you might be thinking that Scrum and Kanban are the same. In addition, you might be wondering which methodology is a better fit for your work.

In this article, I have listed the core differences between Scrum and Kanban. Moreover, I also covered which framework is best suited with different type of work or teams.

 

Scrum in a nutshell

Scrum is an iterative and incremental Agile process framework to build complex products of the highest possible value. In Scrum, the team always works on the highest priority items first. The work is performed in short, time-boxed iterations or sprints. Each sprint begins when the team commits to complete prioritized user-stories based on their available capacity in the sprint. An iteration ends when the team has delivered a potentially shippable product increment of the product. Therefore, the scrum development team delivers business value to users at the end of each sprint.

The three Scrum roles - Product Owner, Scrum Master, and the Development Team - are defined below: 

  • A Product Owner decides what needs to be built and in what order.
  • A Scrum Master acts as a servant leader and coaches the team to follow Agile Scrum principles.
  • A Development Team is a group of self-organizing individuals who develop a high-quality product.

Scrum requires the below ceremonies to be conducted on a regular cadence:

  • Product Backlog Refinement
  • Sprint Planning
  • Daily Stand-Up
  • Sprint Review
  • Sprint Retrospective

Above all, Scrum is most suited for complex projects where things are more unpredictable than they are predictable. In complex domains, there is a need to collaborate with others, have an innovative mindset to investigate, experiment with different ideas, and adapt based on the learnings.

To learn more about Scrum, you may read my book, The Basics Of Scrum – A Simple Handbook to the Most Popular Agile Scrum Framework. 

 

Kanban in a nutshell

Kanban is the most popular Lean framework. This approach works best for teams that have a continuous flow of incoming requests with different priorities. In the Kanban approach, each request or a work item is represented by a Kanban card that flows from one stage of the workflow to another until it is complete.

Kanban is very flexible in nature. New work items can be added to the backlog at any time. Even the workflow can change anytime. If team capacity changes, WIP limits get recalibrated. In addition, Kanban does not prescribe any roles or ceremonies. It optimizes an existing process by eliminating waste and improving time to market.

To learn more about Kanban, you may read my other book, The Basics Of Kanban - A Popular Lean Framework

 

Compare Scrum and Kanban

Below section lists the core differences between Scrum and Kanban:

  • Scrum was formulated for complex product development to mitigate the limitations with the traditional Waterfall approach. Kanban, on the contrary, originated to manage work and control inventory at Toyota with Just-In-Time and Lean principles.

 

  • Scrum prescribes product teams to manage work within time-boxed, short, and consistent length iterations or sprints. However, Kanban specifies a continuous flow of work across different states.

 

  • With Scrum, development teams create a potentially shippable product increment at the end of every sprint. Therefore, teams can release code at the end of every sprint if approved by the product owner. With Kanban, teams can release code anytime or on-demand. 

 

  • Scrum requires three roles - Product Owner, Scrum Master, the Development Team. Kanban, on the other hand, does not prescribe any specific role.

 

  • In Scrum, the smallest piece of business value that a team delivers is a user story. Each user story may then be broken down into smaller tasks or sub-tasks. However, in Kanban, each work item is represented as a Kanban card.

 

  • Scrum is best suited for complex product development efforts that are unpredicatable in nature. Such complex efforts require research, experimentation, and an emergent design. Kanban, on the contrary, is best suited for simple and complicated efforts where things are more predictable than they are unpredictable.

 

  • With Scrum, sprint review and retrospective ceremonies are conducted at the end of every sprint to inspect and adapt. Though Kanban does not prescribe any ceremonies, teams may conduct a review meeting on a monthly or a quarterly cadence to review cycle time, flow efficiency, etc.

 

  • Scrum prescribes user stories to be estimated in terms of story points. However, Kanban does not require work items or Kanban cards to be estimated. In Kanban, estimation is optional. Some teams choose to estimate their work to have more predictability while others prefer to split their cards such that each of the cards is of the same size.

 

  • With Scrum, the most popular metrics are sprint burndown and velocity. Other useful metrics are release burndown, release burnup, and sprint burnup. The most popular Kanban metrics is cycle time. Metrics such as lead time, throughput, cummulative flow diagram (CFD), and control charts are also leveraged.

 

  • Kanban has more flexibility than Scrum as new work items can be added to the workflow at any time. 

 

  • Scrum prescribes ceremonies to be conducted on a regular cadence. For instance, the sprint planning ceremony must be conducted at the start of each sprint. Sprint review and retrospective ceremonies must be conducted at the end of each sprint. In addition, the daily stand-up must be conducted each day of the sprint. However, Kanban does not prescribe any cadence or ceremonies to be conducted. In Kanban, meetings are held as needed.

 

  • With Scrum, additional or new user stories should not be added to the active or an ongoing sprint. However, in Kanban, new work items or cards can be added anytime, provided the WIP (Work-In-Progress) limit hasn't reached yet.

 

  • In Scrum, the sprint backlog is reset after every sprint. However, the Kanban board is continuous.

 

  • To adopt Scrum, enterprises need to develop an agile mindset. Scrum requires a considerable change to the existing organizational structure and processes. As a result, leaders invest into Scrum training and create new roles or positions to build the best Scrum teams. On the contrary, Kanban does not require any significant changes to onboard onto this framework.

 

Now that you understand the differences between the two frameworks, you can decide which approach works best for your team. For more on Agile, Lean, Scrum, or Kanban, you may read below books:

What is a Sprint Burndown Chart?

The Sprint Burndown or the Iteration Burndown chart is a powerful tool to communicate daily progress to the stakeholders. It tracks the completion of work for a given sprint or an iteration. The horizontal axis represents the days within a Sprint. The vertical axis represents the hours remaining to complete the committed work. The purpose of a sprint burndown chart is to show the total amount of work remaining.

Let's take an example. The below table shows the actual number of hours remaining at the end of each day within a Sprint to create a sample Sprint burndown chart. The table also captures data for ideal remaining hours. The ideal remaining effort is calculated by assuming a uniform rate of task completion each day. In the below example, you can see that the ideal and actual remaining hours were same at the beginning of the sprint, however, as the sprint progresses, the actual remaining hours vary from the ideal remaining hours. 

Sprint Burndown Chart Table

Next, we will plot a sample Sprint Burndown chart using the above table data. The below diagram depicts the sample chart with ‘Date’ represented on the horizontal axis and ‘Remaining Effort (Hours)’ represented on the vertical axis. 

Sample Sprint Burndown Chart

From the above chart, you can conclude that:

  • There were no spillover stories since the actual remaining hours at the end of the sprint were zero.
  • The team started a bit slow in the first week of the sprint, but then caught-up during the second week.

Other articles that you may be interested in:

Learn all about Agile Scrum with my book, The Basics Of SCRUM. Other Books:

How to manage estimate inflation?

How to manage estimate inflation?

What is estimate inflation?

Estimate inflation is the term used when team estimates start growing over time. If a product backlog item was earlier estimated as 3 story points and a similar item is now estimated as 5 story points, this is referred to as estimate inflation.

What causes estimate inflation?

Higher management, often, mistakes high velocity as high productivity and put a tremendous pressure on the scrum teams to increase their velocity. In order to achieve a target velocity, the team then starts inflating their story point estimates. This inflation has a ripple effect on other product backlog items that have not yet been estimated. During planning poker, when team estimates a product backlog item, they compare their story to an already inflated story and provide another inflated estimate. 

How to limit estimate inflation?

Mike Cohn recommends comparing the product backlog item being estimated to two or more other items to ensure consistency among estimates. When you compare the item with two or more backlog items during planning poker, the probability to compare against inflated estimates is reduced. 

What are the challenges to this approach? 

Time is the biggest challenge. The development team does not like to spend additional time during planning poker to compare the item against multiple backlog items. 

What is Technical Debt?

If you are applying the Agile Scrum Framework at work, you must have heard the term, Technical Debt. So, what is the Technical Debt? Who invented this term? In this article, you will understand what it means and how to manage it.

 

The Definition

Ward Cunningham introduced the concept of technical debt in 1992. He defined it as follows:

“Shipping first-time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite…. The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.”

Cunningham used the ‘Technical Debt’ metaphor to emphasize the benefits and limitations of speedy development. The metaphor was well received by both business and technical people as it resonates with the financial debt. Like the financial debt, this debt accumulates interest with late repayment.

 

In 2004, Joshua Kerievsky describes ‘design debt’ in his article ‘Refactoring to Patterns” and the associated costs. Then again in 2014, Grady Booch compared evolving cities to evolving software and described how lack of refactoring can lead to the technical debt. He stated:

“The concept of the technical debt is central to understanding the forces that weigh upon systems, for it often explains where, how, and why a system is stressed. In cities, repairs on infrastructure are often delayed and incremental changes are made rather than bold ones. So, it is again in software-intensive systems. Users suffer the consequences of capricious complexity, delayed improvements, and insufficient incremental change; the developers who evolve such systems suffer the slings and arrows of never being able to write quality code because they are always trying to catch up.”

 

Main Causes of the Technical Debt:

Review your product, system, code, practices, and processes for:

  • Outdated design
  • Low code coverage
  • Lack of automated regression test suite
  • Lack of continuous integration
  • Lack of appropriate testing
  • Tightly-coupled code
  • Delayed code refactoring
  • Aggressive timelines

 

How to manage the Technical Debt:

An organization should enforce good technical practices to all the developers, including but not limited to technical design review, code review, automated regression suite, high code coverage, adequate testing, and continuous integration. This will limit the accrual of this debt. The products that have accumulated the debt should focus on refactoring to reduce technical debt and improve maintainability. Below techniques will reduce your debt:

  • Have a strong definition of done
  • Create visibility for the tech debt
  • Prioritize the tech debt
  • Leverage an incremental approach

 

Learn more about the technical debt and other concepts of the Agile Scrum Framework with my book, The Basics of SCRUM: A Simple Handbook to the Most Popular Agile Scrum Framework, available on Amazon.

 

More articles:

 

Recommended books: