Driving Towards Agile Metrics

What can velocity tell us?

In my last two blog entries, I discussed scaling the Sprint to a roadmap and scaling the User Story to a full Requirements Breakdown Structure (RBS). I mentioned that I would be attaching “Metrics” to the RBS model as an entity hanging off Features.

Before moving on to a birds-eye view of Agile metric, I’d like to establish a shared understanding of a few other concepts.

In this entry, let’s discuss velocity.  In bringing up the topic of Agile metrics, I’ve often heard, “Metrics? Do you mean velocity?”  Yes, velocity is a numeric value which can be tracked, but what does it really measure?

Productivity or Predictability?

The Velocity Chart: Story Points per Sprint

Simply put, velocity doesn’t measure the productivity of a team.  It measures a team’s ability to accurately estimate its work.

What exactly is velocity?  It’s the number of Story Points of effort a team can accomplish in a Sprint.  As you probably know, Story Points are arbitrary units of measure used to “size” one Story relative to another.  Over time (usually 3 - 5 Sprints), a team comes to a shared understanding of what a Story Point is and how many it can consistently execute in one Sprint.

In the chart to the right, you’ll notice a wider fluctuation in the number of Story Points accomplished in Sprints 1 - 4. By Sprint 5, the team begins leveling out at about 16 Story Points per Sprint. What does this mean? It means that the team can estimate and deliver work more consistently.

Starting the Car

Coming up with an initial velocity estimate is similar to estimating how fast a race car can go.  You can invest in collecting and analyzing detailed data to develop your estimate from the bottom up, or you can take a leaner, heuristic approach.   What if you were to say, “Based on my 10 years of experience driving race cars, I think I can comfortably sustain a pace on this race track of 110 mph.”  To validate this estimate, you could then hop into the car, drive it around the track a few times, watch your actual velocity, and adjust your estimate based on real-time feedback.

Estimating Arrival Time

In this unpredictable world we are not all driving race cars on race tracks.  Is a car which can go 110 mph around a smooth track better than a pickup truck which can drive 80 mph over back roads?  It depends on business needs and market conditions.

So what’s the real value of knowing your velocity?  The value is in being able to consistently predict when you’ll arrive at your next destination.  And the next.  And the next.

There will always be anomalies: a key team member is sick with swine flu or a Microsoft critical update downs all database servers.  In such situations, the velocity for that Sprint can be footnoted.  It’s no more than a pot hole on a stretch of highway which blows a tire.  You fix it and keep moving.

Breaking Down a Plan

Sprint to Roadmap, Roadmap to Sprint (click to enlarge)

In my last post, we saw how we can break down an enterprise strategic plan into more granular levels of detail: the roadmap to the project to the release to the Sprint. At higher levels, we have less detail. Remember that Lean principles support the management practices of adaptive, feedback-based planning and just-in-time elaboration of detail.

As we break down our planning to greater levels of detail, we also need to break down our requirements specifications. You’re likely familiar with the Story and Verification Condition formats, which provide enough granular detail to define a Sprint’s tasks. However, in longer term strategic planning, such detail is waste. Why spend the resources to define detail which will likely change or may never be used?

As we plan at the highest strategic level and then drill down to projects, releases, and Sprints, we add more and more detail. Much like the work breakdown structure (WBS) in traditional project planning, we can break down our requirements to additional levels of detail.

Mapping the Plan to the Requirements Breakdown Structure

Since a picture is worth a thousand words, let’s let an image show the breakdown of requirements. The following is inspired by Dean Leffingwell’s work (see his blog post), discussion groups, and my own client work.

An Agile Requirements Breakdown Structure

Defining Requirements Levels

At the roadmap and project levels, planning is more structured. The plans are still reviewed and adapted, but less frequently than at the release and Sprint levels. Let’s take a closer look at definitions and examples.

The Initiative

One or more initiatives make up a roadmap and is the result of the strategic planning process. It delivers the capabilities to support the product strategy. Note that Scrum uses the term “Product” though many companies use the term “application.”

Example: Add paid subscriptions to the current free subscription model.

Epics

Initiatives are broken down into Epics to deliver a project. Epics are high level, abstracted requirements which deliver a portion of the initiative. One or more make up a project.

Example: Attract Premium subscribers by offering new, more desirable ways to receive Premium subscriber content.

Epics are typically defined in the project charter and support the business case.

Features

Epics are broken down into Features to deliver a release. Features provide the bridge between the business value and the end user value.

Example: Offer iPhone access so that the business can sell premium memberships to more iPhone users BECAUSE Premium subscribers can access their content on-the-go.

Both business value and end user value can be measured through quantitative and qualitative metrics. For example, the business value metric may be Sell 5,000 premium memberships in the first three months. The end user value metric may be Premium subscribers access their premium content one or more times per day.

Features can be estimated, often in Story points. They can also be tested through high level user acceptance tests.

Stories

Features are broken down into Stories to deliver a Sprint. The User Story specifies who, what, and why from the end user’s perspective, but not how. Because it does not tell how, it opens a conversation with the team about how the functionality can best be implemented, leveraging the team’s collective expertise.

Example: As a Premium subscriber and iPhone user, my subscription preferences apply to any interface I am using so that I only need to update them in one place.

Stories can be estimated, often in Story points. They can also be tested through verification conditions, which specify both the tests and the business logic (see Test Driven Development).

But Is It Agile?

Requirements decomposition has existed since the early days of Waterfall, but is it Agile? The answer is yes for two reasons. First, we follow Lean principles and avoid waste by drilling down into further levels of detail only when needed. We also drill down as late as possible (often know as the last responsible moment, or LRM) in order to leverage the latest available information. Second, we more frequently review and adapt at all levels in order to be more responsive to ambiguity and changing market conditions.

In my next blog post, I’ll take a look at Agile metrics.

Building Upon Itself

The famous Mendelbrot Set fractal

At the core of team-based Scrum management practices is the Sprint, a 2 - 4 week iteration which results in an increment of software which can be demoed or released. Like a fractal, the Sprint can be scaled to a product roadmap with several stops in between.

What is a fractal? It’s a geometric shape built from smaller versions of itself.  This shape then becomes one piece in a larger version of itself, which in turns makes up an even larger version… and so on.

Can the Sprint, complete with its adaptive planning and just-in-time elaboration of detail, build upon itself? The answer is yes. However, in enterprise planning the reverse usually happens: the product roadmap it broken down after strategic planning takes place.

Sprint to Roadmap, Roadmap to Sprint

The terminology for taking a roadmap and breaking it down to the Sprint level after setting (or reviewing and adapting) the strategic plan provides the foundation for my next blog posting on the Agile Requirements Breakdown Structure (RBS). Since a picture is worth a thousand words, let’s let an image show the relationship between sprints, releases, projects, and roadmaps:

From Sprint to Roadmap

The Roadmap

The roadmap comes out of the strategic planning process and provides the vision for a software product or application. For smaller applications, it may look only six month into the future. More complex applications may have a three year roadmap. Keep in mind that an agile enterprise will review and adapt the roadmap periodically, often in conjunction with the budgeting process.

The Project

Roadmaps break down into projects. At this level, the business case is made, often in the form of a project charter. ROI is forecasted and budget approvals takes place.

The Release

Projects break down into releases. Releases result in the internal or external release of a version of software. Before software is released, heavy “hardening” takes place to test business logic, code integrity, scalability, and the other “ities” like security and usability.

The Sprint

Releases break down into Sprints. Sprints result in demos to gather feedback from constituents and to keep the business engaged. Lighter hardening takes place to balance the value of early testing with the cost associated with heavy hardening.

Round-trip from Sprint to Roadmap

Though decomposition from the higher to the lower level is a necessary part of planning, beginning with the Sprint can be critical to proving a concept or testing an assumption before investing too much in the planning process. Remember that decisions based on feedback are more reliable than those based on forecasts and speculation. As well, failing fast is as important as succeeding fast! Nothing can chew up a budget quicker than running off in the wrong direction.

Sprint to roadmap. Roadmap to Sprint. It’s often a round-trip!

In my next blog post, we’ll take a look at the breakdown of requirements at each level of planning.