Driving Towards Agile Metrics

What can velocity tell us?

In my last two blog entries, I discussed scaling the Sprint to a roadmap and scaling the User Story to a full Requirements Breakdown Structure (RBS). I mentioned that I would be attaching “Metrics” to the RBS model as an entity hanging off Features.

Before moving on to a birds-eye view of Agile metric, I’d like to establish a shared understanding of a few other concepts.

In this entry, let’s discuss velocity.  In bringing up the topic of Agile metrics, I’ve often heard, “Metrics? Do you mean velocity?”  Yes, velocity is a numeric value which can be tracked, but what does it really measure?

Productivity or Predictability?

The Velocity Chart: Story Points per Sprint

Simply put, velocity doesn’t measure the productivity of a team.  It measures a team’s ability to accurately estimate its work.

What exactly is velocity?  It’s the number of Story Points of effort a team can accomplish in a Sprint.  As you probably know, Story Points are arbitrary units of measure used to “size” one Story relative to another.  Over time (usually 3 - 5 Sprints), a team comes to a shared understanding of what a Story Point is and how many it can consistently execute in one Sprint.

In the chart to the right, you’ll notice a wider fluctuation in the number of Story Points accomplished in Sprints 1 - 4. By Sprint 5, the team begins leveling out at about 16 Story Points per Sprint. What does this mean? It means that the team can estimate and deliver work more consistently.

Starting the Car

Coming up with an initial velocity estimate is similar to estimating how fast a race car can go.  You can invest in collecting and analyzing detailed data to develop your estimate from the bottom up, or you can take a leaner, heuristic approach.   What if you were to say, “Based on my 10 years of experience driving race cars, I think I can comfortably sustain a pace on this race track of 110 mph.”  To validate this estimate, you could then hop into the car, drive it around the track a few times, watch your actual velocity, and adjust your estimate based on real-time feedback.

Estimating Arrival Time

In this unpredictable world we are not all driving race cars on race tracks.  Is a car which can go 110 mph around a smooth track better than a pickup truck which can drive 80 mph over back roads?  It depends on business needs and market conditions.

So what’s the real value of knowing your velocity?  The value is in being able to consistently predict when you’ll arrive at your next destination.  And the next.  And the next.

There will always be anomalies: a key team member is sick with swine flu or a Microsoft critical update downs all database servers.  In such situations, the velocity for that Sprint can be footnoted.  It’s no more than a pot hole on a stretch of highway which blows a tire.  You fix it and keep moving.

Breaking Down a Plan

Sprint to Roadmap, Roadmap to Sprint (click to enlarge)

In my last post, we saw how we can break down an enterprise strategic plan into more granular levels of detail: the roadmap to the project to the release to the Sprint. At higher levels, we have less detail. Remember that Lean principles support the management practices of adaptive, feedback-based planning and just-in-time elaboration of detail.

As we break down our planning to greater levels of detail, we also need to break down our requirements specifications. You’re likely familiar with the Story and Verification Condition formats, which provide enough granular detail to define a Sprint’s tasks. However, in longer term strategic planning, such detail is waste. Why spend the resources to define detail which will likely change or may never be used?

As we plan at the highest strategic level and then drill down to projects, releases, and Sprints, we add more and more detail. Much like the work breakdown structure (WBS) in traditional project planning, we can break down our requirements to additional levels of detail.

Mapping the Plan to the Requirements Breakdown Structure

Since a picture is worth a thousand words, let’s let an image show the breakdown of requirements. The following is inspired by Dean Leffingwell’s work (see his blog post), discussion groups, and my own client work.

An Agile Requirements Breakdown Structure

Defining Requirements Levels

At the roadmap and project levels, planning is more structured. The plans are still reviewed and adapted, but less frequently than at the release and Sprint levels. Let’s take a closer look at definitions and examples.

The Initiative

One or more initiatives make up a roadmap and is the result of the strategic planning process. It delivers the capabilities to support the product strategy. Note that Scrum uses the term “Product” though many companies use the term “application.”

Example: Add paid subscriptions to the current free subscription model.

Epics

Initiatives are broken down into Epics to deliver a project. Epics are high level, abstracted requirements which deliver a portion of the initiative. One or more make up a project.

Example: Attract Premium subscribers by offering new, more desirable ways to receive Premium subscriber content.

Epics are typically defined in the project charter and support the business case.

Features

Epics are broken down into Features to deliver a release. Features provide the bridge between the business value and the end user value.

Example: Offer iPhone access so that the business can sell premium memberships to more iPhone users BECAUSE Premium subscribers can access their content on-the-go.

Both business value and end user value can be measured through quantitative and qualitative metrics. For example, the business value metric may be Sell 5,000 premium memberships in the first three months. The end user value metric may be Premium subscribers access their premium content one or more times per day.

Features can be estimated, often in Story points. They can also be tested through high level user acceptance tests.

Stories

Features are broken down into Stories to deliver a Sprint. The User Story specifies who, what, and why from the end user’s perspective, but not how. Because it does not tell how, it opens a conversation with the team about how the functionality can best be implemented, leveraging the team’s collective expertise.

Example: As a Premium subscriber and iPhone user, my subscription preferences apply to any interface I am using so that I only need to update them in one place.

Stories can be estimated, often in Story points. They can also be tested through verification conditions, which specify both the tests and the business logic (see Test Driven Development).

But Is It Agile?

Requirements decomposition has existed since the early days of Waterfall, but is it Agile? The answer is yes for two reasons. First, we follow Lean principles and avoid waste by drilling down into further levels of detail only when needed. We also drill down as late as possible (often know as the last responsible moment, or LRM) in order to leverage the latest available information. Second, we more frequently review and adapt at all levels in order to be more responsive to ambiguity and changing market conditions.

In my next blog post, I’ll take a look at Agile metrics.

Building Upon Itself

The famous Mendelbrot Set fractal

At the core of team-based Scrum management practices is the Sprint, a 2 - 4 week iteration which results in an increment of software which can be demoed or released. Like a fractal, the Sprint can be scaled to a product roadmap with several stops in between.

What is a fractal? It’s a geometric shape built from smaller versions of itself.  This shape then becomes one piece in a larger version of itself, which in turns makes up an even larger version… and so on.

Can the Sprint, complete with its adaptive planning and just-in-time elaboration of detail, build upon itself? The answer is yes. However, in enterprise planning the reverse usually happens: the product roadmap it broken down after strategic planning takes place.

Sprint to Roadmap, Roadmap to Sprint

The terminology for taking a roadmap and breaking it down to the Sprint level after setting (or reviewing and adapting) the strategic plan provides the foundation for my next blog posting on the Agile Requirements Breakdown Structure (RBS). Since a picture is worth a thousand words, let’s let an image show the relationship between sprints, releases, projects, and roadmaps:

From Sprint to Roadmap

The Roadmap

The roadmap comes out of the strategic planning process and provides the vision for a software product or application. For smaller applications, it may look only six month into the future. More complex applications may have a three year roadmap. Keep in mind that an agile enterprise will review and adapt the roadmap periodically, often in conjunction with the budgeting process.

The Project

Roadmaps break down into projects. At this level, the business case is made, often in the form of a project charter. ROI is forecasted and budget approvals takes place.

The Release

Projects break down into releases. Releases result in the internal or external release of a version of software. Before software is released, heavy “hardening” takes place to test business logic, code integrity, scalability, and the other “ities” like security and usability.

The Sprint

Releases break down into Sprints. Sprints result in demos to gather feedback from constituents and to keep the business engaged. Lighter hardening takes place to balance the value of early testing with the cost associated with heavy hardening.

Round-trip from Sprint to Roadmap

Though decomposition from the higher to the lower level is a necessary part of planning, beginning with the Sprint can be critical to proving a concept or testing an assumption before investing too much in the planning process. Remember that decisions based on feedback are more reliable than those based on forecasts and speculation. As well, failing fast is as important as succeeding fast! Nothing can chew up a budget quicker than running off in the wrong direction.

Sprint to roadmap. Roadmap to Sprint. It’s often a round-trip!

In my next blog post, we’ll take a look at the breakdown of requirements at each level of planning.

Agile’s Head: Lean

A mind to know, two hands to act

In two previous blog entries, we saw how Lean Software Development, with roots back to Lean Manufacturing, can provide the values which derive the underlying business value.  In some ways, it acts as the “head,” telling us the “why” behind that “whats” and “hows” of Agile.

Agile’s Two Hands: Scrum and XP

In addition to the “why,” we also need to know “what” and “how.”  I view these as the hands of Agile: one to guide the management practices and one to guide the software engineering practices.

But with so many different Agile methods, how do we choose?  Fortunately, the answer is quickly emerging.  Scrum has wide market buy-in as an Agile management framework, though it lacks any prescribed engineering practices.  Extreme Programming (XP) also has significant buy-in.  While XP has a less developed management framework, its strength lies in its quality-driven engineering practices.

Head and Hands: Lean, Scrum, and XP

We previously explored Lean’s values.  At a high level, you can see the framework emerge.

Eliminate waste Minimize inventory Maximize flow / reduce work-in-process Pull from demand / decide as late as possible Empower workers Meet customer requirements Do it right the first time Abolish local optimization / favor adaptability Partner with suppliers Create a culture of continuous improvement

Simple, clear roles and cross-trained resources Self-organizing teams Short iterations Daily Stand-up meetings Just-in-Time requirements elaboration Continuous requirements inspection and prioritization Negotiable scope Up-front acceptance tests Transparency Simple management tools Continuous process improvement

Test-driven development and automated testing Continuous integration Simple design and incremental architecture Refactoring System metaphor Coding standards Pair programming and a sustainable pace Collective code ownership Spike solutions

This model also allows us to frame Agile readiness and Agile maturity in this way.  Is the organization ready to embrace Lean principles?  How prepared are teams for Scrum’s management processes?  Are the technical infrastructure and software engineering practices ready to become more “extreme” with Extreme Programming?

We’ve already looked at Lean values.  In future entries, we’ll dive deeper into Scrum management practices and XP software engineering practices.  From there, we’ve laid our foundation for evaluating readiness and measuring maturity.

Birthplace of Lean Manufacturing

In Part 1, we discussed how Agile values can provide a foundation for Agile management practices. However, through the time-tested rules of Lean Manufacturing, we can learn more about the Agile value proposition.

Mapping Lean Manufacturing to Agile

There are ten rules which can sum up the fundamental practices of Lean Manufacturing in the 1980’s. By looking at both Lean Manufacturing and Lean Software Development, the “why” emerges for the Agile management practices. These management practices can then be traced to the fundamental goal of producing more value with less work.

Getting the Value

Rule 1: Eliminate Waste

In Lean Manufacturing…

• Value Stream Analysis is used to identify all activities in the manufacturing process and the value they add to the final product, not the process. The process is then re-evaluated to find the most efficient way to add the intended product value.

In Agile…

• The biggest waste in software development comes from building features which will be outdated or have little value when finally released. A high priority feature today may be irrelevant in a year. In Agile, the business re-evaluates requirements against changing customer needs before each Sprint so that only the highest priority ones are built.
• In Agile, the User Stories are only frozen for the Sprint (though how they are implemented is negotiated throughout). The business is less hesitant to commit to requirements for the Sprint because they have the opportunity to re-prioritize before each Sprint. They waste less time on forecasting needs and the sign-off process.
• The production of documentation which has no value after the completion of an iteration translates to the scrutiny of document production. Which documents add value to the final product, and which simply support the process? Agile replaces verbose requirements documentation with light-weight User Stories and Validation Conditions. The design is then elaborated through conversation in the Sprint Planning Meeting and the daily Scrums.
• In the Sprint Planning meeting, only the highest priority requirements are broken down into tasks so that time is not wasted on features which may never be implemented. This process is often referred to as “just-in-time elaboration.” Time is not wasted on details which may never be needed.

Rule 2: Minimize Inventory

In Lean Manufacturing…

• Inventory is minimized because it costs money, consumes resources to manage, and becomes obsolete.

In Agile…

• The inventory of requirements is kept at a high level and broken down “just in time” so that unnecessary details don’t need to be managed. During the Release Planning process, high level Epics are created, but not broken down into stories until they are targeted for a release.
• Epics and stories are managed by the business and regularly pruned to eliminate the constant review and re-shuffling of the low or no-value items.

Rule 3: Maximize Flow / Reduce Work-In-Process

In Lean Manufacturing…

• Smaller batches and faster cycle times increase the flow of products from the manufacturer to the customer.

In Agile…

• Short Sprint cycles produce completed units of functionality which can be released to the customer. This allows a smoother flow of features, more frequent customer feedback, and faster responsiveness to change.

Rule 4: Pull From Demand / Decide as Late as Possible

In Lean Manufacturing…

• Pulling from actual demand reduces reliance on and investment in forecasting models. Driving from actual demand is more reliable in volatile environments.

In Agile…

• Freezing requirements at the beginning of a long development process relies heavily on forecasting. By re-prioritizing the Product Backlog before each short Sprint begins, features are driven by actual demand rather than speculation.
• Rather than trying to forecast architecture needs, Agile software engineering practices keep the initial architecture design light-weight and allows for refactoring (reworking of code). This keeps designs simpler, but allows additional architecture elements to be added if needed.

Rule 5: Empower Workers

In Lean Manufacturing…

• Teams are trained in work measurement and improvement techniques. Changes which are driven from the workers “on the floor” improve productivity and morale over changes mandated by management.

In Agile…

• Teams are self organizing, and re-evaluate their processes after each Sprint in the Retrospective Meeting.
• The team is empowered to decide how the stories are implemented. Through User Stories, the Product Owner defines the “who,” the “what,” and the “why.” The “how” is collaboratively developed by the team.

Rule 6: Meet Customer Requirements

In Lean Manufacturing…

• This rule is self-evident, and follows Rule 1 (Eliminate Waste). Manufacturing products which no one buys results in inventory which will never be sold.

In Agile…

• In traditional Waterfall, success is measured by delivering the required features on time and on budget. The delivery of the right requirements is often overlooked. As in Rule 1, the creation of low or no-value features is the most wasteful aspect of software development. Success is measured by delivering only the highest priority features each release in a “good” or “good enough” manner.
• Before the Sprint begins, the business creates validation conditions for each User Story. These validation conditions become the business’ acceptance test criteria. This means coding is driven by tests written by the business, not designs and tests created by the developers.
• During the Sprint Review meeting at the end of each Sprint, the functionality is reviewed with the business to confirm that the requirements have been met.

Rule 7: Do it Right the First Time

In Lean Manufacturing…

• Lean integrates quality at the source, putting controls in place at each step in the process. Before Lean, manufacturing was optimized for product throughput. At the end of the process, tests were conducted and defects were fixed at rework stations.

In Agile…

• Tests are written before code, and defects are fixed at each step in the development process. Barry Boehm observed that it costs 100 times more to fix a problem after release than early in the process.
• In an Agile organizational structure, QA is not a silo. Instead, QA personnel are integrated into each team.
• The change control process becomes simpler and less costly because fewer defects are passed on.

Rule 8: Abolish Local Optimization

In Lean Manufacturing…

• For rapidly changing markets requiring responsiveness over mass production, faster machine change-over is more important than the higher throughput of individually optimized stations.

In Agile…

• Cross training and team collaboration are more important than siloed roles. This allows team members to jump in where needed rather than passively waiting for someone else to complete a task or correct an issue blocking their progress.
• Teams are responsible for delivery rather than individuals. Rewarding individual performance (local optimization) rather than team performance detracts from each persons focus on the overall objective.

Rule 9: Partner With Suppliers

In Lean Manufacturing…

• Companies are able achieved the the greatest efficiencies in their supply chain by reducing the number of suppliers and working with them as partners. The efficiencies gained by trust, collaboration, improved product flow, just-in-time movement of goods, and less supplier turnover outweigh the savings which come from competitive bidding.

In Agile…

• Just as moving parts between suppliers is expensive, so is moving tasks between siloed departments. Agile distributes siloed departments into the teams.
• Integrating the business into each team with daily contact creates a partnership which provides just-in-time elaboration of business requirements and more cooperative relationships. Requirements are not “thrown over the wall” with visibility only at the end of a long development cycle.

Rule 10: Create a Culture of Continuous Improvement

In Lean Manufacturing…

• As Rule 5 (Empower Workers) states, everyone involved in the process has the power to improve it. In contrast, a command and control environments rewards adherence to a strict process. Assuming that processes can continuously be optimized, especially in changing markets, a culture is needed to support continuous improvement.

In Agile…

• At the end of each Sprint, the team conducts a Sprint Retrospective, discussing two simple questions: what went well and what can be improved. Software maturity models such as ISO9000 and the Software Engineering Institute’s (SEI) Capability Maturity Model (CMM) reward adherence to strict processes. It assumes that their specific process is right for every situation. Agile assumes that as markets change, requirements change, and the processes to meet those requirements change as well.

Agile: More Value with Less Work

Tracing Agile back to the time-tested rules of Lean Manufacturing, we begin to move from philosophical values to business value.

• Rather than telling a CEO that Agile values individuals and interactions over processes and tools, you can tell her that Agile eliminates waste and creates productive partnerships between all parts of the business.
• Rather than telling a CEO that Agile values working software over comprehensive documentation, you can tell her that Agile speeds up the delivery of valuable functionality to the customer.
• Rather than telling a CEO that Agile values responding to change over following a plan, you can tell her that Agile delivers functionality based on current demand rather than forecasted needs or outdated requirements.

And the list goes on: higher quality, continuous process improvement, and stronger collaboration. Your Agile value proposition is suddenly defined.

You’ve likely read the Agile Manifesto, the values which were developed by representatives from Extreme Programming, Scrum, DSDM, Feature Driven Development, Crystal, and other alternatives to the heavy, document-driven Waterfall methodology.

Agile Minds, Lean Thoughts

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The Manifesto statements offer traceability to many of the Agile management practices. If we use Scrum management practices for our examples, we can make the following connections:

• The daily 15-minute Scrum meeting with the team standing around the low tech task board supports the value of individuals and interactions over processes and tools
• The Sprint Planning meeting, where the light-weight User Stories are transformed into a shared understanding of the features to be implemented, supports the value of working software over comprehensive documentation
• The application of “good” versus “good enough” in order to deliver on time supports the value of customer collaboration over contract negotiation
• The re-prioritization of the Product Backlog before each Sprint Planning Meeting supports the value of responding to change over following a plan

Agile Values vs. Business Value

Though the Manifesto articulates the values that provide the foundation for the “what” and “how” of Agile, it doesn’t answer the question “why.”

Why are individuals and interactions more valuable than processes and tools? Many people could argue that individuals are inconsistent, their memories are fallible, and their interactions are unpredictable. Shouldn’t defined processes bring consistency and tools record decisions which may otherwise be lost or reinterpreted?

How do we move from Agile values to business value?

Finding the “why” in Lean

When the Agile Manifesto was developed in 2001 at the Snowbird Ski Resort in the Wasatch Mountains of Utah, Mary and Tom Poppendieck had not yet released their book on Lean Software Development.

Lean Software Development draws heavily from the rules of the Toyota Product System, later called Lean Manufacturing. Simply put, it is a time-tested method which focuses on providing more value with less work.

More value with less work. Now that’s something which any CEO could support.

In Part 2, we’ll map the value-driven rules of Lean Manufacturing to Agile management practices.

Hammering out UX

I see the same org chart in many companies.  User Experience Design (UX) is nestled tightly in with Software Engineering rather than the Product Team.  Often, they don’t become involved in the iteration until engineering  is involved.

Is this org structure agile?

Pre-planning with the Product Owner

I see the answer to the question in the pre-planning phase of a release.  The Product Owner has the inventory of product backlog items, often in the form of User Stories.  The Stories tell the who, the what, and the why without designing the how.  Yet to create the validation conditions, “how” comes into play.

How “how”?  There’s a leap which happens when moving from the Story to the verification conditions.  Let’s use my case study for AgileViews.com, a fictitious site which is adding a paid subscription plan to it’s free blog aggregation service.

As a subscriber,
I can receive my paid subscription content on my phone
So that I can keep up-to-date on high value information when I am away from my computer.

The product owner might then create the following verification criteria:

• Verify that paid content is integrated with the mobile RSS feed’s free content
• Verify that the paid content is branded with the provider’s logo
• …

You may begin to see the problem.  In coming to the Sprint Planning Meeting with the verification conditions, assumptions have been made about the implementation.  Should the information still be delivered through a mobile RSS feed?  Is a logo the best way to indicate a paid content source?

The step between stories and verification conditions

In the pre-planning phase, UX can provide a user-centered reality check on the jump from the story to the verification conditions.  Adding a “solution hypothesis” before developing verification conditions can minimize rework in the Sprint Planning Meeting, or at very least help the team question explicit assumptions.

Let’s take a look at the solution hypothesis for our story:

As a subscriber,
I can receive my paid subscription content on my phone
So I can keep up to date when I am away from my computer.

Solution Hypothesis:

The mobile interface for the paid content will be integrated with the unpaid content’s RSS feeds.

After reviewing the solution hypothesis, UX can then validate whether the hypothesis provides the best user experience, or whether new approaches or delivery vehicles can be leveraged.  If the strategy is to first convert existing users to paid subscribers, more research is warranted.  What if the vast majority are iPhone users?  Blackberry users?  A more usable interface may be possible.

Bring Product and UX together from the start

While it may not be easy to change a historic org structure, UX can still be tightly aligned with the Product Team in the iteration pre-planning.  It provides the least risk of wasted work and drives out a more solid user experience.

Bridge the gap and add a bit of “how” to the who, what, and why.