I am a big sports fan and since I work for a software metrics company, I started thinking about the similarities in productivity measurement in both industries. From draft picks to game planning decisions, managers in sports measure their team’s productivity to help them make better decisions in the future. Software executives and product owners do the same thing; they measure productivity in order to make better planning decisions regarding upcoming projects.

To measure productivity in baseball, we look at measures like batting average, on-base percentage, slugging percentage, and earned run average. When measuring the productivity of agile software projects, we often look at the velocity, which takes into account the number of user stories completed in each sprint. This type of historical data helps us plan effectively at the detailed level.

As part of our work at QSM, we are often asked to provide plans at the release and portfolio level. To provide these plans reliably, we use a macro level, empirically-based productivity measure that encapsulates a number of project-related factors. This measure is called the Productivity Index, an integral part of the Putnam Model.  Also known as the SLIM Model, the Putnam Model was invented by Larry Putnam Sr. almost 40 years ago and is having a big impact on software measurement more than ever today.

Once we know the total number of user stories (or any size measure), the release level effort, and the duration, we can calculate the Productivity Index of a project. The Productivity Index also takes into account the project environment including: the experience level of the team, the complexity of the software, and the quality of the tools and methods being used on the project.

For well over a decade, agile software development methods have been adopted by a wide variety of software organizations across the globe.  QSM has worked with these types of software organizations for more than 35 years to establish data-driven, defensible estimation and lifecycle management practices as the foundation of quality software projects and products. The QSM Agile Round Table was formed to provide a platform to brainstorm the role of estimation in agile environments, and chart a path toward better understanding for all stakeholders.  A mixture of long-standing and newer customers shared their questions, challenges, and experiences to answer the big question, and effectively communicate the relevance and benefits of scope-based estimation.  This article by QSM's Laura Zuber is the first of the QSM Agile Round Table series of publications that will present specific concepts and practices that connect SLIM and agile, creating common ground for the benefit of all.  It is our hope that this series will answer some of your questions, and that you will share your thoughts.  

Read the article!

Recently a friend of mine sent me a link to a YouTube video featuring Jeff Sutherland and Ken Schwaber, the founders of Scrum, discussing the latest update to the 2016 Scrum Guide.  The updates to the guide were comprised of survey feedback from the Scrum community, with the goal to understand what is important to them that should be included in the updated Scrum guide.  

The most compelling part of this discussion for anyone contemplating whether estimating belongs in Scrum happens at the 32:10 point.  Jeff and Ken are posed the question, “what is the difference between a forecast and an estimate?”   They answer with the idea that many people confuse the word estimate with commitment, which plagues any development effort regardless of method.  Subsequently Jeff and Ken changed the word “estimate” to “forecast” in the latest Scrum Guide to reflect the ever-changing growth and reduction in project scope, i.e., the estimate will show our best commitment and what we think is going to happen.  That can change once the project starts, BUT, we can measure that too through forecasting, essentially re-estimating throughout the project.  

There are so many questions around agile planning, one of the biggest being: do we need an estimate? Project managers and scrum masters will spend months developing a system either for internal use or for their clients, yet some of them say that estimates are not needed. Some recommend starting the project without an estimate. They say they will see how the first few weeks go before they generate an estimate. Others say not to worry about an estimate at all; they are a waste of time.

The problem with those recommendations is that there are business decisions that need to be made regarding whether or not to even start the project. Reliable estimates for cost and duration are needed to make these decisions. Also, for the projects that do move forward, there is usually limited information available early in the lifecycle, not enough to provide a detailed plan. Product owners need to see the big picture before a reliable detailed plan is generated.

There is also the IT manager that needs to figure out how they will allocate their resources. There is the vendor manager that needs to evaluate multiple bids for a large software system that could cost the company millions of dollars. There is the proposal manager that needs to write a proposal that must be cost competitive to win business. And, there is an annual budget at stake and the CIO needs to know how much money their development organization is going to spend over the next 12 months. You can’t support these decisions in the best way possible without reliable release level estimates.

Before an agile project starts, many product owners will run an early release estimate. Once the activities get started, managers or scrum masters begin to track the progress. When they track, they usually include the person hours of effort and the number of user stories within each sprint.  There are a number of agile tracking tools and methods in the marketplace for these tasks. 

But wouldn’t it be great if the tracking and estimation process could be combined, using the actual tracked effort and user stories to run new and improved ongoing estimates at the release level? At QSM, we have applied this process to hundreds of software projects. This type of adaptive forecasting can help save time and effort by showing when a software release is headed down the wrong path. It can also help organizations avoid signing up to inflated resource planning numbers that cause many companies to waste millions of dollars at the release and enterprise levels.

In the SLIM-Control charts below, we see the blue plans versus the red actuals and the new forecasts in white. We are capturing the total effort spent and the actual work delivered each week, then using that information to generate mathematical models that produce new empirically based forecasts at the release level.

Agile is about adapting to change, not completely abandoning documentation or dismissing helpful planning and estimating inputs. In this article for Projects at Work, QSM's Jay Daniel explains how the benefits of an agile approach can shine brighter when used in conjunction with a fundamental development practice such as sizing.

Jay Daniel is a Professional Services Manager with QSM's Consulting Services team. He is an IT professional that has served in a variety of consulting roles, ranging from Program and Project Management to providing Independent Verification & Validation (IV&V) support to clients. For the past five years, Jay has focused his attention on agile methodologies in the implementation of software development efforts. He is a certified project manager (PMP), scrum master (CSM), and product owner (CSPO).

Read the full article!

Often I speak with IT project measurement folks about the permeation of agile into their project estimation processes.  While the Agile Manifesto recently celebrated its 15^th birthday, it’s just been the last several years that I’ve seen agile gain substantial momentum in becoming the official method many companies shepherd their projects.  Or has it…?

The graphic above shows that the use of agile over waterfall has yielded desirable results in terms of reaching business goals of delivering on time, within budget and with complete functionality.  Yet I still hear IT project measurement people voice concerns about fully adopting agile for their projects.

My observations, from talking to many people trying to estimate agile projects, are from a purely estimating perspective.  Before agile emerged, many development shops were building their projects via waterfall, ERP, RUP and other methods, all of which provided respective value.  Upon agile’s debut, it was but a whisper, among a few, as the next leading development methodology.  Over the years skeptical minds were slowly changed and adoption increased.  Today more organizations are using agile to develop their projects, but I have found a surprising number of estimators telling me in a hushed tone – they aren’t really developing in agile, despite the proclamation from on high.  They are using more a combination of agile and other development methods, yielding an agile hybrid, sometimes sheepishly referred to as “wagile” or “agilefall”.

I recently came across this blog post by David Gordon and I believe it nicely summarizes the problem many C-level executives find with the #NoEstimates agile view. At the end of the day, they need realistic numbers in order for their organizations to make a profit and remain competitive in their markets. It's simply not enough to start development without some sort of upfront plan as to how much the overall project will cost. Gordon gives a nice analogy:

Now, picture a carpenter who has been asked to bid on framing a group of houses. He replies that he can’t tell how long it will take, because his crew hasn’t built that model before. But if the home builder gives them, say, $400,000, they will work diligently, and when the money runs out, they can decide together whether to put more money into the construction. You’re probably thinking that the builder won’t ever do business with that carpenter, and neither will anyone else in town. But that’s because the carpenter isn’t an employee—he has competition. This helps explain why certain software developers think #NoEstimates is a fine idea—they don’t perceive that they have competition.

Unfortunately for developers, this is not a sustainable way of operating - as many organizations continue to outsource more and more positions, developers will need to be able to justify their work as a business case. Gordon argues it's worthwhile for developers to learn software estimating best practices and to "...take some time away from learning the newest cool development tool to become viable as a contingent worker." I have to say that I agree.

We speak to a number of scrum masters, project managers, and CIOs each month. QSM does research on software development projects - all types, including agile and waterfall. We work with a huge database of completed software projects, updated with new industry data on a regular basis. We provide predictive analytics and we study cost, schedule, risk, quality, size, resource demand management, business intelligence, and vendor management.

Within the last couple of years, we have been hearing some agile managers say that there is no need to estimate agile projects. They say that they are managing in smaller increments or sprints and that they already know how much velocity can be achieved. They also say they are constantly “grooming the backlog” so there is no need for a formal estimation model to forecast the amount of work that needs to be done. They are managing at the sprint and task level so they feel like they have everything under control.

The bottom line is that agile projects still need to be estimated to see the big picture: the release estimate. It's essential to know the release cost and effort before the project plan is handed off to the scrum master and before committing to the customer. The negative feedback we hear regarding estimation is somewhat ironic because agile processes actually lend themselves very well to formal estimation since they promote the use of size measures like user stories, story points, and epics. Having good size information is a big part of estimating successfully.