Quantifying software project risk and having a systematic way of accounting for it in software estimates helps firms determine how much contingency (or management reserve) is needed to protect against factors like scope growth, lower than anticipated productivity, or technical challenges that keep teams from executing project plans as originally intended.  When risk mitigation is an explicit part of the software estimate, we can set reasonable client and management expectations and negotiate practical project plans with a higher likelihood of success.

Dealing with Uncertainty

At the time most software estimates are performed, detailed design is incomplete and major decisions about how the system will be designed, coded, tested, and delivered have not yet been made.  Faced with imperfect information, estimators must supply educated guesses – hopefully based on sound requirements and performance data from completed projects - about the final delivered size, productivity, team size, and schedule.  Because the inputs to the estimate are uncertain, the final outcomes are also uncertain.

When we estimate that a project will most likely take 6 months and 8 full-time staff to execute, we really should say, “Based on our analysis and past performance data, we expect the project to take 6 months and use 8 people, but the schedule could vary by as much as 15% and the budget by up to 20%.” But all too often, clients and management expect “single point” estimates based more on optimism than careful risk analysis.

Football season is well underway, and as a loyal football fan I enjoy watching my team go through the highs and lows of wins and losses. Many of the weekly games come down to crucial plays where coaches must decide the plays to call.  Coaches now more than ever are analyzed by what plays they called or how they managed the game.  It’s interesting that these days you often hear of data driven decision making and what the analytics say.  More and more, technology is playing a big role in our lives, including sports.  Like so many industries, data-driven approaches are using models to determine “win probability”.  The data is now being used to determine if a team will go for it on 4th down, punt, kick the extra point, go for a 2-point conversion, and even when to take time outs!  This has changed how coaches manage the game. The analytics provide coaches the ability to make informed decisions and defend their actions based on data.  Coaches can now make critical decisions with confidence.  John Harbaugh (Baltimore Ravens Head Coach) is known to have an analyst in his ear during games feeding him probabilities. 

What if you could do the same for your software delivery?  QSM’s SLIM-Estimate tool leverages industry trends on similar projects along with your own historical data to provide valuable insight on potential cost and schedule outcomes. That’s like having your own personal analyst in your ear. 

Moving to the cloud is big business these days. With any type of digital transformation; managing the migration, integration, and development work can be a stressful experience with a lot of risk. Wouldn't it be great to have a crystal ball that allowed you to make these big cost, staff and schedule decisions early in the planning process? I'll propose the next best thing: a database of completed and validated projects spanning multiple industries and development methodologies that can help you predict the way your project will behave.

The QSM Industry Database can provide this type of knowledge. With over 40 years of research and development, it includes metrics from thousands of completed technology projects from various application domains and industries. By leveraging this type of database, you can access industry benchmarks that can help you navigate the uncertainty that comes with planning IT transformations, software engineering, and cloud migrations.

What should this cloud migration cost? How long should it take? How many people will you need? What level of quality and productivity needs to be achieved to have a successful delivery? What are the chances of finishing within a 6-month time frame versus a 10 month? The secret to finding these answers is in the historical data. The way to bring that data to life is to use good scope-based estimation methods. In a view from QSM's SLIM-Estimate tool below, you can see some examples of how you might measure the size of a particular cloud migration to get started with an estimate.

I like listening to audiobooks when I go for a morning run. This month it is a David Baldacci thriller about two CIA professional killers pitted against each other who end up working together to save us all from global catastrophe.  Apparently, there is a ton of planning involved in stealthily hunting a target, making the kill, and then getting away unseen.  That’s because there is a lot of risk.  Timing is critical, down to the split second, and the slightest mistake can end your life.  Discussing the highly complex plan to foil an assassination attempt with his partner, one agent says to the other, “There’s no risk in planning. The risk is in the execution.”

That got me thinking about software development and QSM’s SLIM-Suite estimating, tracking, and forecasting tools.  Do I agree with that statement?  Yes and no.  Let’s look at it both activities – project planning and execution.  

Planning

The activity of planning is not risky as far as your personal safety is concerned. You probably aren’t in danger of getting attacked or making a mistake that will cause bodily injury (you may experience emotional trauma or at least endure a minor headache).  It is most definitely risky for software development programs and initiatives, however, because aggressive plans based on poor estimates handicap the delivery team.  Without understanding the dynamics of software development projects or the ability to rapidly compute a range of potential outcomes to identify risky scenarios, planners may inadvertently commit to unrealistic schedule, budget, and staffing goals.  In fact, most plans are “goal based” ― task lists and staffing plans derived to give management or the customer what they want, because there is no solid framework or supporting data to defend against it. 

If you were unable to attend our recent webinar, "How to Identify Unrealistic Project Expectations and What to Do about Them," a replay is now available.

Many software projects fail simply because customers (internal and external) have unrealistic expectations about schedules and budgets. The desired outcomes do not align with known capabilities - based on industry data or your history. Decision makers are simply unaware, absent an estimation process based on scope and a way to assess the reasonableness of project goals. Presented by Laura Zuber, this PDU-approved webinar will demonstrate how to identify unrealistic expectations and generate estimates that set you up for success. Laura will show you best practices for developing viable estimates that balance risk and opportunity, enabling executives to commit to plans that meet the most important business goals.

Watch the replay!

Many software projects fail simply because customers (internal and external) have unrealistic expectations about schedules and budgets. The desired outcomes do not align with known capabilities - based on industry data or your history. Decision makers are simply unaware, absent an estimation process based on scope and a way to assess the reasonableness of project goals. Presented by Laura Zuber on Thursday, May 9 at 1:00 PM EDT, this PDU-approved webinar will demonstrate how to identify unrealistic expectations and generate estimates that set you up for success. Laura will show you best practices for developing viable estimates that balance risk and opportunity, enabling executives to commit to plans that meet the most important business goals.

Register now!

The application of contingency buffer, more commonly known as “padding” or “management reserve” is the final step in any project estimation process.  The most common practice is for the estimator to use an intuitive multiplier which is added to base estimate.  Unfortunately, everyone has a different multiplier which is shaped by their own personal bias about risk and it is hidden in their head.  This creates a fundamental problem with transparency and consistency within most organizations.

Fortunately, there's a better way.  One solution is to define and configure agreed upon standards that are matched to specific business risk situations.  These should be collaboratively agreed to by all the stake holders in the organization.  Then they can be codified into a configuration that can be selected at the time when contingencies are typically applied to an estimate.  This helps solve the consistency issue.

To attack the transparency issue, you can use a technique of overlays to visualize the contingency in comparison to the base estimate. 

I was recently reading an article by Moira Alexander titled “Why Planning Is the Most Critical Step in Project Management” and I was stuck by her observation that one of the primary reasons that projects fail is because they commit to unrealistic expectations.  In my 35 years of experience, I believe this is the number one reason projects fail.  Yet it is competence that few organizations or product owners ever get good at.

 Today there are good simulation tools that make it simple to establish realistic project boundaries.  The results can be used effectively to communicate and negotiate expectations with clients.  

For example, imagine that you are a product owner planning out your next release.  Your team of 10 people has been working on a 5-month release cadence.   A backlog refinement has shown that there are approximately 100 story points to be completed in this release.  The project plan is shown in the figure below.

However, there are some uncertainties and we need to deal with them in a realistic way.  Since the schedule and the team size are fixed, the only area that can give is the functionality.   Simulations are a great way to quantify uncertainty.  In our case, we are confident in our team’s productivity and labor cost, but we are somewhat more uncertain about the new capabilities in this release.   It is easy to adjust the uncertainty settings and run a business simulation.  The uncertainty slider bars are in the image below. 

The story of QSM and software application estimation begins during my time in the Army. I was assigned to Sandia Base, NM to research methods for protecting soldiers from the effects of nuclear explosions.  I had to do several calculations to determine the impact of an explosion (blast calculations) on soldiers using a slide rule, which was very tedious.  Sandia National Laboratory was next door to my office, and they had just gotten the biggest and best engineering computer available at the time.  They offered computer time for anyone needing it and even offered to teach me programming, so I decided to take a course in FORTRAN programming over my lunch hour so I could do my blast calculations quicker. These lessons aided me in completing my work at Sandia and followed me to my future assignment at the Pentagon. 

For my tour at the Pentagon in the 1970s, there was not a lot of need for my nuclear experience so I was assigned to the Army’s computer program. We had to defend our program budget to the Department of Defense (DoD) budget review authority (OSD). One system, SIDPERS, the Army enterprise personnel system, had been in development for five years and after having a peak staff of 110, we were projecting 93 people for the next five years. The analyst looking at the budget asked what should have been a simple question, “What are these people going to do?” I did not have a good answer, and later, going back to the project team, neither did they. Because of this we lost $10M in our budget.

The time has come, once again for QSM’s annual March Madness tournament.  As we enter our 6^th year of friendly office competition, I looked back at some of my previous strategies to help me figure out how I wanted to go about completing my bracket this year.  In doing this, I realized that many of these concepts can be applied towards IT project management.

Three years ago, I built my bracket around an emotional desire for my preferred team to win.  I paid very little attention to their previous performance that season, or any of the other teams for that matter.  Needless to say, I did not do as well as I had hoped that year.  Unfortunately, this strategy is applied fairly frequently in software estimation, with stakeholders committing their teams to unreasonable schedules and budgets for projects that are “too big to fail.”  Committing to a plan based off of the desired outcome does not produce a good estimate, and often results in cost overruns and schedule delays (or in my case, quite a bit of ridicule from the Commish).