Quantifying software project risk and having a systematic way of accounting for it in software estimates helps firms determine how much contingency (or management reserve) is needed to protect against factors like scope growth, lower than anticipated productivity, or technical challenges that keep teams from executing project plans as originally intended.  When risk mitigation is an explicit part of the software estimate, we can set reasonable client and management expectations and negotiate practical project plans with a higher likelihood of success.

Dealing with Uncertainty

At the time most software estimates are performed, detailed design is incomplete and major decisions about how the system will be designed, coded, tested, and delivered have not yet been made.  Faced with imperfect information, estimators must supply educated guesses – hopefully based on sound requirements and performance data from completed projects - about the final delivered size, productivity, team size, and schedule.  Because the inputs to the estimate are uncertain, the final outcomes are also uncertain.

When we estimate that a project will most likely take 6 months and 8 full-time staff to execute, we really should say, “Based on our analysis and past performance data, we expect the project to take 6 months and use 8 people, but the schedule could vary by as much as 15% and the budget by up to 20%.” But all too often, clients and management expect “single point” estimates based more on optimism than careful risk analysis.

What you measure matters! Your data analysis is only as good as the data you are collecting. As a project manager and process improvement consultant, I was so excited when I first learned about QSM's project tracking software, SLIM-Control^®, because product size, quality, and custom metrics are the best metrics to show the true project status.

Mind you, this discovery took place a long time ago when project tracking was primarily limited to data on staff levels, cost, and duration. These measures provided indirect evidence that work was being done, but gave little insight into exactly what work was actually being done and when it would be completed. I remember a large project team at my company that got into hot water because they overran a fixed-priced project by $1 Million!  My process improvement team member said, "You don't overrun by $1 Million overnight!"  There were probably several issues that contributed to this colossal failure, but certainly one was the lack of visibility into the project status or health on a regular basis to help managers and teams identify potential problems and fix them before they became unsurmountable.

I like listening to audiobooks when I go for a morning run. This month it is a David Baldacci thriller about two CIA professional killers pitted against each other who end up working together to save us all from global catastrophe.  Apparently, there is a ton of planning involved in stealthily hunting a target, making the kill, and then getting away unseen.  That’s because there is a lot of risk.  Timing is critical, down to the split second, and the slightest mistake can end your life.  Discussing the highly complex plan to foil an assassination attempt with his partner, one agent says to the other, “There’s no risk in planning. The risk is in the execution.”

That got me thinking about software development and QSM’s SLIM-Suite estimating, tracking, and forecasting tools.  Do I agree with that statement?  Yes and no.  Let’s look at it both activities – project planning and execution.  

Planning

The activity of planning is not risky as far as your personal safety is concerned. You probably aren’t in danger of getting attacked or making a mistake that will cause bodily injury (you may experience emotional trauma or at least endure a minor headache).  It is most definitely risky for software development programs and initiatives, however, because aggressive plans based on poor estimates handicap the delivery team.  Without understanding the dynamics of software development projects or the ability to rapidly compute a range of potential outcomes to identify risky scenarios, planners may inadvertently commit to unrealistic schedule, budget, and staffing goals.  In fact, most plans are “goal based” ― task lists and staffing plans derived to give management or the customer what they want, because there is no solid framework or supporting data to defend against it. 

We live in a multi-dimensional world.  We see in three dimensions.  Our brains’ ability to process spatial relations of height, width, and depth are essential for navigating everywhere we go and every task we perform.  The scientific world has used computers and data to generate 3D graphics for years, because it is the only way to understand complex phenomenon and ideas.  I remember the first 3D color seismic map of rock formations underground I saw.  The image told a story the numbers alone could not.  The business world has caught on in recent years.  The wave began with data warehousing and business intelligence and has grown to include data analytics, artificial intelligence, and machine learning.

QSM has brought a scientific approach to software development project estimation and management for over 40 years.  Our SLIM-Suite of tools contain statistics on the largest repository of completed software projects that enable you to model software projects to predict potential outcomes and assess the reasonableness of your estimates and project goals.  They have always provided four common types of Data Analytics¹ – Descriptive, Diagnostic, Predictive, and Prescriptive.  SLIM tools now provide more in-depth data analytics using quadrant charts.

Beginning with the release of SLIM-Suite v8.1, two new SLIM-Estimate solution methods were added to let you see what a “typical” project would look like: that is, the resources it would require, based upon historical projects from either the QSM database or your own.  The two methods are:

• Solve from Trends Wizard
• Trend Based Solution

SLIM-Estimate provides several different ways to solve the software production equation and produce an estimate.  The solution method you select depends upon the information you have available.  The traditional methods, known as Quick Estimate Wizard and Detailed Method, take inputs of Size and Productivity (PI), and calculate Effort and Time.  The Solve for Size Wizard, perfect for time-boxed estimates, takes inputs of Time, Effort, and PI, and determines the amount of functionality that combination of resources can produce.

The trend solutions require only one input – Size.  Using the specified Primary Trend Group, Time and Effort are read from the average trend line and productivity (PI) is calculated.  These methods extend the capabilities of producing a defensible project estimates very early in the life cycle.  You can determine the feasibility of project goals, assess risk, and manage stakeholder expectations even if all you can estimate is a relative application size, expressed as a “T-shirt” or bin size, as shown in the figure below.

I am excited about the resource demand management capabilities in our newest SLIM-Estimate release (8.2). Software project estimates can now provide a breakout of Full Time Equivalent (FTE) staffing requirements by role by month. The staffing profile below shows how different roles, or skills, are required at different points in the schedule, based upon a particular development methodology. You can see that 6 FTE Programmers are needed by the month of May. Producing a high-level, scope-based estimate early in the software development lifecycle with detailed resource demand data helps the PMO and portfolio managers determine the best timing for project resource allocation, and setting project start dates that maximize productivity and reduce bottlenecks.

Once the estimate and resulting project skills allocation plan has been approved, resource demand management has not ended. Tracking actual staffing at the skill and task level for in-flight projects not only ensures the right people are working on the right things, meaning that product development is on track, it also allows timely resource plan adjustments to address unforeseen staffing needs.

Software process improvement efforts often fail because we try to accomplish too much too soon.  Aside from the cultural and organizational obstacles to change, people need time to learn and assimilate new ideas and skills.  “Human memory and comprehension are limited, and it is easy to design processes that are beyond peoples’ capacities,” says Watts Humphrey  (Humphrey, 1989).  This is true in any situation, but I think it is compounded in the software world because time is always a scarce resource.  The pressure is high in every organization to justify process improvement dollars and increase capabilities.

Establishing and maintaining software best practices requires that you design clear processes and plan a pace of implementation that promotes lasting change.  A key component is accommodating human learning and skill development challenges.  Borrowing from a training class I developed 14 years ago (yes, implementing best practices is still a challenge), let’s follow a team of water bugs as they progress through Watts’ four stages of Human Methods Adoption to understand what good pacing requires.

Installation – Initial installation of the methods and training in their use.  Process documentation and training should answer questions like: