Part II: Estimate the Estimate

In Part 1 of How Much Estimation, we observed that both too much time and effort and too little time and effort spent on estimating are less than optimal.  Combining:

• The cost of producing an estimate—which is a function of the number of people working on the estimate and how long they work
• The cost of variance in the results of the estimate—that is, how much the estimate varies from experienced actuals and what that variance will likely cost the project.  This is typically a function of the number of unknowns at the time of estimating for which the project cannot easily adjust and which will require additional unplanned resources of time, effort, and staff.

We get a U-shaped curve, at the bottom of which is the optimal time: we’ve spent enough time and effort to minimize the sum of the cost of estimate and the cost of variance.

The question is: how to calculate this point?  It will not be the same for a very large complex project and a very small simple project.  Also we don’t want a complicated and time-consuming approach to calculate the cost of estimate—it should be quick and simple.

NASA’s Deep Space Network (DSN) projectⁱ developed a mechanism for this calculation based on two simple parameters:

Target Cost of Project

This is goal cost of the project as first envisaged in the project concept.  It is NOT the estimated cost of the project (which hasn’t been calculated yet).  Projects for which we expect and plan to spend a lot of money should clearly have more time and effort spent in estimating simply because more is at risk.

The Business Practice Being Supported

Very early in a project, it is normal that the data available is sparse, ambiguous, and of low quality.  Also at this point, the business practice being supported is a concept one: should we consider this project?  If we were to invest in it, would be get a reasonable return?  Can we likely build the system within the timeframe available?  Investing a lot of time and effort in producing an estimate at this stage is usually not very helpful.  The estimate produced will be (should be) only used to ok the project for continued analysis or to reject it altogether.  In the latter case it is clear that developing a highly detailed estimate for a project that won’t go anyway is not a good use of resources.

Later in a lifecycle, assuming the project has been given the go ahead, we have better data and we are at the point of committing significant resources to move the project ahead.  Therefore it is worth spending more time and money to produce an estimate.  The business practice being supported at this point is financial budgeting and resource allocation.

Later still, when the resources have been allocated and the project is ready to launch, we need more detailed estimates still.  These estimates will provide the bounding box in which the project plan will sit.  The business practice being supported here is project planning.

The Formula

The formula devised by NASA is a simple one:

Cost_of_Estimate = Practice_Parameter * Target_Cost^0.35

Where the values of Practice_Parameter are related to the estimation phase by:

And the value of 0.35 is fixed (for NASA).

Using the Formula

Since the formula only uses two simple parameters, it is straightforward to apply.  A project with a Target Cost of (say) $10m should expect to spend:

Simply by dividing by a personnel cost we can arrive at a simple effort value and using a personnel loading factor we could deduce an approximate schedule to produce the estimate.  So using the numbers above, assuming the average personnel cost per hour is $100 and three people are allocated half time of eight hour days to producing these estimates, our $10m project estimates should take:

Caveats!

I have made some assumptions in these calculations that we should almost never make in producing a project estimate (don’t forget, we are not estimating the project, we are estimating the estimate).  Can you guess what they are?

There are also other factors at play that I have found need to be accounted for in actually implementing this practice at clients.  I will discuss these in Part 3 of this series.

ⁱ  D. S. Remer and H. R. Buchanan, ”A Model for the Cost of Doing a Cost Estimate," The Telecommunications and Data Acquisition Progress Report 42-110, April{June 1992, Jet Propulsion Laboratory, Pasadena, California, pp. 278-283, August 15, 1992.