Two common questions we receive are “how quickly can we get SLIM deployed within our organization?” and “how accurate will our estimates be?” On the accuracy question, our answer is usually that “it depends”. Like most things, accuracy and success in estimating are directly correlated to the effort put into the task. If you buy a commercial estimating tool and try to use it “out of the box” with no tailoring and calibration, accuracy usually suffers. One of the most efficient ways to achieve both of these goals is to use your own historical data. This was an excellent example of how one of our global systems integration clients was able to get almost immediate value by calibrating SLIM-Estimate to their historical data.
This major integrator wanted to improve estimation at one of their accounts, a large multinational company. In addition they wanted to use function points as their sizing metric. After evaluating a number of estimation options, including custom built and other internally developed solutions, they decided to use QSM’s SLIM-Estimate, which was being deployed globally across the rest of the organization. They decided to start with a pilot implementation. The estimation pilot lasted a total of 5 months with a staff of 2 people. After the fact they looked at the effort expended on their pilot activities and found that 60% of the total effort went into their calibration activities. The calibration process included assembling a historical data sample of over 50 projects. As part of their function point deployment, they empirically determined FP/ESLOC gearing factors.
After completing this calibration effort, they performed 35 estimates with the SLIM-Estimate tool. They compared their estimation results to the actual project performance. The results confirmed the benefit of tailoring the tool to their own historical data. Of the 35 estimates completed with the calibrated tool, 29 (83%) of the estimates completed were within +/-10% of the actual, 32 (91%) were within +/-20%, and only 3 were > +/- 20%.
An interesting observation that they made was that the accuracy of their estimates increased with the size of the projects estimated. 70-85% of projects <30 FP were within +/-10% of the actual, while 100% of projects >30 FP were within +/-10% of the actual. At QSM, we have seen similar trends, as small projects often have more variability.
This group had such a high level of success with their estimates, because they concentrated their effort on calibrating the tool with realistic historic data and creating a good sizing model, which included empirically determining their gearing factors. We find that clients “get out of what they put into our tools.” At QSM, we’re happy to work with you to tailor your estimates to your environment through ramp-up sessions, customer support, and consulting.