How many times have you been involved with a software project or a portfolio of projects where the schedule or budget is doomed from the start? It happens all the time. One of the best ways to avoid this problem is to leverage historical data – actual performance of completed software projects. QSM has over 46 years experience in software estimation and control. We have seen thousands of projects and products delivered, both in-house and vendor driven. One of the biggest problems we help our clients solve is negotiating the right cost and schedule targets. Whether advising a client on an in-house project, a vendor on their proposal, or an end user with a bid evaluation decision, one thing becomes very clear − all sides are trying to negotiate a cost and timeline that they feel comfortable with. The problem is that they often negotiate with little to no data of past performance.
Negotiating Initial Schedule and Budget Commitments
You don't need data from hundreds of projects, and it doesn’t need to be granular. Software project or release level data is a great place to start. It establishes a baseline that you can count on because the delivery targets have been achieved in the past. It is tough to argue with cost and schedule numbers that have already been proven, and of course, having the data at your fingertips gives you a leg to stand on when negotiating.
The best practice we recommend is to capture a few core metrics:
- Duration
- Effort
- Reliability
- Productivity
- Scope
Once you have these key measures for a few projects then you can make reliable comparisons to what the vendor is proposing; or from the vendor point of view, they can see if their proposal is realistic before making a commitment. This way both sides can have achievable targets. QSM’s founder Larry Putnam Sr. co-authored a popular book with Ware Myers called Five Core Metrics, a very helpful resource in case you have a chance to check it out.
In addition to capturing your own historical data you can use industry data. Benchmarks from industry let both sides see if software bids are competitive and realistic from an independent point of view. The QSM Industry Database is one of the largest repositories of completed projects in the world, including over 46 years of research and development. The data includes typical cost, scope, schedule, reliability and productivity numbers on software and technology development projects and lets us see the relationships that affect project and portfolio delivery. QSM updates the database every 2-3 years with hundreds of new projects from clients and partners with their permission. The data is stratified across multiple application domains including Business, Avionics, Command & Control, Microcode, Process Control, Real Time, Scientific, Systems Software, Cloud Migration, and Telecom. The data contributors are from both the government and commercial sectors and the data spans across multiple industries including finance, insurance, banking, manufacturing, IT, telecom, engineering, aerospace, utilities, and defense.
The software estimation tool example below shows a vendor’s schedule estimate compared to industry trendlines for like projects and some of the vendor’s own historical data. The schedule aligns well with industry average performance and a handful of completed projects, so it is considered reasonable and defensible. An estimate outside of the statistical bounds would be considered risky. By showing this type of analysis you can see if the estimates are realistic and if they are competitive.
Negotiating Throughout the Software Project Lifecycle
Leveraging monthly actual performance data can help you with ongoing negotiations once projects are in-flight. QSM tools provide rigorous plan versus actual analysis to see if agreed upon cost and schedule targets are being achieved and allow you to set new targets if requirements change. By capturing monthly actuals including functionality delivered, defects found, and staffing numbers, you can generate new model-driven forecasts to see when the product will be delivered and if it will be delivered with high quality. Once these forecasts are generated, you can see how they compare to the initial estimates, the vendor or end-user client’s historical data, and how they compare to QSM industry trends. In the software project tracking example below you can see plan (blue) versus actual (red) comparisons, a forecast (green) estimate to complete, and the comparisons to historical data. You can quickly run multiple forecasts to evaluate and select the best strategy to achieve successful project outcomes.
Negotiating Software Programs and Portfolios
When working with multiple vendors or proposals there is often a need to see this type of estimation and tracking analysis at a portfolio level. Some of the challenges with software program and portfolio delivery include:
- Negotiating annual budget targets
- Determining whether you have the resource capacity to fulfill demand
- Dealing with complexities due to project dependencies
Managing these complexities typically center on finding answers to questions like:
- What if one release schedule slips? How will that slip affect the schedule, staffing, and cost of subsequent releases?
- What will happen if one vendor is over budget? How will that affect the cost of the overall program? Will budget constraints cause some initiatives to be postponed?
- What if you need to increase the staff on a project? How will that affect the total effort and timing of resources needed to achieve strategic goals?
- Are the release level estimates within an acceptable range of risk compared to the rest of the estimates in our portfolio and history?
In the program estimate example below you can see the overall schedule and project dependencies (Gantt chart), aggregate staffing by project, and a quadrant chart showing multiple release estimates that are consistent with history because they fall within the risk target zone.
A key historical core metric used in release and portfolio estimates and adaptive forecasting is productivity. QSM tools calculate an empirically based measure called the Productivity Index (PI), using actual software project duration, effort, and scope. It accounts for impacts due to environmental factors like experience level of the development team, complexity of the work, development methods, and overall process maturity. Historical PI values can be used as a baseline to determine if agreed upon delivery targets are consistent with past performance and if development teams are improving over time. In the example below you can see a project’s estimated PI and how it compares to past vendor productivity and QSM Industry trends.
So, there you have it! A small amount of complete software project data can go a long way. Historical data adds credibility to software and IT negotiations. In addition to helping you set reliable delivery targets, it allows both sides to build good faith, improve development processes, solidify relationships, and potentially save big money on cost and schedule overruns. Capturing historical data vastly improves software development negotiations. The data proves it!