Organizations often come to us in the early stages of shopping for a software estimation tool and, oftentimes we find that they could be asking some additional questions. They often focus on the tool’s operating system, database structure, and architecture, when they could also be focusing on the quality of the data behind the tool. They also ask a lot of questions about inputting detailed information when really it would be in their best interest to focus on solid project-level information since detail-level inputs are often not available early in the planning lifecycle. Instead of focusing on the number of hours allotted to each individual person, it would be more beneficial to focus on how much work the overall team needs to finish.
In our 30+ years of experience in this industry, we've found that, no matter what tool an organization ultimately chooses, they need to be asking the right questions. Here is the criteria they should consider.
Tool Capability
As with any tool, it is important to match the tool with the job at hand. Using a screw driver to perform the task of a chisel will yield poor results. The same is true with trying to use a detailed planning tool in place of a software estimation tool. Make sure that you consider at what point in time formal estimates are required and how the resulting information is used to support negotiation and business decision making. Here are the main issues that should be taken into consideration when assessing an estimation tool.
|
A good product needs to be a mature product. Look for a solution that has a field use track record. What is the product operational lifetime in the marketplace? How many companies and users are supported? Are the customers satisfied with the result? How accurate are the resulting estimates? |
|
A good product needs to be able to be configured to your business and development environments. It needs to be able to speak your language.Can the product be configured to the way your company does business? Can the project lifecycle and milestones be customized? Can the Work Breakdown Structure be customized? Can the skills and defect categories be customized? Can sizing methods be customized to the engineering artifacts that are available at the time the estimate must be performed? |
|
The product should be able to accommodate historical data from your organization. Can the historical data be calibrated to reflect the application domains? Can the historical data be used to sanity check new estimates? Can the historical data be stored in templates for estimating specific application domains? |
|
The product should be flexible enough to accommodate new development methods and technologies. Does the product have configuration properties sufficient to model new ways of developing software? |
|
A good solution should be easily accessible to the users, be it standalone PC, networked or cloud based. |
|
A good product should be easy to use. Is the user interface well thought out? Does it allow efficient operation for the user? Will it significantly increase the efficiency of estimators? Are the outputs sufficient to support negotiation briefings and project planning? |
|
Does the tool allow solutions subject to project or organizational constraints (schedule, effort, staffing, and functionality)? |
|
A good tool should allow for multiple alternatives to be compared. Does the tool allow for the capture, storage and comparison of alternative estimation scenarios? Is it easy to select and reload alternatives? |
|
A good tool should be able to quantify the quality of the estimate relative to known uncertainties and risk. |
|
A good tool should provide the capability to monitor actual progress against the estimate. Can the tools provide comparisons of schedule, effort, staffing, functionality and defect actuals to baseline plans? |
|
A good tool should provide as robust a solution as the problem demands. Can the tool estimate single and multiple releases? Do multiple release estimates incorporate dependency relationships? Can global updates be done on multi release estimates? |
Supplier Capability
The supplier capability can be as important as the tool itself in a successful tool implementation. You are buying more than a tool, you are selecting a partner to help you implement and deploy the solution. These are important facts to consider about the supplier of your estimation tool.
|
How long has the supplier been in business? Do they have a good reputation? What does their customer profile look like? How stable is the employee base? |
|
How passionate are the people who support the product? How many years of domain specific experience do they have? What do their customers have to say about the competence of the supplier’s staff? |
|
Is the supplier an innovator or a copier? Do they have a track record of bringing new ideas and products to market? Are they passionate about finding new and better ways to do things? Are they open to new ideas? |
|
What is the track record of helping clients over hurdles? Do they provide technical as well as professional help desk support? Are they responsive to your time line? What do current customers have to say about the customer support? Do they provide initial support sessions and mentoring? |
|
To what extent is the supplier able to assist you in deployment, planning, and execution? Do they have teams that focus on deployment and change management? |
|
Is there product training available? How is it made available - public, online, or in-house? How frequently is it offered? How much does it cost? How effective is the training and what do existing customers say about the training experience? Are there professional certification programs available? |
|
Is additional consulting support available if required? What is the skill and track record of the services operation? |
|
Does the supplier have the capability to build or assist in the building of product integrations? Is there a published Application Programming Interface (API)? Are there published examples of API integrations? |
Underlying Estimation Methodology
A tool is only as good as the underlying methodology. It is important to be able to explain the basis of the method to the people involved in the estimation process to help with credibility and adoption. Here are some important issues you should consider.
|
Is there a solid theoretical basis and framework that the tool is constructed around? Is it intuitive and easy to explain? |
|
Does the tool provide a capability to enhance process? Is it flexible enough to adapt when processes change? |
|
Can the tool replay projects that have already been completed to a satisfactory level of accuracy? |
|
Does the tool provide transparency into the estimation process? Are the inputs easy to understand and explain? Do the outputs relate to management issues like time, effort, cost, staffing, and reliability? |
|
Is the method proven in your application domain and industry? |
|
Is the method flexible enough that it can be tailored to natural artifacts that result from normal engineering work on a project? |
|
Is the method a closed loop learning process? Does history feed estimates that are tracked to completion and is performance evaluated at the end of a project? |
|
Can the tool be used to support business negotiation at all levels of the organization? |
Price and Availability:
Price is always a major factor in purchasing transformative tools. It is important to evaluate the totality of the investment as well as the potential returns. You should be looking for the best value and you also want a great tool and a successful implementation. Here are a few pricing issues you should pay attention to.
|
Look for the best value. It is a combination of price, support, stability and peace of mind. |
|
How flexible can the implementation be? Can one start with a pilot and scale up? |
|
What is the scope of the likely deployment? Are there volume discounts for large purchases? Is there help desk and deployment support locally and globally? |
There's a lot to consider when selecting an estimation tool. It's a monetary investment as well as a process investment. Ultimately, you want to know if it will help you accomplish the goals you set out to accomplish. Take the time to find a solution that will work best for you. You will be glad that you did.