Dear Carol: 

Given all your international experience, I’m hoping you can tell me where I can find a large, freely available industry database that project managers could use for software estimation and/or benchmarking.  After 5 decades of software development wouldn’t you think that we could put together a software estimation or benchmarking database that the world could use for free? 

- Hopeful in Hartford

Dear Hopeful:  

Great question – and the dream of many IT project managers.  It might seem like an easy concept (just collect actual effort and project size and use it for future estimates); in practice it’s not that simple.

What I know is that in software estimation and benchmarking, there is no free lunch -- you get what you pay for.  And I’ll explain why…

Management decisions made before a software project is underway are a significant factor in determining whether it succeeds or fails. In a recent article for Projects at Work, QSM's Don Beckett identifies seven principles, based on comprehensive studies, that leaders must support and uphold to help create an environment in which projects can succeed. Ignoring them practically guarantees failure.

Don Beckett has 18 years of experience in software project estimation, measurement, and analysis. His responsibilities at QSM include research, consulting, and customer support. Don was an analyst/co-author of the 2006 QSM Software Almanac and has contributed articles to Crosstalk and Software Tech News.

Read the full article!

In a recent interview on WUSA9's Government Contracting Weekly, a show devoted to winning government contracts, QSM Co-CEO Doug Putnam discussed data management with the show's host, Jim McCarthy. QSM was founded by Larry Putnam, Sr. in 1978 because he saw the need to supply quantitative evidence when justifying budgets for large government software projects. Doug explains the importance of leveraging quantitative measurement in today's dynamic goverment IT environment, from managing team size to improving quality. He also identifies the core metrics to track when starting your own metrics program. Watch the full episode above!

QSM's recent webinar, How a Collaborative Estimation Process Produces Realistic Estimates Fast, presented by Laura Zuber, sparked a lively Q&A session from our audience. Here are the highlights:

Q: How do I know which template to use when creating a new estimate in SLIM-WebServices?

A: That’s a good question. Your back office administrator configures templates to map to your software development lifecycle, your standard processes and different project types. The back office person can provide detailed descriptions and key information that are viewable from the Feasibility Wizard so you know which one to select.

Q: What makes a project assessed as conservative or risky?

So, just why do we want to measure software productivity (without using the root word “productive” in the answer)?  I believe that it comes down to the desire to numerically evaluate an inherently complex process so that quantitative comparisons can be made to provide a basis for decision making:

• Is output per unit of labor or cost increasing or decreasing?
• Benchmarking against “the industry” or “the competition”
• Identify practices that either promote or impede increased output and better quality

I’m sure there are many others that could be added to the list.

Issues

Traditionally, software productivity has been measured as a ratio between units of output and units of effort.  Simple productivity measures worked fairly well for well defined, repetitive manufacturing processes where a 10% increase in input reliably translates to a comparable increase in output, but there are massive problems with applying simple productivity measures to complex, non-repetitive design processes like software development.

Dear Carol: 

I’m a developer in our IT department and we know that project estimating is a big deal for our customers.  Somehow, no matter what we do, we can't seem to get it right.  We do know that project size is an important input to good estimating  and our gut feel is that if we get sizing right, we’ll do better estimates!  I know you recommend using function points, but I’ve also been reading a lot about use case points, story points, SLOC, sizing by analogy, T-shirt sizing, COSMIC and other sizing metrics.  We do a mix of waterfall, agile, iterative and even Kanban to do our projects so what’s the best choice for sizing to get the best results? 

- Size Challenged in Milwaukee

Dear Size Challenged:    

Sometimes I wonder if the internet and the proliferation of (mis)information is a good thing. Before the internet, our choices (for sizing or estimating or anything) were limited and we didn’t have such an overwhelming task to first sift through many options before taking action.  Your list of software sizing choices is an example of this. 

I rather enjoyed the Google Analytics April Fools prank earlier this month, Welcome to Data-Less Decision Making on Analytics Academy.  Though satirical, this video brings to light an important reason why individuals have such trouble making decisions in a business environment: they don’t have data.

I’ll agree that without data it’s really appealing to turn to the coin flip method and be done with it.  After all, 50/50 odds really aren’t terrible, right?  But project management software such as SLIM-Estimate make empirically-based business decisions possible, even when company data isn’t immediately available.

Leveraging our database that contains over 10,000 projects, QSM has developed and regularly updates 17 distinct industry trends.  When creating an estimate or benchmarking a past performance, simply select the QSM industry trend that most closely reflects the type of system being built.  This will serve as a reference point.

If historical data is available but you’re unsure of which metrics to collect, SLIM-SmartSheets is a new downloadable feature in SLIM version 8.2 that mimics the look and feel of SLIM-DataManager and allows users to collect project data, even when they’re not on a network computer.  Each project can then be pulled into one SLIM-DataManager file using the API.  

In a recent, highly-discussed article for Dr. Dobb's, QSM's Carol Dekkers asks a tough question: why are we so woefully poor at estimating software projects? It's a tough pill to swallow considering software developers are among the smartest people on the planet, often boasting advanced degrees in mathematics, engineering, or computer science. Yet study upon study cites that less than one-third of projects are delivered on time or on budget. The problem of software project estimation is not straightforward. To get the heart of the issue, Carol Dekkers takes us through the five top misperceptions about software estimating, and what we can do to address them.

Read the full article on Dr. Dobb's!