On our Software Productivity Declines Over Time post, ISBSG's Peter Hill left an interesting comment:

... your findings that 'average productivity increases with project size' might raise a few eyebrows, but the ISBSG data produces the same finding. Of course we only have data on projects that were completed. I wonder about the incidence of abandonment of large projects.

The QSM database shows a positive correlation between system size and productivity. The following chart shows PI vs. System Size regression fits for 9 different application domains (Business, System Software, Process Control, Telecom, Scientific, Command & Control, Avionics, Microcode, and Real Time). Regardless of domain, average productivity increases with system size:

View larger image

In the SLIM-Estimate user's guide, we speculate about possible explanations for the relationship between productivity and system size:

Small systems tend to have lower average PIs than large systems. We can only speculate as to the reasons, but two important influences may be management influence (the way large and small systems are planned, managed, and funded) and economies of scale.

Since we're speculating about why (in the aggregate at least) software productivity appears to have declined over time, here's another possible cause. I call it The Housework Theory of Productivity.

Back in great-Grandma's time, cleaning was a labor intensive endeavor. Rugs were swept once or twice a week and taken outside and beaten by hand once a year. Those cheery little scrubbing bubbles weren't around to whisk the pesky soap scum from our bathtubs - they hadn't been invented yet. When it came to cleaning, our aspirations were limited by the time and effort required to complete even the simplest tasks.

Over time various "labor saving" devices ushered in heretofore unheard of levels of springtime fresh cleanliness. But there was a downside to these advances: as our ability to get things clean increased, so did our expectations. The work expanded to fill the time (and energy) available to us.

If we were to compare two software projects that are same size and perform the same function (but were completed 20 years apart), we'd likely find the modern billing system smaller in size but far richer in features. It would also be more complex, both from an algorithmic and an architectural standpoint. Because today's programming tools and languages make it possible to deliver more value per line of code or function point, our standards have risen. We expect more for our time and money.

Peter Hill of ISBGS poses an interesting question:

Has software productivity improved over the last 15 years? What do you think? Perhaps it doesn't matter as long as quality (as in defect rate) as improved?

Two widely used productivity measures are Function Points/Person Month and QSM's PI (or productivity index). To answer Peter's question, I took a quick look at 1000 medium and high confidence business systems completed between 1996 and 2011. Here's what I found:

Whether we look at PI or FP/PM, the story's the same - on average, productivity has actually decreased over time.  What could be causing this? One possible explanation is the correlation between measured productivity and the volume of delivered functionality. As the next chart shows, regardless of the metric used average productivity increases with project size:

Which led me to wonder: what has happened to average project size over time? Again, regardless of whether the delivered functionality was measured in SLOC or Function Points, the story was the same: projects are getting smaller.

Peter's question is a good example of why we often need more than one metric to interpret the data. More on that topic coming up shortly!

It’s hard to believe it’s been 36 years since an IBM manager named Fred Brooks came out with his seminal insights about software development, the most famous of which ("adding more people to a late software project makes it later") came to be known as Brooks’ Law. These days, most software professionals accept and appreciate Brooks’ analysis, yet we continue to make the very mistakes that prompted him to write The Mythical Man Month!

Which leads to an interesting question: armed with such a clear and compelling argument against piling on staff at the last minute, why do we repeatedly employ a strategy that not only fails to achieve the hoped-for schedule reductions but often results in buggy, unreliable software?

The most likely answer combines schedule pressure with the human tendency to over-optimism. Basing plans on hope rather than experience is encouraged by a constant parade of new tools and methods. Faced with the pressure to win business, please customers and maintain market share, is it really surprising that new  technologies tempt us to discount the past and hope that – if we use this tool, this team, this methodology - this project will be different?

How can software developers counter the human tendency to fall for overly optimistic estimates and unachievable schedules?

What's needed is perspective: the kind of perspective that comes from honestly examining - and reminding ourselves - how things have worked in the past. In a paper called, “Technology Can Only Do So Much”, I look at the human and technological factors that trip up so many software projects.  Good historical data provides a sound empirical baseline, against which both conventional wisdom and future plans can be assessed.

What do you do if you want to improve a process and have 100 candidate predictor variables?  How do you decide where to direct causal analysis effort?  Similarly, what if you want to create an estimating model and you have so many factors you do not know where to start? Data mining techniques can be used to filter many variables down to a vital few to attack first or to build estimating models to predict important outcomes. CrossTalk Magazine, January 2011.

Read the full article here.

Paul Below has over 25 years’ experience in measurement technology, statistical analysis, estimating, forecasting, Lean Six Sigma, and data mining. He serves as a consultant for QSM, providing clients with statistical analysis of operational performance leading to process improvement and predictability.

Mr. Below is a Certified Software Quality Analyst, a past Certified Function Point Specialist, and a Six Sigma Black Belt. He has been a course developer and instructor for Estimating, Lean Six Sigma, Metrics Analysis, Function Point Analysis, as well as statistical analysis in the Masters of Software Engineering program at Seattle University. He has presented papers at numerous conferences. He has one US patent and two patents pending.

Recently I attended a seminar on a commercial reporting and data sharing product. In the sales material and discussion, the phrase “Single Version of the Truth” was used several times. But what does it mean?

“In computerized business management, svot, or Single Version of the Truth, is a technical concept describing the data warehousing ideal of having either a single centralised database, or at least a distributed synchronised database, which stores all of an organisation's data in a consistent and non‐redundant form.” - Wikipedia

The concept is attractive to decision makers who collect and analyze information from multiple departments or teams. Here's why:

“Since the dawn of MIS (Management Information Systems), the most important objective has been to create a single version of the truth. That is, a single set of reports and definitions for all business terms, to make sure every manager has the same understanding.”

Sounds simple, doesn’t it? Sales pitches for svot imply that if distributed data sources were linked into a single master repository, the problem of unambiguous, consistent reporting and analysis would be solved. Yet reports are often based on different data using different definitions, different collection processes, and different reporting criteria.

For many projects, duration is just as important a constraint as cost. In this installment we will tackle the question:  How do changes to team size affect project duration and the resulting productivity?  Once again we will use our database of business applications completed since January, 2000.

Continue reading...

On Thursday, September 30th at 1 pm EDT, QSM will host a Webinar on Agile Estimation Methods.

You can view the replay of this webinar here.

This week we turn to another question triggered by the Performance Benchmark Tables: how does duration affect productivity? To many managers, project schedule and cost are equally important. There are significant tradeoffs involved: if the project takes too long, important market opportunities may be lost. But adding people to compress the schedule can drive up cost dramatically. For this reason, QSM uses a productivity metric that explicitly accounts for duration: the Productivity Index (or PI). Unlike ratio based productivity measures, the PI is a three dimensional measure that adds duration to the traditional size/effort equation. It explicitly accounts for the distinctly non-linear relationships between size, effort, and time.  To see the benefits of this approach, let’s look at how project duration relates to simple (SLOC/effort) productivity.

Continue reading...

In Part I of this series, we demonstrated that average productivity (effective size/effort) increases with project size. This relationship holds true across the size spectrum whether we’re talking about projects in the very small range or projects that deliver a million lines of code. Above this cutoff, the sample size is too small to be definitive.

But productivity isn't the only metric that increases with project size. On average, large projects use more effort, take longer, and use bigger teams.  How can these results be reconciled with previous studies which conclude that the large team strategy results in lower productivity? It would seem that we have a contradiction on our hands.