A couple of years ago at a lean software and systems conference, I delivered a “lightning talk” about software metrics. In the two-minute time span, I illustrated the folly of gathering data without a measurement plan and the audience grasped the concept immediately. “Why don’t more companies get this?” remarked several attendees, “it just doesn’t make sense to collect all the data we do without a plan.”
It doesn’t take a rocket scientist to succeed with software measurement; professionals with a straightforward plan can quickly and easily reap its benefits. Two concepts are fundamental to embrace for metrics success: 1. Goal-Question-Metric (GQM), and 2. Simplicity.
Goal-Question-Metric (GQM) Approach to Metrics
First introduced by Victor Basili as an approach to measurement, and later the subject of a book by the same name by Rini vanSoligen and Egon Berghout, GQM is a straight-forward, stepwise approach to measurement. While it has applicability to measurement in any industry, Basili created GQM specifically to address the chaos in the software world. GQM involves three steps:
- Establish the Goals for measurement.
- Ask the Questions that will answer whether the goals are being met.
- Design and collect the Metrics to answer the questions.
The Software Engineering Institute (SEI) at Carnegie Mellon University in Pittsburgh, PA expanded Victor Basili’s GQM approach to GQIM, the “I” being indicator, but that is the topic of a future post.
GQM provides a simple solution to a concept that we often get backwards (see the analogy below) and make far too complex. Before collecting metrics prematurely and haphazardly (haphazardly meaning anything collect anything readily measurable), GQM sets out two planning steps. This is similar to doing project management (plans before execution) for metrics – it gives a measurement program direction and objectives. It is these two components: planning and asking the questions that are missed in metrics programs – and as a result, organizations end up drowning in useless data.
The analogy is this: If I am going to have a party for 20 people and I’m going to plan the food, it makes sense first to figure out the goals of the food (appetizers, dinner, snacks, or substance to prevent drunkenness) before doing a menu, writing a shopping list and buying the ingredients. This is akin to doing GQM before collecting data. For any successful event (aside from a Sunday afternoon pizza and chips party,) it would be counter-productive to simply go shopping first, buy a bunch of ingredients (usually on sale) and then trying to figure out how to put them into a good menu once we got them home. Too many measurement programs go this way – we start collecting readily available data (effort, phases, defects, shutdowns, etc.) and then when we have a few months (or years) worth, we try to assemble them into useful metrics. No wonder so many measurement programs fail. GQM can help!
Simplicity
The second fundamental component for metrics success is simplicity. I’ve seen far too many programs “invent” complex (and innovative!) ratios that no one, aside from the measurement team, understands. It is as important to define, communicate, and pilot (to ensure the understanding) metrics to be collected before actually collecting them. If participants do not understand the context of what you want them to collect, when to collect them (at the beginning, middle or end of a project, for example), and the logistics of measurement (i.e., what are the units of measure, what level of granularity, how much precision), you will end up with a mishmash of data. Do it right the first time and do it simply are important to remember with measurement.
Successful software measurement programs can provide uncover valuable information about best practices, the better tools, and what steps people are doing that yield higher productivity (efficiency) – when they are planned and simple. Follow this advice and your measurement program will become an investment that pays off instead of a costly measurement “exercise.”