Ethan Avery's blog

Managing vendors has become increasingly important in recent years.  In my account management role at QSM, I see both sides of the vendor management relationship.  The client wants a proven vendor that will partner with them in achieving their IT goals; and the vendor wants to win that business, employ their workers, and hopefully earn more work.  Unfortunately, that state of client/vendor Zen is not often achieved, usually due to legitimate (and sometimes not) misunderstandings on both sides.

On the client side, they are concerned with selecting a vendor with whom they are confident their tasks and deliverables will be achieved on time, within budget, and of the best possible quality.  After a round of RFI’s, then RFQ’s, then a final down select process, the vendor is chosen and work begins.  Often, at least in my experience, the overriding decision criteria comes down to cost, which makes sense, to a degree.  But in many cases, cheapest, I mean, least expensive bids often rule the day.  This kind of decision-making comes with its own set of risks; the most obvious is you get what you pay for and it’s often an ill-prepared vendor.

“No, we don’t have any historical project data collected” is the statement I hear with some frequency when speaking to organizations about their IT project estimating processes.  Ideally we use client history to calibrate and tune the project estimates we provide.  In my quest to spread the word about parametric estimating I often encounter this notion that organizations don’t believe they have historical data in a retrievable form.  In almost every case that I have been involved, it turned out that the historical data was present, just not in the form of a 1,000 rowed spreadsheet.  Often times the data is more available than the client is aware.

Our approach works at a macro level so we are seeking overall project metrics of cost, schedule, size, staffing and defects.  If the actual formal documentation of history is not available for these five core metrics, then it usually is available by leveraging various sources within the organization.  We have found it’s common to resurrect a project’s outcome by seeking feedback from the team that worked the project, however if that’s not possible due to attrition, re-org or other disrupting factors, we can usually find the project metrics through other means.  Those other means may be time and defect tracking tools, requirements analysis tools and accounting systems.  The data is almost always documented somewhere.   

Recently a friend of mine sent me a link to a YouTube video featuring Jeff Sutherland and Ken Schwaber, the founders of Scrum, discussing the latest update to the 2016 Scrum Guide.  The updates to the guide were comprised of survey feedback from the Scrum community, with the goal to understand what is important to them that should be included in the updated Scrum guide.  

The most compelling part of this discussion for anyone contemplating whether estimating belongs in Scrum happens at the 32:10 point.  Jeff and Ken are posed the question, “what is the difference between a forecast and an estimate?”   They answer with the idea that many people confuse the word estimate with commitment, which plagues any development effort regardless of method.  Subsequently Jeff and Ken changed the word “estimate” to “forecast” in the latest Scrum Guide to reflect the ever-changing growth and reduction in project scope, i.e., the estimate will show our best commitment and what we think is going to happen.  That can change once the project starts, BUT, we can measure that too through forecasting, essentially re-estimating throughout the project.  

Recently the correlation between seeking the best gasoline prices and software project overspending collided for me.  On one hand, IT projects will easily outspend their budgets, and on the other hand, in our private lives, we are doing all we can to save pennies in our personal budgets.

From time to time I have found myself pondering whether to drive the extra 5 miles out of my way in order to beat the system and enjoy the best gasoline price in my area.  I don’t believe I am alone in this quest as evidenced by the websites that publish the lowest gas prices within a certain radius.  Typically the competing stations have very small differences between their prices.  If I play my cards right, I may be able to save a total of $.50 - $.75 for a fill up.  In almost all other walks of life $.75 is not even worth a second thought.

Often I speak with IT project measurement folks about the permeation of agile into their project estimation processes.  While the Agile Manifesto recently celebrated its 15^th birthday, it’s just been the last several years that I’ve seen agile gain substantial momentum in becoming the official method many companies shepherd their projects.  Or has it…?

The graphic above shows that the use of agile over waterfall has yielded desirable results in terms of reaching business goals of delivering on time, within budget and with complete functionality.  Yet I still hear IT project measurement people voice concerns about fully adopting agile for their projects.

My observations, from talking to many people trying to estimate agile projects, are from a purely estimating perspective.  Before agile emerged, many development shops were building their projects via waterfall, ERP, RUP and other methods, all of which provided respective value.  Upon agile’s debut, it was but a whisper, among a few, as the next leading development methodology.  Over the years skeptical minds were slowly changed and adoption increased.  Today more organizations are using agile to develop their projects, but I have found a surprising number of estimators telling me in a hushed tone – they aren’t really developing in agile, despite the proclamation from on high.  They are using more a combination of agile and other development methods, yielding an agile hybrid, sometimes sheepishly referred to as “wagile” or “agilefall”.