Software Estimation Best Practices

Ditch the Madness: SLIM Your Brackets

Monday morning I received an email that read:

Hi All,
You can set your clocks to it: the birds flying north for spring, daylight savings time, and this email being sent on the Sunday before the tournament begins.  That's right, March Madness is upon us my friends, and we’ve officially made it through winter. 

The message continued with details about how to participate, but as you can see, it’s time for QSM’s annual March Madness tournament.  So how do I justify spending company time filling out brackets?  By blogging about how this is actually related to project management.  As I went through the exercise of predicting the course of this tournament, I realized that many of the thoughts I had also go through the minds of project managers.

Before I reveal my picks I want to give some background information.  I’m new to this whole March Madness tournament thing.  I’m not familiar with the teams.  I don’t know the players’ strengths and weaknesses.  I didn’t watch their games earlier in the season, so I don’t know their stats.  All I know is that my significant other went to Ohio State so I want them to win.

This is the position a lot of project managers find themselves in when they’re in charge of a new project.  Perhaps they aren’t familiar with the type of project being built.  Perhaps there isn’t good historical data available from past projects so you do not know what to expect from this one.  Perhaps you’re leading a team that you’re unfamiliar with and as a result do not know which tasks would be most appropriate for each individual.  And finally, you want the project to go well because you don’t want a project failure to tarnish your reputation.  These are all areas of uncertainty that have the potential to greatly impact the outcome of the project and they need to be accounted for.

So here is my bracket:
Taylor's Bracket

You’re probably wondering how I got to these predictions.  Well, a friend of mind told me that in the first round it’s unlikely that a 16th rank team will beat a 1st rank team, so I chose my first round picks based on that bit of information.  I knew that upsets happen randomly in this tournament, so I arbitrarily added in some upsets.  For the following rounds I based my picks on the outcome of a head to head duel between mascots.  This strategy made me favor Wisconsin (probably much more than I should have) because I became quite attached to Bucky Badger.  He seemed like someone I wouldn’t want to get in a fight with, especially since in this image he’s carrying an axe.

Bucky Badger

However, I ultimately chose Ohio State to be the winner because I’m more personally invested in their success.  At this point I started to over justify the ability of Ohio State’s mascot, Brutus Buckeye, by reminding myself that he does a cumulative number of pushups for every point scored during a game, or that he can spell out O-H-I-O with his legs while doing a headstand, or that buckeyes are poisonous and no other mascot could stand a chance if they ingested one.  In short, I let my emotional investment in Ohio State’s success determine my prediction of the tournament.  

Brutus Buckeye

After making my selections I thought it might be good to compare my bracket to someone’s who knows a little more about basketball, so I enlisted the help of my colleague, Brad.

Here is what he came up with:

Brad's Bracket

When Taylor asked if I was joining the QSM office poll, I figured I would spend less than a minute filling out a bracket and join in on the next few weekends of upsets and buzzer beaters.  She then mentioned it might be used in a blog post highlighting decision-making strategies.  Suddenly I realized additional thought might have to go into this. 

Unfortunately, I didn’t watch and analyze most of the teams in the tournament this season so spending my lunch break doing some homework looked to be in my future.  I needed a strategy as well.  In looking at how the pool is setup, individuals get points for each win in each round with points weighted higher in the later rounds.  My approach would be to get as many teams correct in the later rounds and work backwards, trying not to leave any points on the table during opening weekend.  Easy enough, right?

In order to do this, I wanted to take a look at some evidence that would help me better understand the likelihood of certain teams reaching the Sweet 16 or Final Four.  If I could have a good amount of teams still playing in those rounds, any picks gone horribly wrong in the early rounds wouldn’t mean as much since I’d get more points for wins later on.  Rankings such as strength of schedule, wins vs. top 25 teams, hot/cold streaking teams and the injury report are some of the things I’ll consider when penciling in my selections.  For a deeper understanding, I might leverage ratings from expert statisticians like Sagarian and KenPom who use popular formulas to determine the best of the best.   With all that, I’m ready to pick my final four and work backwards from there.  This year, I like Louisville, Ohio State, Florida and Indiana in the Final Four.

While working backwards I’ll need to keep a few things in mind, I’ll need a 15-2 upset and a handful of others as well.  

Although Brad’s Final Four predictions were the same as mine, his methods for getting there were quite different.  He based his predictions on the teams’ historical performance, rather than his favorite team.  This strategy is similar to the History Tuning feature in SLIM-Estimate.  After loading historical projects from your project database, you can calibrate your project metrics to those measured in your historical data.

Historical Tuning Factors Dialog

Unfortunately, many project managers go about the decision-making process the way I did.  They let their preference for a particular outcome dictate their estimate of how the project will run.  For instance, a project manager might ignore the fact that historically it takes the development team 12.3 months to produce a 275 Object project, if Christmas is 11 months away and (s)he wants to enjoy the holiday without thinking about the project.  Shortening the schedule by 1.3 months would take the project to the end of Phase 3, which would make it appropriate to ship.  However, the reliability of the project would be much higher in January, where Phase 4 was scheduled to end.  Therefore, it might be wise for the project manager to carry on with the schedule as planned, and not let vacation schedules determine the course of the project.

Staffing & Probability Analysis View

To conclude, decisions based on personal preference are more likely to fail than those based on historical data.  That’s not to say that there won’t be some Cinderella stories, but they’re much more likely to happen in basketball than in project management.  If gambling is something you enjoy, just make sure to weigh the risks before placing your bets.  I’d much rather lose a $10 March Madness buy-in fee than hundreds of thousands of dollars in cost overruns for a late project.

 


I'd like to give a special thanks to Brad Verazin for co-authoring this blog post and for agreeing to share his March Madness bracket for the purpose of process improvement.

Blog Post Categories 
SLIM-Estimate Project Management