Technical Debt: Why We Work Harder Instead of Smarter

We’ve all heard the cliché, "Work smarter not harder."  In an article he wrote for ACM’s Communications magazine, Phil Armour wonders, “If we were smart enough to do that, wouldn’t we already be doing it?” Sometimes we simply don’t know how to work smarter.  But sometimes we don’t work smarter because we choose ... or are not allowed to do things right the first time. 

Abraham Lincoln said, “Give me six hours to chop down a tree and I will spend the first four sharpening the axe.”  Sharpening our axe requires several things:

• First we must continually improve our skills, our understanding of the application problem space, and our knowledge of the advances in tools available to do the job.  Nursing professionals are required to complete continuous coursework in order to maintain their licenses but for software professionals, this kind of ongoing professional education is usually optional.
  
  
• Second, we must allow sufficient time to think about the proposed system and plan the right strategy and architecture.  Many organizations shortcut or forgo adequate business analysis and requirements gathering, electing instead to beat the problem into submission by sheer force of numbers (this is also known as the Mongolian horde strategy). But good planning is worth the time and effort. Sun Tzu said in the Art of War, “If you know your enemy and know yourself, you will not be imperiled in a hundred battles”. 
  
  There’s a story about Andrew Carnegie, the steel magnate of the late 1800’s early 1900’s.  It seems one of his managers came to him complaining about another manager he had just seen sitting back in his chair with his feet on his desk apparently doing nothing.  Carnegie said to the complaining manager, “Do you remember that idea we implemented a few months ago that made us a million dollars?  That man was sitting in exactly that position when he came up with it.”
  
  
• Third, we must apply the right resources at the right times.  We must be willing to spend the time to understand the problem, build the system properly, and test it adequately.  It doesn’t matter what methodology you use; waterfall, RUP, Agile, …  Nor does it matter what we’re creating, be it software, hardware, or dinner for a houseful of guests.  If you’re going to build something, you have to plan it, execute it, and test it. 

The problem is, all of these things take time, as the cliché says. Since we are unwilling to take the time to do things right the first time, we are condemned to take the time to redo them.

Larry Putnam Sr. has a list of what’s necessary to execute a project successfully.  Right at the top of the list is, “Somebody has to think.”  Organizations who are habitually unwilling (or unable) to invest in good up front thinking almost invariably accumulate technical debt:

Taking short-cuts generally means that the next time the software is touched, it needs to be fixed before any further work can be done. So the second piece of work is even more expensive to do correctly than the original piece of work, and you can be sure that the deadlines are just as tight as they were the first time. Worse, developers generally prefer to play it safe - if someone has left them a dodgy-looking piece of code, they prefer to leave well enough alone. So, unless there are strong individuals present who are really dedicated to good engineering, the team takes another short-cut and works around the code affected by the previous short-cut1. The third change involves working around the first two short-cuts, and so on.

If one follows the trend to its logical conclusion, and in my experience many teams do, one finds that the code complexity grows at an increasing rate. After several changes to the software, it reaches the point where nothing can be changed without significant time and risk. Usually at some point, the team begins to realise that they need to fix the things they've broken. But by then it's too late because they are spending all their time just keeping a fragile system running and have no spare capacity to fix the code. They've painted themselves into a classic Catch 22 situation.  

Optimizing the present at the expense of the future - when chosen deliberately and accompanied by an explicit and honest accounting of deferred costs - may be good tactics, but it's rarely good strategy. Software organizations tend to be good at tactics. Tactics favor action and reward agility and creativity.

Strategy, on the other hand, is rarely rewarding in the short term. Everyone loves deferred costs but few of us enjoy deferring rewards. Working smarter is strategic, but it requires us to eat our vegetables first. Is it any wonder that so many software organizations go straight for the dessert table?

QSM consultants can help you work smarter, not harder. Learn more about our Project Health Check and Forecasting Service and High Performance Benchmark Service.