Like most people who’ve played it, I love Tetris. I still remember playing it for the first time on a friend’s Nintendo Game Boy. You may already have the theme song stuck in your head. Not only is Tetris one of the best games of all time, it’s also an excellent analogy for technical debt. The point of this analogy is to reach a common understanding of technical debt and its impact.
I’ll also share a personal story of how my team and I reduced technical debt in some billing code and in doing so, fixed a $1M/year bug.
Within software companies, product/project managers (PMs) work with software developers to prioritize what code will be written and shipped to customers next. Finishing a Tetris row is like shipping a feature.
Shipping a complex feature requires more rows.
Often, the business needs (new features, new products) will lead to tradeoffs within the code (hacks, shortcuts) in order to ship on time. Or, changes in the product strategy will be incompatible with its previous design, requiring additional effort to either migrate customers or support both the “new” and “old” logic.
Scenarios like these create technical debt within the product code. A buried gap in Tetris represents technical debt.
All code has technical debt. That’s normal. You can keep playing Tetris with a few gaps.
Too much technical debt will prevent features and bug fixes from shipping in a reasonable amount of time.
This isn’t a problem that can be solved by adding more developers or, more dramatically, replacing your existing developers. It’s called technical debt because, at some point, it needs to be paid down.
Paying down technical debt keeps you competitive. It keeps you in the game.
Similar to running a business, Tetris gets harder the longer you play. Pieces move faster and it becomes harder to keep up.
Similar to running a business, you can never win Tetris. There is no true finish line. You only control how quickly you will lose.
Similar to running a business, allowing too many gaps to build up in Tetris will cause you to lose.
The Million Dollar Bug
Not long ago in my career, my team and I were tasked with updating billing/invoicing logic in our product code to support new pricing plans, a new payment processor, and an improved billing workflow. Some of the details were still being decided by the product team, so we used that time to do a deep dive into the existing code. This improved our understanding of it so that we could give accurate estimates for the upcoming changes.
The basic purpose of the code we studied was to go through every customer account, calculate their bill, and send it over to the invoicing API. It had clearly been written with care and good intentions — not so much messy as it was inflexible. It was a monolithic function. There were no tests. There were very few logs. There were barely any documentation. There was some unexplained randomization. It had been written over 5 years earlier by one of the co-founders. The only changes since then were from an early employee, who was no longer at the company.
Was it really a problem? Invoices were going out. The company was making money. There was no indication of an issue. All of this could have dissuaded us from a refactor, but we also knew that big changes were coming, this function wouldn’t scale to our needs, and we could move faster if this piece were simplified.
We refactored the function within a single sprint and added some much-needed logs. That’s when we discovered what we had actually fixed. Someone from our accounting team stopped by our desks to ask why the number of outbound invoices had unexpectedly increased. The old code had been silently timing out and some customers’ usage wasn’t being tallied for the invoice. That weird randomization? It hid any patterns that might have alerted us customers weren’t being billed. When we ran an estimate, the missing invoices totaled over $1M per year.
Paying It Down Doesn’t Always Pay Off
While the story is completely true, paying down technical debt does not always have such a dramatic effect. We got lucky.
I wish I could offer sage advice on when to pay down technical debt. Unfortunately, the answer is: it’s complicated, and it will always be a balancing act. You can have the cleanest, most well tested code in the world, but you may also have no paying customers. Conversely, your company could be running some really messy code that delights customers and makes money hand over fist.
The advice I can give is that product owners and developers should share an understanding of technical debt is and that it can’t be avoided forever. After all, like Tetris, you can never win at software.
