Testing Redux
On the latest Build and Analyze podcast, Marco Arment disses automated and unit-testing. Responding to a listener question, he gives voice to all the myths about testing. That it is “overhead”. That is slows down development and can hinder competitiveness. That it is only worth it when the cost of failure is high.
That is horrible advice.
Before I lay out my disagreement, please take what I say with a huge grain of salt. Marco’s hacking credentials are impeccable. He was an early (the first?) programmer at Tumblr, which is serving up these very words to you, and has single-handedly built some of the finest mobile apps for people who appreciate pure reading (like me). He’s clearly doing something right.
But it is those very credentials that give his advice great weight, and which, in this instance, is dangerous. If indeed he built complex apps like Instapaper without a whiff of automated testing, that means he’s superhuman. Just because Bolt can run 100 meters in under 10 seconds doesn’t mean we can all do it.
If there’s one process-related lesson we should have all learned from the last thirty years of programming, it is this: automated testing pays off.
There’s hard data to back this up. Test-driven development reduces defects drastically, while not hurting development time.
The terminology is varied: unit-testing, functional testing, integration testing etc. The important thing is that it be automated, and to a lesser extent, quick. Something you can run quickly (preferably with a key combination) every time you make even a small change, to make sure you didn’t break anything. And write new tests to cover any new code you’re writing. What else are you going to do? Run through every functionality in your app manually to make sure everything is still OK? How long do you think that will take? Why? When you could run a script?
Having automated tests increases your velocity, because you can make changes with more confidence. It can become a competitive advantage if done right. And it is worthwhile even in areas where the cost of defects is not high. My personal experience is that for anything that’s larger than a small self-itch-scratching script, that is expected to grow, the small price of writing tests is always worth it. And I’ve almost always seen my code get cleaner and more modular under the pressure of making them testable.
Please code like you’re in the 21st century. Write automated tests.
(After writing this I found that Joel Spolsky made more or less the same point here.)