Engineering is all about failure
The debate of how the practice of programming differs from what a computer science curriculum teaches is an old one. Which topics are neglected? Which are not? But in the larger scheme of things, a solid foundation in mathematics, algorithms and data structures, coupled with an analytical and inquisitive mind are all one needs to come up to speed with unfamiliar topics that a job might require.
The missing ingredient is an an attitude of dealing with failure.
To be a successful software engineer (or indeed, any engineer), one first needs to be utterly and completely broken by failure. One must be so humiliated by a complex system that they give up and realize that the only chance of moving forward comes from being a supplicant to the complexity, by approaching it with humility and caution, not with hubris. You have to listen to the system, coax it into behaving. Commanding it does not work.
In the real world, progress follows a path akin to a blindfolded person bouncing off the walls of a canyon. The path zigs and zags. You often hit hard walls, but inch forward.
This is why it’s important to take “hard” courses. They’re hard not just because the projects and subject matter are hard, but because they make you comfortable with failure. The difference between the “hard” sciences and the “soft” arts is that in the former there is a clear, objective line between success and failure. Either the system works or it doesn’t. Unlike the soft arts, failure is stark and undismissable. It is not just another interpretation or point of view. There is certainly no “no wrong way.”
Let failure teach you. Don’t be afraid of it. Only then do you realize the crucial truth about engineering. Engineering is all about failure. The daily life of an engineer is one of constant failure, peppered with a few rare successes. In fact, you can tell a seasoned engineer by how quickly they get over their successes, because they know they’ll be engulfed by failure soon enough.
Failure is not just pervasive in the daily practice of an engineer. The consideration of failure also permeates the design and implementation of the artifacts they build. You know the difference between your neat CS assignment and the million LOC behemoths that the world actually runs on? The latter must deal with failures both seen and not yet imagined, and still try to do their best. The first question any artifact – whether it’s just a design or a real running system – must be subjected to is: “under what conditions will it break?”
A corollary of this failure-paranoia is that solid engineering goes hand in hand with solid testing. Real engineers run a constant loop of build-test-build-test. Even then, there are so many variables that nothing short of a real test in a real environment will convince an engineer, however begrudgingly, that the artifact he has engineered has some chance of working.
I’ve painted a depressing picture of the professional life of a software engineer. Don’t they go crazy dealing with all this failure? Why would anyone want to subject themselves to this on a daily basis?
First of all, there is the pleasure of having created something. True engineers are loyal first to the thing they’re building. The artifact produced is an objective and independent statement of success, and is recognized as such by the community of engineers. “Your crew built a system that handles 10k qps @ 20ms latency? Respect.” But more than that, creating it taught you something. It didn’t kill you, it made you stronger. You now have a brighter flashlight with which to approach the complexity of your next task.
There is also, paradoxically, a sense of calm that comes from attacking problems with objective measures. Even behind the most maddening this-is-impossible bug is clear, unassailable logic. It’s just that you can’t see it. But the bug is trying to teach it to you. The artifacts an engineer deals with, no matter how frustrating they sometimes are, still possess a clean rationality. In this sense, engineering is a refuge from a world that can sometimes be too irrational.