Building on Sand: Software without theoretical backing

I've been working in a corporate setting for seven months now. It's a different setting to be clear than what I have been used to in college and otherwise. There's a stark difference when we compare the two settings. This post is a discussion on what I found those differences to be and why they are important.

First off, in academia writing code is usually the last step in building anything. The way we usually went about it was to first define the inputs and outputs to the system. With that in place we defined the constrains and expectations from the system. Perhaps something needs to run in under a second, or perhaps we require something to run with a RAM cap of 500 MiB and so on. The constrains are the harshest we can set without inching onto the territory of premature optimization. We cannot allow ourselves the luxury of powerful hardware since most of academia is penny rich. Academia thus suffers at times because of people's inability to define the problem in academic terms.

In industries it seems to be different. The output is usually defined in simple terms without the formalism of academia. This lets it change as per customer need. The downside is that it also allows people to abuse this ability. With a hazy target you can get away with changing our goal to state that your system is working. Since the industry is usually richer, it allows sub-par algorithms to be used in production and letting the hardware pick up the slack. 32 / 64 GiB RAM systems are commonplace especially since bad algorithms are everywhere and RAM is cheap. Inputs to the system are usually defined well since without that code cannot be put down.

The major difference is the hurry to put down things in code. This is the part which begets the title of this post. Algorithms and data structures are the bread and butter of any software company. Writing code without being clear on the algorithm is a bad practice. Of course one need not write algorithms on paper before writing the code but one must always think of them before writing the code. An important quality arises from this kind of thinking. You are able to clearly argue about the structures that your code really needs and about the ins and outs of the system. Once you begin to write code, the algorithm gets lost in the implementation.

Often times in industrial settings one hears the term "academic" with a condescending tone behind it. It is rarely the case that the word is associated with reverence. I find that to be disturbing. Those who prize practical knowledge above theoretical knowledge must at least be able to boast of being as good as the "academics" in implementing things. It is after all "practical" knowledge that they seek. In all my time writing software so far, I am yet to see someone who writes beautiful code without knowing the theory behind it. Theory is that tool which lets you kill problems before they arise instead of waiting for them to arise in practice and then dealing with them. Theory is what distinguishes a normal system from a civilization level system like Google search. I won't talk more on this as more accomplished minds have spoken on this topic.

Another itch I have to scratch with industries is their inability to distinguish features from distractions. With the ability to create software comes the enormous responsibility of deciding what to create. When you don't consciously question the objective of your creation you run the risk of creating useless things. Since your time is a limited resource you have successfully spent time on things you will not use ultimately.

Building software is to be treated with as much care as building rockets. Simply because it is cheaper for you to build it does not mean you can adopt evolution as your software development strategy. When you build on sand, don't expect to build castles.