In the old days, software creation was intensively geeky. Myriad of minutiae required total concentration and focus to keep track in one’s head. One mishaps and the whole program crashes. It takes days to find that proverbial needle in the haystack: sometime a misplaced punctual mark or an extra character, else a subtle inconsistency in the logic or data format. End of the work day meant blurry eyes, sore shoulders, stiff neck, hunger, and mental exhaustion.
The concentration meant isolation: close the door, turn off the light, unplug the phone, get comfortable with the chair, and just submerge yourself into the code. A good software engineer will lose social skills and touch to the reality. As time progresses, the most talented became the most eccentric. We, the normal, tolerate them as we do weird scientists. They are the necessities of the modern society, but normal people need not play with them.
This changed around the Internet bubble time (1998~2001?), then again during the out-source craze (2004~2008). Heavy duty algorithm development is still cool, but large sum of money can be made by putting together a program with readily made modules and give it a nice skin to appeal to the mass. New tools enable amateurs to create software that looked professional, although with a crude engine. New generation of script languages encourage applets that glue together a system with many simple and single-function small programs that are easily doable by people with little or no formal programming trainings. Soon, some very serious software, such as the software behind this blog site, were done with one of those scripting languages.
Of course, those brilliant algorithm developers are still highly valued and respected. But the mainstream software engineers now thrive on communication and coordination skills and agility. They need to reach out to someone halfway across the world to understand how to interface with a module to get things done. If they try to create that module themselves, they will find the company losing the market, even with arguably better technologies.
Strangely, this blurs the lines between software engineer, engineering manager, and program manager. All of the sudden, everyone in this industry must be cross-trained for all three disciplines.
The famed Netflix competition add another twist to this topic. World-wide talents, one sponsored by AT&T, tried many approaches. It was not possible for Netflix to recruit those talents of the winning team with traditional in-house development. The entire effort would have costed Netflix far more than the $1m prize money. And the winning team demonstrated that global communication and coordination was no less important than their raw algorithm development skills.
Nerds these days are sociable and smart. These traits also give them high earning power. Isn’t that just cool!