Theory Versus Practice

In theory, the first day of Spring was March 21 in the Northern Hemisphere. In practice, however, the Denver area today is seeing more winter-like weather than we've seen this past winter. The photograph shown next shows what it looks like in the Denver area as I write this. As I was out shoveling snow from the driveway and sidewalk for the third time, the pain reminded me of times in software development when "in theory" ended up costing me when transitioning to "in practice."



It seems that experienced developers are generally better at understanding the difference between actual practice and theory. Certainly, this has been true in my case as I have learned (often the hard way) that practice is seldom as easy as theory. I somewhat sheepishly recall some of the cases where I was certain that some action was trivial, but the effort ended up being substantial as unexpected surprises occurred.

With hindsight, it is often easy to look back and see that we should have realized and even expected some of the troubles that we did not think about when we were talking about "in theory." The Denver area sees blizzards like today's almost every March so for the experienced Denver area resident, there it little surprise about this springtime blizzard. Likewise, experienced software developers are more likely to have seen the wrinkles and regular obstacles that make "in practice" more difficult than "in theory."

There are many specific software development examples of "in practice" being more difficult than "in theory." One example is moving from one Java EE application server to another. In theory, this is seamless because of the compliance of these servers with the JEE specifications. However, use of JEE server-specific extensions often make this transition more difficult in practice.

Another example of "in practice" being more difficult than the anticipated "in theory" is the effect of underlying COTS products. For example, an otherwise trivial upgrade (in theory) in the version of Java used for an application can turn out hairy if underlying COTS products are not compatible with the upgrades. For example, in the days of transitioning from Java 1.3 to Java 1.4, a common problem that was not always realized or considered when thinking about the migration was the change in treatment of the unnamed package in Java. Even if developers knew their own code only used named packages for their own classes, they were sometimes surprised when the COTS products they used had or generated classes in the unnamed package. Their own classes in a named package could not access the COTS product's classes in the unnamed package.

Perhaps the most frequent example of "in theory" not transitioning well to "in practice" is when large applications are developed in a language, framework, or tool based on success or reading about success with that same product on a very small Hello World style example application. Often, these tools, languages, and frameworks don't scale particularly well in terms of ease of use when the application needs scale.

As a final example, even (perhaps especially) concepts often don't scale particularly well from "in theory" to "in practice." Although I do think I'm getting better at more quickly understanding whether the latest software development fad really has any substance to it, it is certainly true that I have been burned many times in my software development career by adopting a technology, language, methodology, or other approach that seemed really good "in theory" or "on paper," but turned out much worse or even mildly disastrous when actually put into practice.

Making decisions based on "in theory" has cost me significant time and effort. This is especially painful when someone else makes the decision based on theory despite my better judgment about likely issues that will make "in practice" much more difficult and expensive. Because of this, I try to remember that "in theory" is almost always easier and less expensive than "in practice" and therefore add time and money to the estimates for an "in theory" solution when comparing alternative solutions. The more I am familiar with a solution, the less sizable this additional buffer needs to be. This is because there is significantly less "in theory" involved when I have worked with and used the approach "in practice" already.

I don't know if it implies that we're all a bunch of optimists, but I have definitely seen repeatedly that we as software developers tend to underestimate how long it will take to accomplish certain tasks. This is especially true when we make decisions and estimates based on "theory" rather than on actual "practice."

Now I need to go back out and shovel again. The theory that it is now Spring seems so much nicer than what I am dealing with in practice.