Motivations and Explanations for Poor Software Decisions

In a previous blog entry called What's In a Name, I wrote about names of concepts in software development that have likely aided in the adoption of the underlying concept. In that blog entry, I used examples like POJO and Ajax to illustrate the point. I also mentioned vendor-driven architecture and resume-driven development as two other examples. These latter two examples describe influences of our software design and development decisions and that is the focus of this blog entry.

In this entry, I want to highlight some of the other common influences on our software development decisions. Each of these influences has a pithy name associated with it in this entry. I did not come up with these names and will try to link to sources that cover these names in more detail, but I do intend to add why I like each name and how it fits.

Because this is a relatively lengthy blog entry, I am listing the titles of each idea here. Note that this entry does not talk about obvious negative effects on software decisions such as insufficient funding, insufficient schedule time, unreasonable management expectations, or even unreasonable customer expectations (the customer is always "right" if they're paying). I am also not talking in this entry about failures due solely to inexperience or lack of skill. Rather, I am talking about what types of motivations and explanations we can find to explain why even bright and talented software developers sometimes make poor decisions with significant negative consequences.

The specific items I will focus on in this entry are:
* Resume-Driven Development
* Magpie Effect
* Vendor-Driven Architecture
* Not Invented Here Syndrome
* Copy-and-Paste Programming
* Lemming Effect
* The Emperor's New Clothes Syndrome
* Maslow's Hammer
* Can't See the Forest for the Trees


Resume-Driven Development

The term resume-driven development refers to development decisions influenced by how the experience will benefit one's resume or, more generally, one's future software development career. Shahid N. Shah covers this concept in his blog entry Resume Driven Development (RDD) [19 January 200] as does Luke Hohmann in his interview with Bill Venners (29 March 2004). Justice Gray also includes coverage of resume driven development in his entry New development methodologies for the 21st century (22 June 2007). This last entry is worth reading for some alternate types of development he cites such as "Cut and Paste Driven Development," "Meeting Driven Development," "Meeting Driven Development," and "Masterpiece Driven Development."

Magpie Effect

Jeff Atwood's blog entry on The Mapgie Developer (6 January 2008) helped provide another nifty name to describe another motivational characteristic present in many software development efforts. In this entry, he likens software developers to magpies because both (software developers and magpies) are typically very bright, but are sometimes distracted from more important things and ideas by new and shiny things and ideas.

The magpie effect (7 January 2008) is similar to resume-driven development in many ways because both are often related to "new" and "exciting" technologies and languages. However, the magpie effect seems less selfish than resume-driven development. While resume-driven development is largely based on selfish interest in padding one's resume, the magpie effect describes a less intentional natural (and often healthy) curiosity in things that are new. I want developers on my team to be inquisitive and always seeking to learn new things and better themselves. However, to avoid the magpie effect, I also want them to learn how to briefly investigate new things and only spend significant time on truly useful things. Also, I don't want my development team being distracted by new things that aren't necessarily any better than what I've already got except for the "newness" of it.

Vendor-Driven Architecture

Also referred to as "Markitecture" (marketing architecture), this concept references architectures and designs based on vendor stiff arming or other strong selling tactics. This is often a result of vendor promises just short of world peace and the end of world hunger. Decisions largely influenced by vendor hype and/or sales pressure often lead to great disappointment when the down and dirty details are much more difficult to manage on a real application with real users than in a little "Hello World!" Tinkertoy demonstration.

Several people have written about vendor-driven architecture. David Linthicum wrote an article on avoiding vendor-driven SOA architectures (9 November 2007) and Arctec Group has provided this interesting and slightly different perspective on vendor-driven architecture (17 February 2005).

Not Invented Here Syndrome

While we don't want to let vendor-driven architecture drive us into poor solutions, we can go too far in trying to do everything ourselves and miss out on the collective experience of many others. Whether the product be commercial or open source, has licensing fees or is freely available, or is large or small, we can benefit from not reinventing the wheel. Some developers think that they can also do something better than anyone else. They may be unwilling to reuse code from anyone even in their own organization. This attitude is often referred to as the Not Invented Here Syndrome.

This syndrome is often manifest by the developer who finds one minor disadvantage in someone else's product and thinks he or she needs to create their own alternative without the disadvantage. In many cases, this results in something with far more significant problems than the product being replaced. Even when the replacement is equal to or slightly better than the already available solution, it often comes at great cost to the project. Often, Not Invented Here Syndrome is a result of developers having inflated opinions of their own work and overly critical opinions of others' work. However, there are also times when this occurs simply because developers do not spend appropriate time researching available offerings.

Michael Nash's Overcoming 'Not Invented Here' Syndrome summarizes this syndrome and some of its costs. Most people understand that there are times when something should be done because great customization is needed or a particularly important characteristic is not emphasized in the currently available offerings. Joel Spolsky has written In Defense of Not-Invented-Here Syndrome (14 October 2001) and emphasizes that core business logic often probably should be invented here. I also like SourceMaking's Cut-And-Paste Programming anti-pattern entry (see also their entry on Reinvent the Wheel). Justice Gray's New Development Methodologies for the 21st Century (cited earlier as well, 22 June 2007) also demonstrates that Cut and Paste Driven Development is the opposite (and equally negative) extreme to the Not Invented Here syndrome.

Perhaps one of the best examples of avoiding the Not Invented Here Syndrome is the Spring Framework. From the beginning, the key participants behind Spring have followed a philosophy of not recreating portions of their framework that were already satisfactorily covered by other available open source products. This is very evident when you download Spring with its dependencies (such as spring-framework-2.5.1-with-dependencies.zip) and look at what those dependencies are.

Copy-and-Paste Programming

I covered Copy and Paste Programming already in the Not Invented Here section above. While it is best to not reinvent the wheel and to make use of better tested frameworks and components when they will take care of the job, a software developer ultimately must (and hopefully wants) to write his or her own code for certain situations. Because the same developers who copy-and-paste the code will likely maintain the code, it is important for the developers to understand "their" code and most copy-and-paste enthusiasts don't take the time to truly understand code they did not write themselves. We should shy away from writing our own frameworks and libraries when sufficiently functional ones exist with a broad user community and its associated testing and support, but likewise we should be willing to write code of our own when it is needed (which is still fairly often in many cases).

Another bad form of copy and paste programming is outlined by James Carr in Rant: Copy and Paste Programming. Carr points out that lazy developers will sometimes copy-and-paste their own code. While the code may be understood by the developer in this case, it is a very poor form of "reuse" because of the CS 101 identified problems of maintaining the same code base in multiple locations.

Whether copy-and-paste is the opposite of not invented here (copying everything from other sources found via Google, other search engines, or books) or the copy-and-paste is copying one's own code (it is invented here), it almost never is a good idea in the long run. It is even dangerous in prototypes because they often morph into production applications.

I remember a representative of a major IDE vendor years ago bragging about how her company's IDE supported reuse because it made copying-and-pasting so easy. My first thought was the obvious "That's not real reuse, or at least not effective reuse" and that was quickly followed by the thought that even if that was good (which it isn't), it was nothing new compared to CTRL-C/CTRL-V.

Lemming Effect

The Lemming Effect (23 November 2007) is a reference to an almost instinctual desire in software developers to do what everyone else is doing. For instance, an early EJB adopter may have have thought to himself or herself, "This is unnecessarily complex and I don't even see the point of using this." However, he or she may have then continued to be inundated with articles, books, bulletin boards, and forums (not so much blogs in those early EJB days) talking about the virtues of EJB and it would be easy for that developer to doubt his or her own experience and just go with the crowd.

The Emperor's New Clothes Syndrome

This is similar to the Lemming Effect and often accompanies the Lemming Effect. However, this more specifically focuses on pressures to adopt a new concept or technology because one or more "experts" (collectively forming the Emperor) advocates the new concept or technology vociferously. If a respected expert is pushing the idea, it is easy for those of us who don't feel so expert in that area to believe the expert's words even over our own contrary experience. There may be many other developers who feel this way, but no one seems to come out and say so.

The Lemming Effect and the Emperor's New Clothes Syndrome often go together and that is a particularly dangerous combination. It takes an extremely confident developer with extremely trusting management and clients to stand up to the peer pressure of the other lemmings and to at the same time stand up to the authority of the "experts."

Every major software development platform has its own examples of The Emperor's New Clothes syndrome. Harshad Oak points this out for the Java world in Java and the Emperors New Clothes (10 May 2006). Charles Cooper made observations of the Emperor's New Clothes related to .NET's earlier days in .Net and the Emperor's New Clothes.

Maslow's Hammer

Maslow's Hammer is the well-known concept that we in the software development community have also known as the Golden Hammer Syndrome.

The idea here is that once a developer learns something new and usually useful in their first application of it, that developer is likely to try to make that same approach work for everything he or she encounters after that. For example, a Java developer may think that everything should be written in Java even when the problem isn't Java-friendly. Some close-to-home examples in my own young development career included the desire to use object-oriented implementation inheritance far more often than I should have in my early days of object-oriented design and then later an overuse, misuse, and abuse of design patterns. It is very easy to see how these have their strengths, but then let them get out of control trying to apply them to just about anything. I think the Singleton design pattern is especially overused and abused in this manner.

It is not surprising that we want to use things that have worked well for us before. In fact, this is often a good thing. However, we need to realize why these approaches worked for us so well before and then apply the approaches to problems with the same needs. Applying the right solution to the wrong problem just won't work. No single technology, framework, approach, pattern, or programming language is best for all situations. When we start bending the problem requirements to match our implementation choice, we should see this as a red flag warning of a potential Maslow's Hammer situation.

Can't See the Forest for the Trees

The expression "Can't see the forest for the trees" is usually used to signify that someone is missing the bigger picture because of his or her focus on much smaller and usually less important details. In software development, this is most commonly manifest as a disproportionate focus on a single characteristic (or "ility") at the expense of other just-as-important (or even more important) "ilities."

In an ideal world, our software would have all the desirable characteristics. It would have characteristics such as flexibility, maintainability, extensibility, simplicity, scalability, reliability,testability, and readability in addition to high performance and other positive characteristics. In the real world, however, we often have to make trade-offs between these characteristics or at least consider all of them in any design decisions. It is all too easy to get so focused on one of these positive characteristics that we neglect the others or neglect the overall system needs.

An example of missing the forest for the trees is such a strong focus on making the software so flexible that it might be easily adapted in case it might so happen that one day years from now someone will need that flexibility. Meanwhile, our current development staff and maintainers must deal with all of the complexity that comes with this flexibility and we may never need to realize any of the benefits of that complexity anyway.

It is becoming commonly accepted in software development that we should not focus on performance first (except for the small percentage of cases where performance is known to be the most important issue). Instead, increasing consensus is that we should strive for other positive characteristics (especially maintainability) and then make improvements for performance as needed. This is not to say that there aren't cases for performance-driven development, but rather is to say that there is no reason to sacrifice code readability and maintainability for performance improvements that might be gleaned from obscure code or tricky design.

In the end, it is not so much a question of whether performance is more important or maintainability is more important. Rather, it is a question of which combination of positive characteristics is most necessary and desirable for our application. We should be prepared to make the overall system best for our clients without regard for a single "ility" unless that single "ility" is overwhelmingly the desired "ility" in our client's case.

---

UPDATE (9 August 2008): An interesting look at some different (or same but from a more cynical perspective) software development motivations is available at Madconomist.com Wrong Number and is actually a reposting of the original version on the Berkun Blog.