Monday, March 14, 2011

Software Development Isn't About Beans

The term bean counter is often used with a negative connotation. One of the negative connotations often associated with the term "bean counter" is that these folks are incapable of looking past quantitative measures to look into alternative measures that are more qualitative and/or intangible. In software development, bean counting is often particularly ineffective. Attempting to count beans to measure productivity or estimate future work often leads to disappointment and unsatisfied expectations. In this post, I look at some of the ways that software development managers attempt to count "beans" that are not really there or are not really indicative of what they're actually trying to measure.


Lines of Code Are Not Beans

Perhaps the most misleading and falsely assuring software development metric of all time is the infamous line of code. I have blogged previously on the unintended consequences associated with counting lines of code. The reason this is such a terrible metric is that it presumes that every line of source code is a bean that can be counted. There can be some value to measuring lines of code in a large aggregated sense, but any value in this metric is lost as the numbers of lines of code is reduced.

The "bean" assumption is that all lines of code are equally difficult to write and maintain. This is, of course, simply ridiculous. One programming language may be more difficult to write in then another, one developer make write faster than another, one piece of code is more difficult to write than another, and so on. Tactics such as trying to only compares lines of code for the same language or same team of developers might help, but the truth of it is that lines of code will always remain only useful in a very large macro perspective.

I have often found that fewer lines of code can be a little more challenging to write than a greater number of lines of code that do the same thing. In other words, it may take me longer to write fewer, more concise lines than it does to write longer verbose code. The paradox is that fewer lines of code may require more time to write (though often less time to maintain). Not all lines of code are created equal and any presumption of such is folly.


Developers Are Not Beans

Another area in which the bean counters like to count "beans" even when it makes little sense to do so is in terms of staffing. To the bean counter, developers are resources that are easily switched as needed. There is little concern about the differences in experience, skill, and skill set between developers. The idea is that any developer can be plugged into an position interchangeably with any other developer. This is absurd except for on the most trivial of projects.

The consequences of the terrible assumption that all developers are the same quality "bean" is one of the core covered topics in Frederick P. Brooks, Jr.'s software development management classic The Mythical Man-Month. In Chapter 2, Brooks states, the now famous quote, "The man-month as a unit for measuring the size of a job is a dangerous and deceptive myth." He adds, "The man-month is a fallacious and dangerous myth, for it implies that men and months are interchangeable" (Chapter 18, 2.6). Not everyone agrees that this presumption is a problem.


Requirements are Not Beans

Software requirements vary significantly in terms of difficulty involved in implementing them, but they still are often treated as beans of the same type. To the bean counter, implementing ten requirements sounds better than implementing one requirement, although in practice it may be that the single difficult requirement is for functionality that is far more useful than the ten smaller, simpler requirements. Great progress can be perceived in rapidly working off requirements only to become surprisingly slower when the more difficult requirements are encountered. To a bean counter, the "rapid initial productivity" followed by "reduced productivity" is baffling.


Defect Reports Are Not Beans

It is very difficult to resist the urge to count each defect report the same. In practice, there are vast differences in the difficulties of fixing different defects. Appropriate priority settings can help measure the differences in importance or value of defect reports, but it is more challenging to measure the real effort required to resolve a defect. One really nasty defect might require much more effort than numerous easier defects. To the bean counter, though, the expert who takes one week to work off the nasty defect appears less productive than the average developer who can work off many more than that of the easier defects in that same week. Not all bugs are created equal.


Other Bean Counter / Software Development Mismatches

In Open Cubicles Must Die, Jim Showalter discusses how smaller cubicles makes sense to bean counters who can only focus on quantitative measures and not on less tangible consequences such as reduced productivity.


Why Aren't There Software Development Beans?

Brooks's The Mythical Man-Month provides background into why software development tasks are not easily categorized as beans or widgets. Brooks states, "Programming is fun because it gratifies creative longings built deep within us and delights sensibilities." Brooks likens the programmer to the poet, stating, "The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air from air, creating by exertion of the imagination."

Software development is interesting because it combines technology with creativity. It is difficult to measure the "quality" of a poem, a painting, a sculpture, or other creative work of art via quantitative measures. It is similarly difficult to measure the significant creative portion of software development with quantitative measures. Only trivial and uninteresting software development (that which lacks need for creativity) can be measured with quantitative measures. The more challenging and interesting a software project is, and the more creativity it requires, the more difficult it is to measure the project's progress or estimate its costs using the quantitative measures discussed in this post. Software development tasks are not beans and cannot be counted or treated as such.


Why Insist on Beans Where There Aren't Any?

The bean counters don't uniformly misunderstand the nature of the software development business. Most of them understand that not all developers are the same, that not all lines of code are the same, that not all requirements are the same, and that not all defects are the same. The reason they continue to use these metrics like beans is that this is the only way they can make these things fit into their Excel spreadsheets, Microsoft Project projects, or other favorite bean counting tools.

The problem is that the measures described above and similar metrics are "countable" (quantifiable), but are typically so different from each other that counting them arbitrarily makes little sense. It would be like a bean counter counting lima beans, chili beans, coffee beans, and other types of beans as the same type of beans. If one was trying to make coffee or chili, it would matter which type of bean was available! The bean counter has the option to measure the beans by weight rather than by number. This might be a more accurate metric for certain needs. Because software development lacks physical entities that can be counted or weighed, we are stuck doing our best trying to assign "weights," but it is an inexact science.

We often talk in software development about the object-relational impedance mismatch, but a far more difficult such impedance mismatch may be the mismatch between the art of software development and the quantitative side of software development management.


Conclusion

For Java developers, there are many types of "beans" in software development: JavaBeans, Enterprise JavaBeans, JMX Managed Beans, Spring Beans, and so on. These are about the only beans in software development. There is too much creativity involved in software development to easily categorize components of it as beans. That doesn't mean we don't try or that we won't continue trying. After all, we've been doing it since before Brooks published The Mythical Man-Month in 1975.

1 comment:

Dustin said...

John Sonmez's post We Can’t Measure Anything in Software Development is an interesting read with some of the same themes as my post but also with many different and interesting examples and observations about the difficulty of measuring things in software.