Software Development Atrocities Committed in the Name of Flexibility

Although it is difficult to measure and quantify, most of us have heard and do believe that flexibility is an important part of software development. There are numerous advantages gained from software that is designed and implemented to be flexible and adaptive. However, I think it is worth asking, "Can we have too much of a good thing?"

The human body, in some situations, can indeed have too much flexibility. I think the same is true for software. In this blog posting, I will explain why I believe that we as software developers sometimes allow the desire for extreme flexibility to overshadow more important software characteristics.


Benefits of Flexible Software

There are several benefits to flexible software that have made this flexibility desirable to software developers.

Software Maintenance Benefits

It is a generally accepted observation that software maintenance costs significantly exceed development costs. Therefore, it is sensible to want to make the software as adaptive and flexible to bug fixes and other maintenance activities as possible.

Software Modifications/Enhancement Benefits

Another generally accepted observation in the software development world is that requirements and customer desires can and do change. Again, flexible software is key to dealing with this likely scenario.

Software Reuse Benefits

Advertised benefits of software reuse can sometimes be difficult to realize without significant cost because the original software is not flexible enough to be easily reused outside of its very narrow initial application. The idea here is that one can increase the reusability of one's software by making it more flexible.


Drawbacks of Developing Flexible Software

Software flexibility is generally a desirable characteristic because of the benefits described above. However, this flexibility is often far from free and, in some cases, can even be a detriment.

Higher Initial Cost of Flexible Software

In some cases, it can be significantly more difficult (and correspondingly more expensive) to introduce the desired flexibility into the developed software. In the extreme case, it is definitely cheaper to initially hard-code values all over the place, but we have all learned (usually through hard experience) that the costs of doing this are so great that we should take the time to use properties, XML, or other more dynamic values for our software. This is a case where the flexibility does come at a higher initial cost, but almost always "pays for itself" in the long term. This situation is one in which long-term benefits seem to dwarf the initial higher cost of more flexible software. However, not all situations are the same and there are situations where the long-term benefits may not justify or even cover the initial costs.

YAGNI

Software development projects have a notoriously bad record for not completing on time or on budget. There are several explanations for this as eloquently called out in the software development classic The Mythical Man-Month, but certainly one part of this is the desire we seem to have to add features and flexibility that will likely not need or only remotely may need. Extreme Programming introduced the term "You Aren't Gonna Need It" (YAGNI). This well-known term essentially serves as a reminder that we should not waste time and effort on things that we will never need. As the blog posting Using YAGNI Responsibly reminds us, this does not mean that we should shortchange long-term benefits like flexibility. Instead, we should be more realistic about the ratio of costs to benefits.

Greater Difficulty to Understand

Flexibility can sometimes make the software more difficult to understand or use initially. This occurs when certain techniques that make software more flexible also happen to create more classes, interfaces, and other pieces of code that must be understood.

Less Practical Use

Although appropriately chosen flexibility can increase the overall usefulness of a piece of software, too much or inappropriate flexibility can actually have the opposite effect. For example, a framework or library that is supposed to help improve development productivity will not likely be successful in that goal if it is as flexible as the programming language upon which it is based. To be useful, the framework usually needs to make some assumptions and limit some flexibility.


Examples of Flexibility Gone Wrong

Using Java String Rather than Java Enum for a Constant

A specific example that I have repeatedly observed in which the desire for flexibility has blinded experienced developers is overuse of the Java String. We had to wait a long time to get a Java enum, but the wait was worth it. It is painful to see the enum go unused even in situations screaming for its use. When I ask a developer why he or she is not using an enum rather than a String when the range is finite, the usual answer is something like "It is more flexible. What if I find myself needing to add a new value down the road?" This answer effectively implies that the benefit of being able to add an arbitrary String at any point in the future is worth more than the type safety, readability, comparison performance, and other benefits of the enum. This actually might be the case if the range of values changes frequently or is so large that it feels like it is not finite, but I have heard this argument multiple times for small sets of finite and fixed data with no obvious changes or additions in the near future or in the future at all.

Wide-Open SQL Statements

You don't have to work in software development with databases long to realize the dangers of using dynamic SQL in applications. While allowing any arbitrary string to be executed against the database is certainly more flexible, it is also far more dangerous and unpredictable. In fact, errors related to this kind of thinking are included in the CWE/SANS Top 25 Most Dangerous Programming Errors. Although it seems obvious that one should not do this, it is an example where enthusiasm for extreme flexibility could lead one astray and blind him or her to the downsides.


The Flexibility Continuum

Because flexibility in software is generally a desirable quality, the tricky part is knowing how much flexibility to add to a piece of software for long-term benefits without running up extraordinary costs or creating something too complex to maintain or use. Part of the difficulty is that the appropriate degree of flexibility is difficult to identify because it depends on context and because it depends on the often unpredictable future. Experience improves our ability to predict when the extra effort to add flexibility will pay off, but we cannot expect to always be correct. Sometimes, we will plan more flexibility than is actually needed while at other times we may wish we had spent a little more time making our application more flexible. Still, I think it is better to use our best engineering judgment when deciding how flexible to be than to blindly go to one extreme (no flexibility) or the other (complete flexibility at all cost).

In general, "throw-away" code such as demonstration code or prototypes will require less flexibility while larger and longer-term applications will require much greater flexibility. The longer an application is in production and use, the more likely that users will want new features and changes. Frameworks and libraries intended to be used by applications tend to need much greater flexibility than end applications that are not intended to be used as part of another application. Even with these guidelines, deciding on the exact level of flexibility for a given software project is still difficult due to the largely unforeseeable future events

It seems that sometimes we see extremism in which the developer does not believe in any flexibility, hard codes everything, doesn't design for extensibility, and generally makes software nearly impossible or extremely costly to even slightly enhance. However, the opposite extreme is to spend undue time and effort to obtain unnecessary flexibility and to sacrifice other software benefits to obtain that flexibility.


Flexibility's Best Uses

Where we choose to apply flexibility is an important consideration. Flexibility tends to be most useful if it is at an architecture or design level. Low-level code is almost always easier and cheaper to change than architectural concepts. Many good practices will naturally lead to greater flexibility while at the same time delivering other benefits. For example, modularity can make code easier to understand while at the same time increasing code's flexibility. Extensibility can be a benefit now and in the future. In fact, extra effort to justify additional flexibility is easiest to justify when we can realize benefits of that flexibility both now and in the long-term.

The worst time to expend significant resources on flexibility is when that proposed flexibility has minor and unlikely benefits now or in the future. What is especially frustrating is when one exerts significant time and effort for a level of flexibility that is truly unattainable or unrealistic, but wastes time and effort trying to achieve it anyway.


Conclusion

It is generally a good idea to design and implement our software to be extensible, maintainable, and flexible. However, even flexibility can be taken to an extreme where we spend inordinate amounts of time, money, and effort to design and write the software features that may never be needed and we may even end up with something too flexible to be used practically. Software development is filled with trade-offs and the degree of flexibility we should employ in our applications is a trade-off with the amount of time and resources to spend to gain that degree of flexibility.

Flexibility is a necessary characteristic of any software application of long term significance, but the desire for extreme flexibility does not justify ignoring all other desirable software characteristics. Depending on the context, duration, and future use of a software application, flexibility can be anywhere from a highly desirable characteristic to a necessary characteristic. However, flexibility must not be the excuse for over-engineering, over-designing, or over-complicating an application; or the excuse for indulging in a software development guilty pleasure; or for making a software project go over time and budget. Above all, the ability to easily change our software for something that might happen in the future should not limit our ability to deliver what is needed and desired today.