Significant Software Development Developments of 2011

As I did in 2007, 2008, 2009, and 2010, I summarize some of the software development events of 2011 that I find most significant. All of the normal caveats still apply: this list is definitely shaped by personal experience, interests, and biases.

10. Functional Language Popularity

I cannot recall a single day in 2011 where I didn't run across a headline touting the virtues of functional programming in general or a functional programming language in particular as I browsed software development headlines. The ability to enjoy advantages of functional programming in the JVM have been one of the touted advantages of Scala (see honorable mention section of this post for more on Scala's big year). Programming languages such as Haskell (or Jaskell on the JVM) and LISP (invented by John McCarthy, who is mentioned in item #9) are obviously known for their Functional nature, but we have seen articles in 2011 about using functional aspects of other languages. These include Functional Programming in JavaScript (25 December 2011), the IBM developerWorks's series Functional thinking: Functional features in Groovy (started 22 November 2011), and Functional Programming in Java (15 December 2011).

The JVM space has seen a lot of recent activity related to functional programming. Besides languages such as Jaskell, Scala, and Clojure, there are also functional programming-oriented frameworks for Java such as Guava, fun4j, lambdaj, op4j, and Commons Functor.

Regarding writing code in functional programming languages, Bruce Eckel writes, "For me, one of the best things about functional programming is the mental discipline that it produces. I find it helps me learn to break problems down into small, testable pieces, and clarifies my analysis. For that reason alone, it is a worthwhile practice."

Functional programming languages have been around for a long time and 2011 was not even the beginning of renewed interest in them. However, 2011 seemed to be the year that FP really has taken off in mainstream software development blogosphere and press.

9. Legends Lost

This year has seen the deaths of multiple prominent technologists. Steve Jobs was probably the most well-known of these, but we also lost Dennis Ritchie and John McCarthy in the same year.

Physical deaths were not the only losses our industry suffered in 2011. In October 2011, Mark Pilgrim removed his considerable online contributions. This was reminiscent of why the lucky stiff's similar removal of his online presence (with considerable Ruby contributions) in 2009.

8. C++: Another 'Dead' Language Makes a Comeback

2011 was a big year for C++. C++11 (AKA C++0x) was approved and published, the first new standardized version since C++03. C++ was back in the news again and seems to be experiencing an invigoration similar to that Java is experiencing. See C++ and Beyond 2011: Herb Sutter - Why C++? for an example of this.

As mentioned in relation to concurrency, Microsoft is making overtures toward renewed interest in C++ with its C++ AMP (Accelerated Massive Parallelism). In addition, the July 2011 edition of MSDN Magazine featured an editorial called Why C++ Still Matters in which the author stated that "one of the things [MSDN readers have] consistently told us is that we need to not treat C++ like the crazy uncle in the attic." Bjarne Stroustrup's FAQ has an interesting answer to the question Is C++ in Decline?:

No, I don't think so. C++ use appears to be declining in some areas and to be on an upswing in others. If I had to guess, I'd suspect a net decrease sometime during 2002-2004 and a net increase in 2005-2007 and again in 2010-2011, but I doubt anyone really knows. Most of the popular measures basically measures noise and ought to report their findings in decibel rather than "popularity." Many of the major uses of C++ are in infrastructure (telecommunications, banking, embedded systems, etc.) where programmers don't go to conferences or describe their code in public. Many of the most interesting and important C++ applications are not noticed, they are not for sale to the public as programming products, and their implementation language is never mentioned. Examples are Google and "800" phone numbers. Had I thought of a "C++ inside" logo in 1985, the programming world might have been different today.

I think much of what Stroustrup says here about C++ and lack of coverage online has become true for general Java as well. It's not as exciting or trendy to write on traditional Java as it is on newer languages, but one should be careful about assuming that percentage of writers on a topic equates to percentage of users. C++ is not dead yet.

7. Java Community: OpenJDK and Java Community Process

2011 continued to be a big year for OpenJDK. In late 2010, IBM joined OpenJDK and Apple joined OpenJDK shortly thereafter. In 2011, Twitter also joined OpenJDK and Apache Harmony retired to the attic.

Other big news in the Java community involved the Java Community Process (JCP). JSR 348 ("Towards a new version of the Java Community Process") is described in the post JCP.next, JSR 348 -- Towards a New Version of the Java Community Process. This post concludes, "The success of the Java community depends upon an open and transparent JCP, so JCP.next is worthy of our praise and attention." The post lists some early actions of JSR 348, including "greater transparency by requiring that all development is done on open mailing lists and issue trackers" and that "recruiting process for Expert Group members will be publicly viewable." The aim is for "a more public, open, accessible and transparent JCP."

As additional evidence of the big year that 2011 was for the Java community, I cite the first-ever Java Community Keynote at JavaOne during JavaOne 2011. To me, the Java community seems more energized and enthusiastic than it has for several years.

6. JavaScript

2011 was a huge year for JavaScript. First, my citing of Dart, CoffeeScript, and Node.js as "honorable mention" developments (later in this post) and my citing of the year's biggest winner as HTML5 are evidence in and of themselves of the influence of JavaScript in 2011. Oracle announced at JavaOne 2011 their intention to provide a new server-side JavaScript implementation (Project Nashorn) to illustrate and test non-Java language JVM support and for a high-quality server-side JavaScript implementation that runs on the JVM. jQuery's success (and its own 2011 growth) is also another example illustrating the rising prominence of JavaScript.

5. The Return to Java and the Return of Java

A significant trend seen in 2011 was the return to Java by several prominent projects. Twitter, for example, joined the Java Community Process, after earlier moving their search architecture from Ruby on Rails to Java/Lucene. Another recent example has been Yammer moving part of their offering from Scala to Java. Other informative posts that provide evidence of resurgent interest in Java include Edd Dumbill's O'Reilly Radar posts in advance of OSCON Java 2011. Oracle Technology Network's Our Most Popular Tech Articles of 2011 is dominated by Java-related articles.

Alex Handy writes about the recent positive direction of Java in his post Look what 2011 washed in: The return of Java. He writes, "After a long hiatus and seemingly endless dithering by Sun, Oracle has officially given Java the kick in the pants it needed. ... Java is no longer standing still. "

Markus Eisele's post Moving Java Forward? A definition. A year in review. summarizes events in the world of Java in 2011. Eisele references the Oracle slogan "Moving Java Forward" and describes this as "probably the sentence of the year 2011." Regarding 2011, Eisele states, "For me personally this was a powerful Java year."

One could argue that, outside of bloggers and consultants, most Java developers never really left Java. Evidence for this argument include Java's consistently high (almost always #1) ranking in the TIOBE Programming Community Index and in programming language ratings such as Top 10 Programming Languages of 2011. Java remains remarkably popular and vibrant for a "dead language."

Java.net editor Kevin Farnham sums up the year 2011 from a Java perspective: "It's generally agreed that 2011 was a great year for Java and languages that run on the JVM."

4. Mobile Devices Remain All the Rage

Mobile device development was high on my list of software development developments last year and it seems to continue to dominate software development news and trends watching. This doesn't mean that mobile development is the most commonly performed development, but simply means it gets the most attention.

Mobile development should become even more significant with the proliferation of mobile devices. The Economist article Not Just Talk states, "Mobile phones are the world’s most widely distributed computers. Even in poor countries about two-thirds of people have access to one."

We don't need any more evidence of mobile development popularity, but it is worth noting that Objective-C's high ranking in the TIOBE Programming Community Index is almost exclusively due to the Apple mobile devices such as iPhone, iPad, and iPod Touch. Mobile devices from a single company have almost single-handedly moved a programming language from obscurity into the top ten.

3. Cloud Computing

I ranked cloud computing high on last year's list as well. It continues to be the most trendy of software development buzzwords, but also boasts real-life advantages for consumers and developers. I believe that there is no silver bullet and cloud computing does not change that belief. However, even without silver bullets, we've made strides in the software development industry with less significant steps forward and cloud computing does seem to provide advantages in select situations. Nothing (including the cloud) can be best for everyone all the time and in all situations, but cloud computing definitely seems to benefit certain people and situations.

A good read regarding state of the cloud in 2011 is 2011: When cloud computing shook the data center. This post highlights progress in 2011 in terms of both private and public clouds. One sentence in this article was of particular interest: "The road to simplicity seems paved with even more complexity."

An argument could be made (and I think Kevin Farnham has made it) that cloud computing received significant press in 2011, but really didn't have any one easily identifiable single major development in 2011. Rather, many smaller developments and intense online conversation on the topic have added up to make it another big year for the cloud.

2. Making Concurrency Easier

A major trend of 2011 appears to be that of making announcements about how various languages intend to make writing highly concurrent applications easier. This is one of the often advertised benefits of Scala and some of the other newer JVM languages. Even the JVM's oldest language, Java itself, is expected to continue to enhance its concurrency support (after major improvements in J2SE 5 and Java SE 7) with lambda expressions in Java 8 (see Kevin Farnham's Java.net editorial The State of Java: Meeting the Multicore Challenge for a nice overview). Microsoft has shown renewed interest in C++ for developing concurrent applications on its platforms. Specifically, Herb Sutter and Daniel Moth have talked about how C++ Accelerated Massive Parallelism (AMP) "introduces a key new language feature to C++ and a minimal STL-like library that enables you to very easily work with large multidimensional arrays to express your data parallel algorithms in a manner that exposes massive parallelism on an accelerator, such as the GPU."

It is not a coincidence that the rising popularity of Scala and of functional programming are happening as there is a greater interest in making concurrency easier. Martin Odersky has presented (at OSCON Java 2011) on why it's so difficult to get concurrency correct with shared state/memory. Bruce Eckel reminds us of the Brian Goetz presentation that provided significant evidence supporting the same proposition. Goetz talked about this and about greater support for this in Java 7 in a January 2011 interview:

Not only do improved libraries provide pre-baked versions of common idioms, but developers can often sidestep the complexity of concurrency by limiting shared mutable state in their programs. If you don't have shared state, you don't have to coordinate access to that state, which is where the trouble comes from. Functional languages are built on this principle, but we don't have to switch languages to get the benefit -- we just have to adopt the orientation that immutable state is better than mutable state, and seek to increase immutability and decrease sharing. In Java SE 7, we’ll be adding a framework for fork-join decomposition, which makes it easier to write algorithms that parallelize automatically across a wide range of hardware configurations.

1. HTML5's Ascendancy Continues

In last year's software development highlights post, I stated, "HTML5 finally seems to be gaining the traction it needs among the major browsers and among web developers and authors to be worth paying attention to." Many people having been paying attention to HTML5 in 2011 as its popularity has increased rapidly. There are numerous signs of HTML5's taking over the web development ecosystem as it makes inroads into the mobile devices as well. Indeed, one could say that the desire to have a common platform on which to develop applications for the myriad of mobile devices has been the biggest motivator in the successful rise of HTML5.

The victims of HTML5's success are piling up. We learned in the first quarter of 2011 that Google Gears was being eliminated (one of many Google project prunings that occurred in 2011). The announcement of Gears's demise explicitly acknowledged HTML5's role: "we've shifted our effort towards bringing all of the Gears capabilities into web standards like HTML5."

Later in 2011, Adobe announced that it'd no longer develop a Flash Play for mobile devices and quickly followed that announcement up with the announcement that Adobe would attempt to donate Flex (which was already open source, but under Adobe's stewardship) to the Apache Software Foundation. Many Flex developers, of course, hope that this isn't just Adobe's way of using open source as a dumping ground. In explaining the decision regarding Flex in Your Questions About Flex, the Flex blog acknowledges, "In the long-term, we believe HTML5 will be the best technology for enterprise application development." In announcing abandoning of mobile Flash Player in favor of AIR and HTML5, Adobe stated, "HTML5 is now universally supported on major mobile devices, in some cases exclusively. This makes HTML5 the best solution for creating and deploying content in the browser across mobile platforms."

Microsoft provides more examples of the persuasive power of the HTML5 momentum. Early details of Windows 8 and Windows Phone 7 Metro have scared some in the the Silverlight community as evidenced by posts such as Microsoft has Abandoned Silverlight and All Other Plugins in Metro IE, Did Microsoft Just kill Flash, Silverlight?, and Silverlight Developers Have the Smoothest Road to Metro.

HTML5 is still not fully here (Five Things You Can’t Do With HTML5 (yet)), but it already seems to be the victor.

Honorable Mention

These are topics that did not make my Top Ten for 2011, but were very close.

JavaFX Appears Here to Stay

It was at JavaOne 2010 that Oracle announced that JavaFX Script would be deprecated and announced other plans regarding JavaFX 2.0. However, it was in 2011 (again at JavaOne) that Oracle reaffirmed JavaFX's future by announcing plans to open source it and to make it part of standard Java via the Java Community Process. JavaFX 2.0 is now delivered with Oracle's JDK 7 Update 2. Oracle's statements and actions lead me to believe that JavaFX likely has a future after all.

Now that JavaFX SDK is included with Oracle's JDK 7 Update 2 and the JavaFX runtime is included with Oracle's Java 7 Update 2 JRE, JavaFX will be readily available on more machines. JavaFX includes some support for integration with HTML5 and is likely to add additional support for that integration, which should increase the adoption of JavaFX. 2011 may be the biggest year yet for JavaFX. The only reason it did not make my overall top ten is that I believe the interested portion of the development community is still relatively small, but that could change in the near future.

Node.js

The Node.js platform has generally enjoyed a great year in 2011. Although it was created in 2009, significant press attention seems to have been garnered in 2011. Besides numerous blog posts and articles on Node.js in 2011, several books have become available this year. These include Node Web Development (August 2011), Hands-On Node.js (May 2011), and The Node Beginning Book (October 2011). Two more books expected in 2012 are Node: Up and Running: Scalable Server-Side Code with JavaScript and Programming Node.js: Build Evented I/O Applications in JavaScript.

Not everyone in enamored with Node.js as shown by posts Node.js is Cancer and a mixed report in Is node.js best for Comet?

Scala

2011 has been a big year for Scala. In May, Greylock Partners (venture capitalists) invested $3 million (U.S.) in newly created Typesafe, the Scala/Akka company founded by Scala creator Martin Odersky and Akka creator Jonas Bonér that same month.

Scala has received considerable press in recent years and this has seemed to escalate in 2011. I often read comments to posts on Java or other JVM languages that have some enthusiastic Scala user pointing out how whatever task discussed in the post would have been easier in Scala. Bruce Eckel wrote Scala: The Static Language that Feels Dynamic in 2011. There has also been some negative press for Scala recently as document in my post Recent Community Conversation on Scala.

It is my belief that the very existence of such discussion and the fact that people actually cared about the discussion together imply that Scala is at a point where it's in the process of moving into the mainstream. Barb Darrow's article on GigaOM (same site that published Why Modern Applications Demand Modern Tools by two authors from Greylock Partners) called Scala sets sights on top-tier status among the Java faithful suggests that the folks at Typesafe feel similarly (and they are much closer to the situation than I am). Darrow's article starts, "To hear Typesafe tell it, the Scala programming language is about to join the ranks of top-tier development tools such as Java, C++, Ruby, and PHP. A new Scala plugin for the Eclipse integrated development environment (IDE) should help pave the way."

Speaking of Scala and Eclipse, it was announced this month that Scala IDE V2.0 has been released. One of the most common complaints about Scala has been lack of tool support, so this should definitely be welcome.

Other new developments for Scala in 2011 include the release of the Second Edition of Programming in Scala (release date listed differently in different locations and ranging from December 2010 to January 2011) and Cay Horstmann's announcement of his forthcoming book Scala for the Impatient (of which Typesafe has offered a free preview).

Scala, as was the case for the past year or two, continues to "just miss" making my top ten software development developments for the year. I believe that 2012 will be a significant year for Scala because my perception of Scala is that it is at an important juncture in terms of adoption. It seems that it is poised to really take off or flatten out in terms of adoption at this point and which it will be may start to become clearer in 2012.

Cloud and Service Outages

There were some large and highly visible outages this year. The normally highly reliable Blogger (my blog's host) was out for about 24 hours and Amazon's EC2 also went down in 2011. See The 10 Biggest Cloud Outages Of 2011 (So Far) for a list of cloud outages in the first half of 2011 alone.

Any architecture or infrastructure relying on a single point of failure is likely to end up suffering for that decision. These stories simply remind us that this applies to cloud as well.

Google Chrome and Firefox

Adoption of Chrome as web browser of choice continues to rise. Chrome 15 has been reported (StatCounter) to be the most popular single web browser (by specific version). It's surprisingly difficult to measure with certainty which browser is most popular because it depends on which site is doing the counting, but Chrome certainly has been catching up with Firefox in terms of adoption.

I looked at the Blogger Stats for my own blog for the week between December 16 and December 23 and the break-out (not version specific) was 39% Firefox, 23% Chrome, and 20% Internet Explorer. A snapshot of containing these statistics and other browsers is shown next.

Chrome's rapid rise in popularity seems to put Firefox at a disadvantage. However, reports of significant payments from Google to Mozilla for Google to be Firefox's default search engine show that Firefox still has life in it. The post Chrome Engineer: Firefox Is A Partner, Not A Competitor references a Google+ post by Chrome team member Peter Kasting in which Kasting explains his perspective on why Google made the nearly $1 billion deal with Firefox:

People never seem to understand why Google builds Chrome no matter how many times I try to pound it into their heads. It's very simple: the primary goal of Chrome is to make the web advance as much and as quickly as possible. That's it. It's completely irrelevant to this goal whether Chrome actually gains tons of users or whether instead the web advances because the other browser vendors step up their game and produce far better browsers. Either way the web gets better. Job done. The end.

Woody Leonhard writes that "Google needs Firefox now more than ever." Leonhard also observes, "The money's in search, not the browser."

Firefox enjoyed its own progress in 2011. The blog post Firefox: 2011 begins, "Firefox helped make the Web more awesome in 2011." It highlights the release of Firefox 4, Firefox for Android, and how Firefox "introduced Do Not Track to the industry." The post also includes the "Firefox: 2011" infographic (shown below with link to original).

Dart

In a move reminiscent of its release of the Go programming language, Google's announcement of the Dart programming language has created significant discussion in 2011. Lars Bak's announcement of Dart calls the language "a class-based optionally typed programming language for building web applications." Bak goes on to state that Dart is intended to be a "structured yet flexible language for web programming." In a separate interview, Bak states that "I think it's an exaggeration" to call Dart "a JavaScript killer." That stated, JavaScript certainly has its warts and Google has the clout to make web technologies successful (but some do fail).

Dart can be either compiled into JavaScript (reminiscent of Google Web Toolkit's Java-to-JavaScript compilation) or can be run in a virtual machine in the web browser. Dart is an open source language and is available at http://www.dartlang.org/.

CoffeeScript

Speaking of compiling into JavaScript, CoffeeScript is described on its main page as a "little language that compiles into JavaScript." That same main page adds, "CoffeeScript is an attempt to expose the good parts of JavaScript in a simple way." A book on this language, CoffeeScript: Accelerated JavaScript Development, was published in 2011.

NoSQL

NoSQL has generated tremendous development buzz in recent years, but Oracle's 2011 entrance into the world of NoSQL makes 2011 especially noteworthy for the movement. The announcement of CouchBase and SQLite collaborating on UnQL (Unstructured Data Query Language) could also lead to increased adoption of NoSQL databases.

Big Data

A post that articulates why 2011 was a Big Year for Big Data better than I could is The Biggest Big Data Stories of 2011. McKinsey Global Institute has called big data "the next frontier for innovation, competition, and productivity."

NetBeans

The NetBeans Team's End of Year Message 2011 reminds us of the major developments of 2011 for the NetBeans IDE. The message states that 2011 "has been a milestone year for NetBeans where we've hit the 1,000,000 active user mark." The message adds, "We've also released NetBeans IDE 7.0 and NetBeans IDE 7.0.1, and underpinned the success of many other projects." The message also states that NetBeans 7.1 (which is currently in Release Candidate 2) is expected to be released in early 2012. One of the major inclusions with NetBeans 7.1 is JavaFX 2.0 support.

Tech Jobs Market Making a Comeback

The availability of technical jobs, including software development positions, appears to be rising in 2011. Dan Tynan writes, "Now might be just the moment" to say to again say to oneself, " I'm glad I chose a career in tech."

Mono

Mono, the "cross platform ... open source implementation of Microsoft's .Net Framework based on the ECMA standards for C# and the Common Language Runtime" had a big year in 2011. The post Mono in 2011 covers the year 2011 for Mono fairly exhaustively and states, "2011 was a year that showed an explosion of applications built with Mono." Among several developments related to Mono, one that stood out to me was the creation of Xamarin by members of the laid-off former AttachMate/Novell Mono team. This is certainly welcome news for an open source project that some feared was doomed at that point. Today, Mono promises the sharing of code "between iOS, Android and Windows Phone 7."

Devops

"Devops" continued to be a much-hyped concept in 2011. I still struggle to understand exactly how this movement has helped me or will help me, but I do understand the issues the movement/effort is intended to address. Wikipedia defines devops as "an emerging set of principles, methods and practices for communication, collaboration and integration between software development (application/software engineering) and IT operations (systems administration/infrastructure) professionals." Neil McAllister has called devops "IT's latest paper tiger."

Interconnectedness

Most of the items cited in my top ten of 2011 list and in the honorable mention section are highly related to at least one other item in the lists and often to multiple items. For example, functional programming, Scala, making concurrency easier, and the loss of John McCarthy are all related. Similarly, JavaScript is related to HTML5, Dart, CoffeeScript, mobile devices, cloud development, and web browsers.

Conclusion

2011 was yet another year that saw significant developments and advances in the software development industry. The lists compiled in this post indicate how broadly spread these advances were, affecting different programming languages, different deployment environments, and different stakeholders. When one considers that this is a list written by a single individual, it quickly becomes apparent that the actual breadth of advancement is much greater than these biased lists.