Twitter Changes: Lucene and Java
The post Twitter Search is Now 3x Faster discusses how Twitter's new search architecture uses Lucene and a Java/Netty-based server called Blender.
This post is interesting for a variety of reasons. First, it gives a glimpse into the architecture behind Blender, which is a Java-powered implementation. There is discussion about why such significant performance improvements were seen and how Netty aided the process.
A second reason this post is interesting to me is that the migration discussed in this post is from two most favored open source products (Ruby on Rails and MySQL). I think this is a good illustration of how the correct tool should be selected for the job. The folks at Twitter obviously had good reasons for choosing Ruby on Rails for their search functionality, but the need for improved performance and better scaling has eventually trumped those reasons and they have moved to Lucene/Java/Netty/Blender.
Ruby on Rails has been trendy since the mid-2000s and there are still some people who feel compelled to write about its advantages over Java (even though it's kind of like comparing apples and oranges because Java SE/EE cover far more ground than does Ruby on Rails). There's lots to like about Rails and Rails popularized some concepts that numerous languages (including Java) and frameworks have benefited from. I like Ruby even more than I like Ruby on Rails, but the Twitter story is a reminder that nothing is best in all cases, no matter what the "experts" may tell you and the consultants may sell you.
Software Developers: Big Leagues and Minor Leagues
Although the post Consultants are pros, while corporate IT staff are minor leaguers and a response post Consultants are no better than internal IT departments are specifically about IT departments, but the discussion could easily be applied to software development as well. Any time one makes blanket statements about one group being better than another, there is likely to be good arguments made against that position. For example, I have known and worked with some brilliant and highly skilled consultants and I have also worked and known some brilliant and highly skilled internal software developers. Similarly, I have observed both consultants and internal developers who waste their own and others' time because they lack experience and skill.
Software development consultants do tend to have some of the advantages discussed in Erik Eckel's post Consultants are pros, while corporate IT staff are minor leaguers. I think it is typically (though not always the case) for a developer consultant to have experience with a wider breadth of technologies, libraries, languages, and frameworks than an internal developer. That's not always an advantage. Many of the development consultants I've seen in action are really good at a lot of things as long as you don't need to dive too deeply into those areas. Once you start getting into the everyday grind of bug fixes, maintenance, and just making things really work in a real environment, the consultant has often moved onto his or her next gig and doesn't need to mess with the deep and very thorny issues the internal developer faces.
The best developers I've known are a mix of consultants and internal developers. They share many traits more strongly than their classification as a consultant or as an internal developer. There is always a danger to stereotyping and this is evidence of that. I have known too many good internal developers who can compete with any consulting developer to make a blanket statement stereotyping consultant developers as major leaguers and internal developers as minor leaguers.
Open Source License Flow Chart
Speaking of open source, the OSS License flow chart combines humor with a large dose of truth to present an interesting method for determining which open source license to apply to one's pet open source project. The plethora of open source licenses have made developing and using open source products more difficult than doing so probably should be. Even with its humorous approach, this flow chart is somewhat helpful in differentiating the licenses to a small degree.
The Well-Grounded Java Developer
The forthcoming book The Well-Grounded Java Developer has recently received significant coverage. I've been excited about its content summarized in its subtitle ("Java 7 and Polyglot Programming on the JVM") and in its table of contents and I pre-ordered it as part of the Manning Early Access Program several weeks ago. Java 7 and use of non-Java languages such as Groovy, Scala, and Clojure are among the most popular topics in the land of Java and are the featured topics of this book.
DZone/JavaLobby featured an interview with the authors in James Sugrue's DZone Meets The Well-Grounded Java Developers. This article features questions and answers with Ben Evans and Martijn Verburg that cover topics such as their respective favorite features in Java 7. The post also references the authors' presentation Back to the Future with Java 7. Although I have not yet had the opportunity to review the chapters of their book already available as part of Manning's MEAP, I look forward to the book's completion based on the contents of the first chapter (available online for free review) that covers Java 7 features, based on their Java 7 Developer Blog, and based on Martijn's feedback on some of my blog posts.
Java and the Cloud
Besides Java 7 and polyglot programming, one other very significant topic often covered in relation to Java (and to software development in general) is cloud computing. Given this, I'm interested to see the posts that follow Sylvain Francois's Overview of Java development on the Cloud. The best description of this post and advertised follow-on posts comes from the post itself:
This first post aims to clarify and introduce the key concepts of development on Cloud in order to give a comprehensive view of the market with a focus on solutions announced last weeks in the Java ecosystem: Amazon Elastic Beanstalk, CloudBees, eXo Cloud IDE ... The next posts will focus on some of these solutions or certain types of uses.
Not everyone thinks the cloud is all that, but many do and it will be interesting to see more coverage of Java used in cloud computing.
Guardian Switching from Java to Scala
Similar to the post about Twitter switching its search architecture from MySQL/Ruby on Rails to Lucene/Java, the announcement that http://www.guardian.co.uk/ is switching from Java to Scala is interesting from the perspective, "What are the reasons for the change?" The InfoQ coverage of specifically mentions the switch to Scala for these reasons: "deliver functionality faster" and favoring Scala over Java after writing integration tests in Scala while writing main functionality in Java.
It's quite interesting to read other thoughts and motivating factors on this move to Scala (which is far from complete at this point and it sounds like new stuff is the focus of Scala at this point). I found the following quote about Scala adoption by experienced Java developers interesting and very similar to my experience with Java developers and Groovy:
One of the things we compare learning Scala against is moving to a different platform like Python/Django or Ruby on Rails. With Scala, at least 75% of what you're working with is the same as in Java. You can use the same libraries and IDE, the way you package jars and wars is the same, your runtime environment and runtime characteristics are the same. A good Java developer can learn to write Java-style code in Scala in a day, then they learn the power of closures and implicit conversions and very soon they're more productive than they were in Java.
Conclusion
It is impossible to read all posts and articles of interest out there, but the posts covered above are ones I think are well worth the read.
1 comment:
More details on Twitter's migration from Ruby on Rails to the JVM can be found in today's Twitter Shifting More Code to JVM, Citing Performance and Encapsulation As Primary Drivers.
Post a Comment