Saturday, November 13, 2010

Apple and Oracle Are Behind OpenJDK!

I received the November 2010 edition of Oracle's Java Developer Newsletter in my e-mail inbox last night. The lead story is titled IBM Joins OpenJDK (a subject of my previous post IBM and Oracle Are Behind OpenJDK!). On the same day, it was also announced that Apple is joining the OpenJDK effort as well! Henrik Ståhl supplies additional details and answers to likely questions in his post Oracle and Apple Announce OpenJDK Project for OSX. The numerous comments on Henrik's post demonstrate how well this is being received in the Java development community.

Besides the obvious advantage of having power players like Oracle, IBM, and Apple behind OpenJDK, there are other interesting observations to be had here. For one, there had been lots of wailing, gnashing of teeth and complaining about Oracle's heavy-handedness. Similarly, there had been complaints about Apple and its policies and a desire for Apple to provide its OSX implementation as open source. In the end, this happily turns out to be unnecessary angst. Henrik discusses this in his post:
This announcement is the result of a discussion between Oracle and Apple that has been going on for some time. I understand that the uncertainty since Apple's widely circulated "deprecation" of Java has been frustrating, but due to the nature of these things we have neither wanted to or been able to communicate before. That is as it is, I'm afraid.
 I think it's obvious why many of us had concerns about this, but I'm glad that in the end Oracle and Apple were doing what's best for Java.

Another interesting observation or question is what will Google do because or about this? Will Google join the OpenJDK to use in conjunction with Android?

The announcement about Apple joining OpenJDK is huge and positive news for Java developers. I also wonder if some of the many people who talk about Sun as if they could do no wrong and Oracle as if they can do no right will acknowledge that Oracle appears to be doing something with OpenJDK that Sun failed to do: bringing the biggest players in Java together behind a common open source implementation of Java.

Saturday, November 6, 2010

RMOUG Training Days 2011 Schedule-At-A-Glance Available

RMOUG has made the first edition of their Training Days 2011 Schedule-At-A-Glance available online. There are numerous interesting abstracts highlighted in this schedule. My two presentations are both scheduled for the first day and this is unlikely to change because of their respective durations. There is only one 30-minute slot and one 90-minute slot and I have one presentation in each of these slots. The typical RMOUG Training Days presentation slot is 60 minutes. Before highlighting some of the abstracts that look particularly interesting to me, I'll briefly summarize my own presentation abstracts.



Groovier Java Scripting
Session 2 (Wednesday, 16 February 2011, 10:30 am to noon, Room 406)

Groovy is a dynamic language that runs in the Java Virtual Machine and has full access to the Java SDK as well as third-party Java libraries and frameworks. Groovy supports most Java syntax as-is, but provides an even more concise syntax that makes it more suitable for scripting. Groovy can be run directly without explicit compilation and enjoys many other script-friendly benefits.

This presentation expands on topics discussed at RMOUG Training Days 2010 in the presentation "Applied Groovy: Scripting for Java Development." The presentation briefly summarizes select basic features and syntax of Groovy, advantages of using Groovy as a scripting language, and demonstrates how to use Groovy to improve building, developing, and testing Java-based applications. The presentation will emphasize how easy it is to write and parse XML with Groovy and how easy it is to manipulate database content with Groovy.

Although the basics of Groovy will be covered initially in the presentation for those who have not used Groovy previously, most of the syntax and other features covered in this presentation will be presented in conjunction with the use of Groovy in building, maintaining, and testing applications for the Java platform.



Session 4 (Wednesday, 16 February 2011, 2:45 pm to 3:15 pm)

HTML5 is the the forthcoming standard in hypertext markup for the modern web. Although work on HTML5 has been ongoing for several years, interest and support for HTML5 has increased rapidly in recent months. An example of this is the JavaOne 2010 announcement that JavaFX 2.0 will provide HTML5 support. This short presentation will describe some of the most exciting features of HTML5. The presentation also covers the reasons that HTML5 is important and describes some of its limitations and obstacles to adoption. Finally, HTML5 features already supported in three popular modern web browsers (Microsoft Internet Explorer, Firefox, and Google Chrome) will be demonstrated.




There are numerous other abstracts that appeal to my interests. I probably won't attend the Tuesday University Sessions, but Brad Brown's Building Enterprise Apps for the Android Platform looks interesting. In his abstract, Brown states something that I've felt and been telling people as well:
I liken Android to the 'IBM PC' of the mobile world. Apple came out with a great OS for the Mac, but it only ran on the Mac. The 'IBM PC' (DOS, which later became Microsoft Windows) was generic and ran on a variety of hardware. Android runs on hundreds of devices today.
Brown's abstract states that this University Session will "discuss how you can write an application for the Android that accesses your corporate data securely. From concept to publishing in the Android market - for sale or for free - private or public."

One disadvantage of my presenting A First Look at HTML5 in the 30-minute Quick Tip presentation slot is that I won't be able to attend Rob Osterburg's presentation Scala: A Concise Introduction through Code, Concepts and Demonstrations. I'm not normally in the habit of recommending attendees attend a presentation at the same time as mine, but I would have liked to attend this session because Rob is a friend and because the topic (Scala) is of significant interest to me. In fact, attendees at my Groovy presentation will likely hear me enthusiastically recommend his session on another JVM-based programming language.

Gwen Shapira (blog) has an abstract discussing another topic of high interest to me: NoSQL (see my coverage of a JavaOne 2010 presentation on NoSQL). Her presentation is called NoSQL Deep Dive and her abstract summarizes NoSQL: "NoSQL is a general name for a new type of database - One that throws away the relational model in favor of better scalability."

Another abstract title that stood out to me as I scanned the Schedule-At-A-Glance is Introduction to Mobile Application Development. This abstract states that the presentation will "cover introduction, approach, and leading practices for building applications in a mobile platform" and will provide an iPhone example.

The Rocky Mountain Oracle Users Group is deeply rooted in Oracle database technology and this year's slate of presentations, like most years before, provides numerous in-depth database-heavy topics. Similarly, it is not surprising that many Oracle tools received focused coverage in RMOUG Training Days 2011 presentations. The schedule is subject to change (one presentation still needs a title!), but it is interesting to start planning what I will attend.

RMOUG Training Days 2011 will be held February 15-17, 2011 (15 February is University Sessions) at the Colorado Convention Center in Denver, Colorado. The Colorado Convention Center has a Flash-based interactive floor map available for seeing where the rooms for each session are located.

Monday, November 1, 2010

Highlights of RMOUG SQL>Update Fall 2010 Newsletter

I received a copy of the Fall 2010 edition of the Rocky Mountain Oracle Users Group newsletter (RMOUG SQL>Update) this week and summarize some of its highlights here. The cover of this edition is perfect for a Fall/Autumn edition and features a photograph (by Lisa Collett) of aspen trees whose leaves have already turned yellow.

This edition of the RMOUG newsletter has several references to Training Days 2011. In her "From the President" column, RMOUG President Peggy King writes about the upcoming RMOUG Training Days 2011 and mentions that this conference will be held February 15-17, 2011, at the Colorado Convention Center. She states that there was an "overwhelming response to this year's call for papers" that "promises to continue the long standing RMOUG tradition of providing a conference packed full of exceptional presentations and numerous opportunities to share knowledge and hear the latest information about product development."  She adds, "So look for the Training Days 2011 Preliminary Conference agenda to be posted to the RMOUG website (www.rmoug.org) around the middle of November." Peggy states (and some advertisements in the newsletter reaffirm) that early registration for RMOUG Training Days 2011 will be available in early November.

RMOUG Training Days 2011 is also mentioned in Brad Blake's summary of Oracle OpenWorld 2010. He compares Oracle OpenWorld and RMOUG Training Days and concludes that RMOUG Training Days "is always a much better 'bang for the buck'!" in terms of providing "good, technical session[s]" that provide "actual hands-on tips and tricks of how to do things -- something you can take back to the office and USE."

There is more than one advertisement for Training Days 2011 in this edition, but the largest and most obvious is on the back cover. This advertisement includes the dates (15-17 February 2011) and the location (Colorado Convention Center in Denver, Colorado) and states that early registration will open November 4, 2010, and that early registration ends January 12, 2011.

RMOUG Training Days 2011 is not the only RMOUG training-related event coming up in the relatively near future. Page 5 of this Fall 2010 newsletter has an advertisement for the Fall Quarterly Education Workshop to be held on 19 November 2010. The advertisement on page 31 talks about the "next Quarterly Education Workshop in August at the Regis Science Ampitheatre," but it appears to me that that website should be checked for confirmation on the location.

In "MR Trace: An Interview with Cary Millsap of Method R," Cary Millsap answers a set of questions about Method R's offerings. Millsap talks about Method R Trace, a platform-independent Java-based extension of Oracle SQL Developer. Millsap describes this product's "purpose in life": "to completely eliminate all of the distractions associated with getting your hands on perfectly scoped Oracle extended SQL trace data." Millsap goes onto say that Method R Trace is designed for and targeted at software developers. He also references a 3-minute YouTube video on using this product and references his blog post on MR Trace's new file delete function. Finally, Millsap introduces "the best-kept secret in our company": MR Tools Suite. This suite includes three tools: mrls, mrnl, and mrskew. This article was unquestionably heavy on marketing and sales, but I did find it interesting that MR Trace is developer-focused and the tools do sound useful.

This edition of the newsletter continued two series of articles that began in the Summer 2010 edition of the newsletter. Jed Walker's "Quick Guide to Configuring Oracle 11gR2 Data Guard Physical Standby" had Part 2 covered in this Fall 2010 edition after having Part 1 covered in the Summer 2010 edition. Similarly, Dan Hotka continued his "Oracle Trace Facility" series: the Summer 2010 edition (Part I) focused on TOAD Trace File Analysis and the Fall 2010 edition (Part II) focused on Metalink SQL T (SQLTXPLAIN).

Other technical articles in the Summer 2010 edition of the RMOUG newsletter were Rama Balaji's "Flashback Logs Cleanup Without Downtime" and Kellyn Pedersen's "The CTAS, a.k.a. Processing Data The Jetson's Way." The other technical articles in the Fall 2010 edition of SQL>Update included Sue Harper's "Capture, Model, Design and Generate Using Oracle SQL Developer Data Modeler."

The Fall 2010 edition of the RMOUG newsletter had its "RMOUG Board Focus" focused on Tim Gorman. His column is an interesting case study in how what's important is in the eye of the beholder and how we all seem to think whatever we do is most important. I have a difficult time arguing with him that data is extremely valuable. I'm just glad we all can find things we enjoy that are different so we're not all trying to do the same thing. I don't think I could be a DBA for more than a few days without the desperate need to throw myself off the tallest building I could find, but he probably feels similarly about software development. It's good that we don't all want to do the same job. I'm not arguing that data's not valuable; it's just not that interesting!

As I stated previously, the Fall 2010 edition had numerous references to RMOUG Training Days 2011 and the Preliminary Agenda is expected to be available in November.  Ken Lee has announced that his abstract has been accepted for "APEX-O-MATIC – In a Blender with APEX and Web Services."

I was excited to learn this week that the two abstracts I submitted have been accepted. I will be presenting "Groovier Java Scripting" and "A First Look at HTML5." The Groovy presentation will expand on what I presented last year at Training Days 2010 ("Applied Groovy: Scripting for Java Development") on Groovy and provide more concrete examples of scripting with Groovy. The HTML5 presentation will look at what HTML5 offers in several browsers already and promises to offer for future web development. I plan to post the actual abstracts and additional details about my plans for these two presentations in a future post.

Thursday, October 28, 2010

An Important Software Developer Characteristic: Handling Criticism

It is important in many careers to be able to deal constructively with various forms of criticism. This is evidenced by all the posts and articles on the subject such as Are You Too Sensitive for Your Career, How to Accept Criticism While at Work, Ten Ideas How to Accept Criticism at Work, and Some Helpful Tips in Accepting Criticism. Although the skill of dealing with criticism effectively is useful for any career, it is particularly useful in software development. It is obvious that people working in industries such as retail, customer service, press, and entertainment must be able to deal with regular and sometimes heavy criticism. In this post I look at why the ability to effectively handle criticism can be just as important in software development as in any of these other industries. I only peripherally touch on what it means to "effectively handle criticism" because this is covered by numerous other articles and posts such as the ones previously referenced.

Code Reviews
I have long felt that writing source code is similar to writing prose. One of these similarities is that the author of code can feel a deep affection for what he or she writes that rivals what prose authors can feel about their work. This can make it difficult to hear others criticize that work. It is in many ways similar to an editor editing prose. In both cases, the author must be willing to listen to the criticism and, when the criticism is weighed and followed as appropriate, the product is better for it. The software developer, like the edited author, must be willing to hear that his or her work is not perfect and, hopefully in rare cases, may be severely flawed. For many software developers, this vehicle of criticism (the code review) is a regular part of the job.

Opinions and Subjectivity
Just as the prose editing process can be challenging due to the subjectivity involved in what makes for strong or interesting reading, much of what makes code "good" or even "beautiful" is in the eye of the beholder. It is sometimes even impossible to get two developers to agree on what is "simple" (see the excellent post Simplicity is Complicated). Because there is often more than one way to do something "correctly," the difference in options can become very subjective. This means that no matter how well a developer thinks he or she has done on a particular job, others can and will think that it could have been done better with their preferred approach. Software developers need to be confident, but this confidence can sometimes move into something else that leads to foisting opinions onto others. This is most readily seen in blog posts and even more so in feedback comments on blog posts. I typically dismiss "anonymous" comments or comments with no substance, but a fact-backed comment can be insightful, even if it disagrees with my thinking.

In general, we in the software development community are an opinionated bunch and we're not afraid to "share" those opinions with others even when they didn't ask for them.

Not Invented Here Syndrome
When talking about Not Invented Here (NIH) syndrome, the focus is often on how this leads to unnecessary new design, implementation, and testing of functionality that is already available and could be reused. However, another negative side of NIH can occur when the functionality is reimplemented before the original software's author has even "moved on" to something else. Often, NIH syndrome occurs in relation to third-party software or at least software written by other teams. When it is being reimplemented on the same team, this can become a form of criticism for the original author. There are implications that the original work was not good enough. Often, of course, this is not the case. There are many causes of NIH with only a small number of them being due to the original being truly insufficient for needs. Often, NIH is a result of a developer's desire to "write code" rather than "use code" or a developer's perception that it's easier to write something from scratch than to try and learn how to use something already in place. This can really get out of hand: I've seen pieces of functionality completely or almost completely re-written by each and every developer that touches that area of code.  See In Defense of Not-Invented-Here-Syndrome for a discussion of situations in which NIH is not necessarily a bad thing.

Early Project/Product Termination
Most successful software developers that I know are enthusiastic about software and enjoy building things. They invest so much time, energy, and effort into their work that it often becomes extremely important to see it through to completion and deployment. Given this, it can be crushing when the project is terminated early or does not reach completion and delivery. It can be very difficult to feel motivated and to move onto another project or another product when the last one ends this way. This can be interpreted as criticism when the product or project is terminated prematurely, especially if the developer feels his or her work was part of the reason for it.

Delivered Product Reviews
Even when a software developer is fortunate enough to see their product go to market, the opportunity for receiving criticism is not past. In fact, it may only be beginning at this point. Customers can be fickle and can be demanding. Even products that start fast can quickly lose popularity. Reviews can be brutal, even when not deserved.

Blogs, Articles, Books, and Presentations
Software developers who write blogs, books, and articles and give presentations also open themselves up to potential (even likely) criticism that can be very public. This may be one explanation for why more developers don't do these things.

Conclusion
There are numerous opportunities for a software developer to be directly and indirectly criticized. There is significant room in software development for opinions and subjectivity and this makes it even easier for developers to be critical of each other, even when nothing wrong has been done. These can be constructive or destructive forms of criticism. I don't think anyone really likes to receive criticism, but developers who know how to handle it appropriately are more likely to be happy and successful.

Saturday, October 23, 2010

Cannot Assume Serializability for Java Map's Nested Classes

A colleague of mine recently ran into an issue that I have not run into myself, but found to be interesting and, in my opinion, worth blogging about here. During some distributed Java development, my colleague observed that a NotSerializableException was encountered when he tried to pass the Set returned from Map's (HashMap in particular in this case) keySet() across the wire. Although I had never run into this myself, it is a commonly described issue online. Online references to this issue include Java HashMap.keySet return value, Bug Report 4501848, Bug Report 4756277, HashMap.Entry Not Serializable?, and Java Core APIs: private nonserializable classes in HashMap and Hashtable.

The following class demonstrates which Map implementations and which nested Map classes are Serializable and which are not for several popular standard Map implementations.

package dustin.examples;

import java.io.Serializable;
import java.util.Collection;
import java.util.EnumMap;
import java.util.HashMap;
import java.util.Hashtable;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Set;
import java.util.TreeMap;
import java.util.WeakHashMap;
import java.util.concurrent.ConcurrentHashMap;

import static java.lang.System.out;

/**
 * This class demonstrates that the nested classes for various types of Java
 * maps are not Serializable.
 */
public class NonSerializableCollectionsInnerClassesDemonstrator
{
   final String NEW_LINE = System.getProperty("line.separator");

   /** Enum describing types of Maps used in this demonstration. */
   private enum MapTypeEnum
   {
      CONCURRENT_HASH
      {
         public Map<Long, String> newSingleEntryMapInstance()
         {
            final Map<Long, String> map = new ConcurrentHashMap<Long, String>();
            map.put(1L, "One");
            return map;
         }
      },
      HASH
      {
         public Map<Long, String> newSingleEntryMapInstance()
         {
            final Map<Long, String> map = new HashMap<Long, String>();
            map.put(2L, "Two");
            return map;
         }
      },
      HASH_TABLE
      {
         public Map<Long, String> newSingleEntryMapInstance()
         {
            final Map<Long, String> map = new Hashtable<Long, String>();
            map.put(3L, "Three");
            return map;
         }
      },
      LINKED_HASH
      {
         public Map<Long, String> newSingleEntryMapInstance()
         {
            final Map<Long, String> map = new LinkedHashMap<Long, String>();
            map.put(4L, "Four");
            return map;
         }
      },
      TREE
      {
         public Map<Long, String> newSingleEntryMapInstance()
         {
            final Map<Long, String> map = new TreeMap<Long, String>();
            map.put(5L, "Five");
            return map;
         }
      },
      WEAK_HASH
      {
         public Map<Long, String> newSingleEntryMapInstance()
         {
            final Map<Long, String> map = new WeakHashMap<Long, String>();
            map.put(6L, "Six");
            return map;
         }
      };

      public abstract Map<Long, String> newSingleEntryMapInstance();
   }

   /**
    * Indicate where the provided class defines a class that implements the
    * java.io.Serializable interface.
    *
    * @param candidateClass Class whose serializable status is desired.
    * @return {@code true} if the provided class is Serializable.
    */
   public static boolean isSerializable(final Object candidateClass)
   {
      return candidateClass instanceof Serializable;
   }

   /**
    * Print (to stdout) a simple message describing if the provided object
    * is serializable or not.
    *
    * @param object Object whose class's status as Serializable or not is to be
    *    printed to stdout.
    */
   public static void printSerializableStatus(final Object object)
   {
      final Class clazz = object.getClass();
      out.println(
         clazz.getName() + " is " + (isSerializable(object) ? "" : "NOT ") + "Serializable.");
   }

   /**
    * Process a provided Map instance to indicate if the Map itself and its
    * nested classes are Serializable.
    *
    * @param header Header to be printed before Serializable results are printed.
    * @param map Map to be evaluated for Serializability or it and its nested classes.
    */
   private void processSpecificMapInstance(
      final String header, final Map<Long, String> map)
   {
      out.println(NEW_LINE + "===== " + header + " =====");
      printSerializableStatus(map);
      final Set<Long> mapKeySet = map.keySet();
      printSerializableStatus(mapKeySet);
      final Iterator<Long> mapKeySetIterator = mapKeySet.iterator();
      printSerializableStatus(mapKeySetIterator);
      final Collection<String> mapValues = map.values();
      printSerializableStatus(mapValues);
      for (final Map.Entry<Long, String> mapEntrySet : map.entrySet())
      {
         printSerializableStatus(mapEntrySet);
      }
   }

   /**
    * Method for explicitly creating EnumMap to check it and its nested classes
    * for Serializability.
    */
   private void processAndDemonstrateEnumMap()
   {
      out.println(NEW_LINE + "===== ENUMMAP =====");
      final Map<MapTypeEnum, String> map = new EnumMap(MapTypeEnum.class);
      map.put(MapTypeEnum.HASH, "HashMap");
      printSerializableStatus(map);
      final Set<MapTypeEnum> mapKeySet = map.keySet();
      printSerializableStatus(mapKeySet);
      final Iterator<MapTypeEnum> mapKeySetIterator = mapKeySet.iterator();
      printSerializableStatus(mapKeySetIterator);
      final Collection<String> mapValues = map.values();
      printSerializableStatus(mapValues);
      for (final Map.Entry<MapTypeEnum, String> mapEntrySet : map.entrySet())
      {
         printSerializableStatus(mapEntrySet);
      }
   }

   /**
    * Demonstrate Serializability status of different Map implementations and
    * the Serializability of those Map instance's nested classes.
    */
   private void demonstrateMapNestedClassesSerializability()
   {
      final Map<Long, String> hashMap = MapTypeEnum.HASH.newSingleEntryMapInstance();
      processSpecificMapInstance("HASHMAP", hashMap);
      final Map<Long, String> linkedHashMap = MapTypeEnum.LINKED_HASH.newSingleEntryMapInstance();
      processSpecificMapInstance("LINKEDHASHMAP", linkedHashMap);
      final Map<Long, String> concurrentHashMap = MapTypeEnum.CONCURRENT_HASH.newSingleEntryMapInstance();
      processSpecificMapInstance("CONCURRENTHASHMAP", concurrentHashMap);
      final Map<Long, String> weakHashMap = MapTypeEnum.WEAK_HASH.newSingleEntryMapInstance();
      processSpecificMapInstance("WEAKHASHMAP", weakHashMap);
      final Map<Long, String> treeMap = MapTypeEnum.TREE.newSingleEntryMapInstance();
      processSpecificMapInstance("TREEMAP", treeMap);
      final Map<Long, String> hashTable = MapTypeEnum.HASH_TABLE.newSingleEntryMapInstance();
      processSpecificMapInstance("HASHTABLE", hashTable);
      processAndDemonstrateEnumMap();
   }

   /**
    * Main executable function to run the demonstrations.
    *
    * @param arguments Command-line arguments; none expected.
    */
   public static void main(final String[] arguments)
   {
      final NonSerializableCollectionsInnerClassesDemonstrator instance =
         new NonSerializableCollectionsInnerClassesDemonstrator();
      instance.demonstrateMapNestedClassesSerializability();
   }
}

The above class runs through several popular Map implementations (EnumMap, HashMap, LinkedHashMap, TreeMap, Hashtable, WeakHashMap, ConcurrentHashMap) and prints out whether each Map implementation and its nested classes are Serializable.  The output it generates is shown next.

===== HASHMAP =====
java.util.HashMap is Serializable.
java.util.HashMap$KeySet is NOT Serializable.
java.util.HashMap$KeyIterator is NOT Serializable.
java.util.HashMap$Values is NOT Serializable.
java.util.HashMap$Entry is NOT Serializable.

===== LINKEDHASHMAP =====
java.util.LinkedHashMap is Serializable.
java.util.HashMap$KeySet is NOT Serializable.
java.util.LinkedHashMap$KeyIterator is NOT Serializable.
java.util.HashMap$Values is NOT Serializable.
java.util.LinkedHashMap$Entry is NOT Serializable.

===== CONCURRENTHASHMAP =====
java.util.concurrent.ConcurrentHashMap is Serializable.
java.util.concurrent.ConcurrentHashMap$KeySet is NOT Serializable.
java.util.concurrent.ConcurrentHashMap$KeyIterator is NOT Serializable.
java.util.concurrent.ConcurrentHashMap$Values is NOT Serializable.
java.util.concurrent.ConcurrentHashMap$WriteThroughEntry is Serializable.

===== WEAKHASHMAP =====
java.util.WeakHashMap is NOT Serializable.
java.util.WeakHashMap$KeySet is NOT Serializable.
java.util.WeakHashMap$KeyIterator is NOT Serializable.
java.util.WeakHashMap$Values is NOT Serializable.
java.util.WeakHashMap$Entry is NOT Serializable.

===== TREEMAP =====
java.util.TreeMap is Serializable.
java.util.TreeMap$KeySet is NOT Serializable.
java.util.TreeMap$KeyIterator is NOT Serializable.
java.util.TreeMap$Values is NOT Serializable.
java.util.TreeMap$Entry is NOT Serializable.

===== HASHTABLE =====
java.util.Hashtable is Serializable.
java.util.Collections$SynchronizedSet is Serializable.
java.util.Hashtable$Enumerator is NOT Serializable.
java.util.Collections$SynchronizedCollection is Serializable.
java.util.Hashtable$Entry is NOT Serializable.

===== ENUMMAP =====
java.util.EnumMap is Serializable.
java.util.EnumMap$KeySet is NOT Serializable.
java.util.EnumMap$KeyIterator is NOT Serializable.
java.util.EnumMap$Values is NOT Serializable.
java.util.EnumMap$EntryIterator is NOT Serializable.

The output shown above leads to several interesting observations. First, and most importantly from this post's perspective, is the fact that most (all but WeakHashMap) of the Map implementations are themselves Serializable, but most of them (all but Hashtable) have nested classes (for keyset, values, and entryset) that are NOT Serializable.

There are several approaches that can be used if the data from one of these classes nested within Map need to be distributed. The previously referenced bug reports provide an obvious "work around." One can copy the returned key set into its own new (and Serializable) Set:
Copy the values of the Set returned by keySet(), entrySet() or values() into a new HashSet or TreeSet object: Set obj = new HashSet(myMap.keySet());
This blog post has attempted to demonstrate that Serializable cannot be taken for granted. This is particularly true when dealing with nested classes in Map implementations.

Friday, October 22, 2010

Ten Tips for Using Java Stack Traces

Most Java developers are somewhat familiar with Java stack traces and how to read and analyze stack traces. However, for beginning Java developers, there are some areas of stack traces that might be a little confusing. In this post, I look at some tips for reading Java stack traces and responding appropriately based on what the stack trace reports.


1. Stack Traces Are Not Just for Errors and Exceptions

A common misconception among new Java developers and among people not familiar with Java is that stack traces indicate thrown exceptions. While stack traces are commonly associated with encountered exceptions, a Java developer can access a stack's trace at any time with no exception required. One way to do this is to call Thread.currentThread().getStackTrace() to access an array of StackTraceElements. I demonstrated this previously in my post The Surprisingly Simple StackTraceElement. It can be useful to access a stack trace even outside of an exceptional situation to understand how code flows. The StackTraceElement is also useful because it allows for granular access to pieces of the stack trace.

Another non-error stack trace usage is related to Java's ability to provide details about a running process when prompted from the command-line. This is described in more detail in a later section of this post.


2. Stack Trace is Presented from Latest to Earliest

One of the most important concepts of correctly understanding a stack trace is to recognize that it lists the execution path in reverse chronological order from most recent operation to earliest operation. In many ways, analyzing a stack trace correctly is like analyzing compiler errors correctly: order matters. With the compiler errors, it is generally best to analyze the first compiler error because that often causes subsequent compiler errors. Similarly, with a stack trace, it's often most useful to start with the most recent operation and work backwards as necessary up the stack trace to see how that condition was reached. For some exceptions, like some NullPointerExceptions, analysis of the line on which the exception occurred can be all that is needed to understand what went wrong.


3. Understand Chained Exceptions

Another important observation is that stack traces that do indicate exceptions often include chained exceptions. This is important to understand because it is often the case that one of the fundamental exceptions underlying the last encountered exception is the real culprit that must be appropriately addressed. Just as addressing the last compiler error reported before addressing the first reported error rarely is a good idea, it is similarly important to analyze the underlying exception. This chained exception facility was a welcome addition to Java 1.4 because it has allowed Java developers to more easily determine the real underlying cause of various exceptions. Often the most important exception to pay attention to in a long list of chained exceptions is the first one that occurred (typically listed at the very bottom of a lengthy stack trace and often preceded by "Caused by" notation).


4. Compile with Debug Turned On

When stack traces are being used to determine why an exception occurred or what the code flow was, it can be very useful to have access to the line numbers of each executed step covered by the stack trace. By default in Oracle's Java SE 6 javac, line numbers and source file names are included in stack traces. It is easy to make disable the printing of this information by setting javac's -g:none option, but this is not typically a good idea during development. Most build tools also make it easy to turn on debugging and control the debugging level. Ant, for example, provides the debug attribute for the javac task that can be set to true to turn debugging on.


5. Logging and Handling Exception Stack Traces

Because Java stack traces are so often associated with exceptions, it is useful to know how to report these properly. Fortunately, two of the most popular Java logging frameworks, java.util.logging and log4j, provide convenient methods for logging exceptions that will handle the stack trace information automatically. For example, java.util.logging provides an overloaded version of Logger.log that accepts a Throwable and log4j provides several overloaded methods on its Logger class that allow Throwables to be logged with a normal log message.

It is sometimes necessary or at least useful to access a stack trace in code situations other than logging. One way of doing this is to use the Throwable.getStackTrace() that returns an array of StackTraceElements. Another approach is to use one of the overloaded versions of Throwable.getStackTrace() that do not simply print the stack trace to standard error. These two overloaded methods allow access to the exception's stack trace via either a PrintStream or a PrintWriter.  The next two code snippets demonstrate extracting a String from a Throwable using PrintWriter and PrintStream respectively.

Extracting String from Throwable Via PrintWriter
/**
    * Provides a String representation of the provided Throwable's stack trace
    * that is extracted via PrintWriter.
    *
    * @param throwable Throwable/Exception from which stack trace is to be
    *   extracted.
    * @return String with provided Throwable's stack trace.
    */
   public static String accessExceptionStackTraceViaPrintWriter(final Throwable throwable)
   {
      final Writer writer = new StringWriter();
      final PrintWriter printWriter = new PrintWriter(writer);
      throwable.printStackTrace(printWriter);
      return writer.toString();
   }

Extracting String from Throwable Via PrintStream
/**
    * Provides a String representation of the provided Throwable's stack trace
    * that is extracted via PrintStream.
    *
    * @param throwable Throwable/Exception from which stack trace is to be
    *   extracted.
    * @return String with provided Throwable's stack trace.
    */
   public static String accessExceptionStackTraceViaPrintStream(final Throwable throwable)
   {
      final ByteArrayOutputStream baos = new ByteArrayOutputStream();      
      final PrintStream printStream = new PrintStream(baos);
      throwable.printStackTrace(printStream);
      String exceptionStr = "";
      try
      {
         exceptionStr = baos.toString("UTF-8");
      }
      catch(Exception ex)
      {
         exceptionStr = "Unavailable";
      }
      return exceptionStr;
   }


6. Multiple Approaches for Generating/Accessing Stack Traces

Java supports several approaches for accessing  stack traces. Several of these are outlined in An Introduction to Java Stack Traces and I've covered a couple here already. As described in Tip #1, the stack trace of the current execution can be retrieved from the current thread. The immediately preceding tip, Tip #5, covered extracting of a stack trace from an exception/error (Throwable).

There are times when it is useful to see what the current thread and other threads are doing. Java supports this by accepting a signal and responding by displaying pertinent details. Signal handling can be a little different across platforms, so it is worth learning how to send signals from the platforms used for one's Java applications. The Sun/Oracle java launcher documentation provides significant details about its JVM signal handling in the section that describes the -Xrs option (-X indicates non-standard option).  Besides explaining how to use -Xrs to turn off the JVM responding to received signals, this documentation also provides more details regarding the JVM handling of the signals SIGQUIT, SIGHUP, SIGINT, and SIGTERM.

Assuming that the -Xrs option is not used, using CTRL-BREAK in the Windows console window from which a Java application started leads to a thread dump of the current threads being displayed. This is demonstrated in the following screen snapshot of running a Java application and seeing the thread dump in Windows PowerShell.


For Linux and Unix systems, this same signal is sent differently. It can be sent with CTRL-\ (CTRL button followed by back slash) or with the command kill -QUIT pid (where 'pid' is the process ID).  The jstack utility tool is also easily used to see the threads associated with a particular process (often for specified process identifier).

Java IDEs such as Eclipse and NetBeans also provide mechanisms for seeing current thread dumps and stack traces.


7. Understand Java Class/Interface Naming Conventions

When reading and analyzing a Java stack trace, it is helpful to understand the class and interface naming conventions. One source of information on these conventions is provided in the Javadoc documentation for Class.getName(). This documentation explains the naming conventions for classes, interfaces, arrays, primitives, and so forth. It states that arrays are denoted with square brace: [. It also explains that array elements are marked with various letters depending on their types: L for classes and interfaces, Z for boolean, B for byte, C for char, D for double, F for float, I for int, J for long, and S for short. Knowing what these letters represent in array representations can make it easier to read information regarding these within stack traces.  As I discussed in the blog post Heap Dump and Analysis with VisualVM, these same naming conventions are also described in the JVM specification for .class file format.

The naming conventions in stack traces can be dependent on the JVM implementation, so it's worthwhile to learn what the naming conventions are for the particular JVM being used.


8. Be Cognizant of Synthetic and Bridge Methods

In many stack traces, it is fairly obvious which methods are being successively invoked by tracing through the stack trace.  Sometimes, however, there can be unrecognized methods. These might be from injected aspects or might be compiler-generated methods such as synthetic and bridge methods. I discussed synthetic methods in greater detail in my blog post Java's Synthetic Methods. As I stated in that post, synthetic methods are created by the compiler and usually have, in Sun's/Oracle's JVM, names such as access$100 and access$200.


9. Dude, Where's My Stack Trace?

Stack traces can be extremely useful in debugging and understanding how an application works, but they are obviously less than effective if not readily available. As described above, some Java products (such as the logging frameworks) make logging of stack traces easier. Other products log stack traces or write out their own stack traces to product-specific directories or logs. Therefore, it's important when using a new framework, tool, library, or IDE to learn where it writes its logs and stack traces. This tip also covers other essential steps such as configuring loggers appropriately to write to the appropriate locations.


10. Magnify the Power of Stack Traces with Tooling

The majority of this post has focused on reading and manually analyzing stack traces. Tools are now available that can make use of stack traces and make use of stack traces much more efficient. Eclipse's Stacktrace Console is a good example of what a Java IDE can do to make use of stack traces easier. Stack traces can be pasted into this tool and then portions of the stack trace can be clicked on to be taken directly to that class and method. NetBeans offers similar tooling features that support easier access of stack traces to the underlying classes and methods displayed in the stack trace.  Likewise, these IDEs also support generating/displaying thread dumps.

There are numerous other useful tools available in the Java ecosystem for more efficient use of Java stack traces and dumps. These include VisualVM (see my post Thread Analysis with VisualVM), jstack, and adaptj's StackTrace.

Another "tool" that is useful in conjunction with thread analysis is the powerful search engine. Although I agree that there is merit to not going right to the search engine for every problem, I have also been the beneficiary more times than I can recall of others' insight and experiences in resolving especially difficult problems. This is why I have written before that more software developers should write blogs.  Search engines make it much easier to find a description of a potential fix to some nasty problems. It is often the stack trace that provides the best clues for finding the appropriate matches via search engines such as Google, Yahoo!, and Bing. Stack trace excerpts are often found in forum threads and blog posts that either directly address the issue or provide enough information to help think through what related condition might be causing the issue.


Conclusion

The humble stack trace is one of the Java developer's most powerful tools. It is especially powerful in debugging and analyzing existing software, but can even be useful in software development. In this post I have attempted to outline some simple techniques for getting the most value from these stack traces.

Monday, October 18, 2010

Java's Synthetic Methods

In this blog post, I look at the concept of Java synthetic methods. The post summarizes what a Java synthetic method is, how one can be created and identified, and the implications of Java synthetic methods on Java development.

The Java Language Specification (section 13.1) states "Any constructs introduced by the compiler that do not have a corresponding construct in the source code must be marked as synthetic, except for default constructors and the class initialization method." Further clues as to the meaning of synthetic in Java can be found in the Javadoc documentation for Member.isSynthetic(). That method's documentation states that it returns "true if and only if this member was introduced by the compiler." I like that very short definition of "synthetic": a Java construct introduced by the compiler.

The Java compiler must create synthetic methods on nested classes when their attributes specified with the private modifier are accessed by the enclosing class. The next code sample indicates this situation.

DemonstrateSyntheticMethods.java (Enclosing Class Invokes One Nested Class Private Attribute)
package dustin.examples;

import java.util.Calendar;

import static java.lang.System.out;

public final class DemonstrateSyntheticMethods
{
   public static void main(final String[] arguments)
   {
      DemonstrateSyntheticMethods.NestedClass nested =
         new DemonstrateSyntheticMethods.NestedClass();
      out.println("String: " + nested.highlyConfidential);
   }

   private static final class NestedClass
   {
      private String highlyConfidential = "Don't tell anyone about me";
      private int highlyConfidentialInt = 42;
      private Calendar highlyConfidentialCalendar = Calendar.getInstance();
      private boolean highlyConfidentialBoolean = true;
   }
}

The above code compiles without incident. When javap is run against the compiled .class file, the output is as shown in the following screen snapshot.


As the above screen snapshot indicates, a synthetic method with the name access$100 has been created on the nested class NestedClass to provide its private String to the enclosing class. Note that the synthetic method is only added for the single private attribute of the NestedClass that the enclosing class accesses. If I change the enclosing class to access all private attributes of NestedClass, additional synthetic methods will be generated. The next code example demonstrates doing just this and the screen snapshot following it proves that four synthetic methods are generated in that case.

DemonstrateSyntheticMethods.java (Enclosing Class Invokes Four Nested Class Private Attributes)
package dustin.examples;

import java.util.Calendar;

import static java.lang.System.out;

public final class DemonstrateSyntheticMethods
{
   public static void main(final String[] arguments)
   {
      DemonstrateSyntheticMethods.NestedClass nested =
         new DemonstrateSyntheticMethods.NestedClass();
      out.println("String: " + nested.highlyConfidential);
      out.println("Int: " + nested.highlyConfidentialInt);
      out.println("Calendar: " + nested.highlyConfidentialCalendar);
      out.println("Boolean: " + nested.highlyConfidentialBoolean);
   }

   private static final class NestedClass
   {
      private String highlyConfidential = "Don't tell anyone about me";
      private int highlyConfidentialInt = 42;
      private Calendar highlyConfidentialCalendar = Calendar.getInstance();
      private boolean highlyConfidentialBoolean = true;
   }
}


As the previous two code snippets above and the associated images show, the Java compiler introduces synthetic methods on an as-needed basis. When only one of the nested class's private attributes was accessed by the enclosing class, only one synthetic method (access$100) was created by the compiler. However, when all four private attributes of the nested class were accessed by the enclosing class, four corresponding synthetic methods were generated by the compiler (access$100, access$200, access$300, and access$400).

In all cases of an enclosing class accessing its nested class's private data, a synthetic method was created to allow that access to happen. What happens when the nested class provides an accessor for its private data that the enclosing class can use?  That is demonstrated in the next code listing and in its output as shown in the next screen snapshot.

DemonstrateSyntheticMethods.java with Nested Class Public Accessor for Private Data
package dustin.examples;

import java.util.Calendar;
import java.util.Date;

import static java.lang.System.out;

public final class DemonstrateSyntheticMethods
{
   public static void main(final String[] arguments)
   {
      DemonstrateSyntheticMethods.NestedClass nested =
         new DemonstrateSyntheticMethods.NestedClass();
      out.println("String: " + nested.highlyConfidential);
      out.println("Int: " + nested.highlyConfidentialInt);
      out.println("Calendar: " + nested.highlyConfidentialCalendar);
      out.println("Boolean: " + nested.highlyConfidentialBoolean);
      out.println("Date: " + nested.getDate());
   }

   private static final class NestedClass
   {
      private String highlyConfidential = "Don't tell anyone about me";
      private int highlyConfidentialInt = 42;
      private Calendar highlyConfidentialCalendar = Calendar.getInstance();
      private boolean highlyConfidentialBoolean = true;
      private Date date = new Date();

      public Date getDate()
      {
         return this.date;
      }
   }
}


The above screen snapshot demonstrates that the compiler did not need to generate a synthetic method for accessing the private Date attribute in the nested class because the enclosing class accessed that attribute via the provided getDate() method. Even with getDate() provided, the compiler would have generated a synthetic method for accessing the date has the enclosing code been written to access the date attribute directly (as a property) rather than via the accessor method.

The last screen snapshot brings up another observation. As the newly added getDate() method shows in that screen snapshot, modifiers such as public are included in javap output. Because no modifier is shown for the synthetic methods created by the compiler, we know that they are package level (or package-private). In short, the compiler has created package-private methods for accessing private attributes.

The Java reflection APIs provide another approach for determining synthetic methods. The next code listing is for a Groovy script that will use the Java reflection APIs to conveniently provide details regarding the methods of the nested class shown above.

reflectOnMethods.groovy
#!/usr/bin/env groovy

import java.lang.reflect.Method
import java.lang.reflect.Modifier

if (args == null || args.size() < 2)
{
   println "Outer and nested class names must be provided."
   println "\nUsage #1: reflectOnMethods qualifiedOuterClassName nestedClassName\n"
   println "\nUsage #2: groovy -cp classpath reflectOnMethods.groovy qualifiedOuterClassName nestedClassName\n"
   println "\t1. Include outer and nested classes on classpath if necessary"
   println "\t2. Do NOT include \$ on front of nested class name.\n"
   System.exit(-1)
}
def enclosingClassName = args[0]
def nestedClassName = args[1]
def fullNestedClassName = enclosingClassName + '$' + nestedClassName
def enclosingClass = Class.forName(enclosingClassName)
Class nestedClass = null
enclosingClass.declaredClasses.each
{
   if (!nestedClass && fullNestedClassName.equals(it.name))
   {
      nestedClass = it
   }
}

if (nestedClass == null)
{
   println "Unable to find nested class ${fullNestedClassName}"
   System.exit(-2)
}

// Use declaredMethods because don't care about inherited methods
nestedClass.declaredMethods.each
{
   print "\nMethod '${it.name}' "
   print "is ${getScopeModifier(it)} scope, "
   print "${it.synthetic ? 'is synthetic' : 'is NOT synthetic'}, and "
   println "${it.bridge ? 'is bridge' : 'is NOT bridge'}."
}


def String getScopeModifier(Method method)
{
   def modifiers = method.modifiers
   def isPrivate = Modifier.isPrivate(modifiers)
   def isPublic = Modifier.isPublic(modifiers)
   def isProtected = Modifier.isProtected(modifiers)
   String scopeString = "package-private"  // default
   if (isPublic)
   {
      scopeString = "public"
   }
   else if (isProtected)
   {
      scopeString = "protected"
   }
   else if (isPrivate)
   {
      scopeString = "private"
   }
   return scopeString
}

When the above Groovy script is executed against the class and nested class shown above, the output is that shown in the next screen snapshot.


The results of the Groovy script shown in the previous image verify what javap had already told us: there are four synthetic methods and one non-synthetic method defined on the nested class NestedClass. The script also tells us that the compiler-generated synthetic methods are package-private scope.

The addition of synthetic methods to the nested class at package-private scope level is not the only thing the compiler did in the above example. It also changed the scope of the nested class itself from the private setting in code to package-private in the .class file. Indeed, while the synthetic methods were only added in the case where the enclosing class accessed the private attribute, the compiler always makes the nested class package-private even if it's specified as private in the code. The good news is that this is a resulting artifact of the compilation process, meaning that code cannot be compiled as-is against the changed scope level of the nested class or its synthetic methods. Runtime is where things can get dicey.

The class, Rogue, attempts to access some of the NestedClass synthetic methods. Its source code is shown next, followed by the compiler error seen when trying to compile this Rogue source code.

Rogue.java trying to access synthetic methods at compile time
package dustin.examples;

import static java.lang.System.out;

public class Rogue
{
   public static void main(final String[] arguments)
   {
      out.println(DemonstrateSyntheticMethods.NestedClass.getDate());
   }
}

The above code will not compile, even for the non-synthetic method getDate(), and reports this error:

Buildfile: C:\java\examples\synthetic\build.xml

-init:

compile:
    [javac] Compiling 1 source file to C:\java\examples\synthetic\classes
    [javac] C:\java\examples\synthetic\src\dustin\examples\Rogue.java:9: dustin.examples.DemonstrateSyntheticMethods.NestedClass has private access in dustin.examples.DemonstrateSyntheticMethods
    [javac]       out.println(DemonstrateSyntheticMethods.NestedClass.getDate());
    [javac]                                              ^
    [javac] 1 error

BUILD FAILED
C:\java\examples\synthetic\build.xml:29: Compile failed; see the compiler error output for details.

Total time: 1 second

As the above compilation error message indicates, even the non-synthetic method on the nested class is inaccessible at compile time because the nested class has private scope. In his article Java Insecurities: Accounting for Subtleties That Can Compromise Code, Charlie Lai discusses potential situations in which these compiler-introduced changes are security vulnerabilities. Faisal Feroz goes further and states, in the post  How to Write Secure Java Code, "Don’t use Inner Classes" (see Nested, Inner, Member, and Top-Level Classes for details on inner classes as a subset of nested classes).

Many of us can go for a long time in Java development without needing significant understanding of synthetic methods. However, there are situations when awareness of these is important. Besides security issues related to these, it is also to be aware of what they are when reading stack traces. Method names such as access$100, access$200, access$300, access$400, access$500, access$600, and access$1000 in the stack trace reflect synthetic methods generated by the compiler.