Monday, September 15, 2014

Improving LibreOffice with Coverity Scan

Coverity, Inc. issued a press release this morning announcing that "the LibreOffice team" has "analyzed more than 9 million lines of code to find and fix more than 6,000 defects." In the press release, Zack Samocha, senior director of products for Coverity, states, "LibreOffice’s remarkable results after just two years [since October 2012] of using the Coverity Scan service reiterates the mission criticality of software testing for the open source community to find and fix software defects early."

This press release cites the Coverity Scan 2013 Open Source Report in explaining the degree of success the LibreOffice team has achieved. Specifically, according to the press release, the LibreOffice team has "reduced the project’s defect density from .8 to .08."

I was curious about some of the specific details associated with LibreOffice's use of Coverity Scan to reduce defects and improve quality and so took advantage of an offer to ask Zack Samocha some questions. The remainder of this post indicates my questions, Zack's answers, and some related references.

Q: What programming languages are used for LibreOffice (all/mostly C++ or some Java or other languages)?
A: The language used for LibreOffice is mostly C++.

Q: What is an example of one of the most common types of defects discovered and fixed in LibreOffice?
A: The top issues were:

  • Error handling issues = 2271
  • Null pointer dereferences = 1796
  • Uninitialized members = 1145

Q: Is this typical of other open source projects analyzed with Coverity Scan?
A: This is comparable for OSS projects [with more than] 1 million lines of code (LOC)

Q: What is an example of one of the most serious types of defects (high-impact) discovered and fixed? Is this typical of other open source projects analyzed with Coverity Scan?
A: The most serious are memory related. For example, memory-illegal accesses (there were 23) and memory–corruptions (there were 17). This amount is common for such large code base.

Q: Are there any additional metrics regarding the fixes to LibreOffice using Coverity Scan such as number of developers or number of person hours spent on the effort? Is there any estimate of how much of this effort was identifying the issues (running the scan) versus fixing them and testing the fixes?
A: In the past year, LibreOffice fixed more than 8,500 defects, assuming at least one hour per defects, which is conservative. That's about 365 days of work for a single developer.

Q: How does Coverity Scan differ from FindBugs, PMD, other static analysis tools, and IDEs' built-in static analysis support? What advantages does Coverity Scan offer instead of or in addition to those tools?
A: At Coverity, we believe in open source collaboration. Coverity complements Findbugs, PMD and others. In fact, in Coverity Scan and our enterprise products, we provide FindBugs defects in the same workflow as defects found by Coverity development testing solutions. Coverity and FindBugs are looking for different things. Coverity is designed to find critical, crash-causing defects where FindBugs is best suited for find coding-style and best practice-type issues. To illustrate the point, we conducted an experiment with the Jenkins open source build server under a controlled environment. We found 296 critical defects while FingBugs found 1010 coding style issues. There were only 30 defects that were found by both solutions. Coverity analysis tends to be more inter procedural, in addition Coverity covers OWASP10 for Security issues.

Related References

Thursday, September 11, 2014

Date/Time Formatting/Parsing, Java 8 Style

Since nearly the beginning of Java, Java developers have worked with dates and times via the java.util.Date class (since JDK 1.0) and then the java.util.Calendar class (since JDK 1.1). During this time, hundreds of thousands (or maybe millions) of Java developers have formatted and parsed Java dates and times using java.text.DateFormat and java.text.SimpleDateFormat. Given how frequently this has been done over the years, it's no surprise that there are numerous online examples of and tutorials on parsing and formatting dates and times with these classes. The classic Java Tutorials cover these java.util and java.text classes in the Formatting lesson (Dates and Times). The new Date Time trail in the Java Tutorials covers Java 8's new classes for dates and times and their formatting and parsing. This post provides examples of these in action.

Before demonstrating Java 8 style date/time parsing/formatting with examples, it is illustrative to compare the Javadoc descriptions for DateFormat/SimpleDateFormat and DateTimeFormatter. The table that follows contains differentiating information that can be gleaned directly or indirectly from a comparison of the Javadoc for each formatting class. Perhaps the most important observations to make from this table are that the new DateTimeFormatter is threadsafe and immutable and the general overview of the APIs that DateTimeFormatter provides for parsing and formatting dates and times.

Characteristic DateFormat/SimpleDateFormat DateTimeFormatter
Purpose "formats and parses dates or time in a language-independent manner" "Formatter for printing and parsing date-time objects."
Primarily Used With java.util.Date
java.util.Calendar
java.time.LocalDate
java.time.LocalTime
java.time.LocalDateTime
java.time.OffsetTime
java.time.OffsetDateTime
java.time.ZonedDateTime
java.time.Instant
Thread Safety "Date formats are not synchronized." "This class is immutable and thread-safe."
Direct Formatting format(Date) format(TemporalAccessor)
Direct Parsing parse(String) parse(CharSequence, TemporalQuery)
Indirect Formatting None [unless you use Groovy's Date.format(String) extension)] LocalDate.format(DateTimeFormatter)
LocalTime.format(DateTimeFormatter)
LocalDateTime.format(DateTimeFormatter)
OffsetTime.format(DateTimeFormatter)
OffsetDateTime.format(DateTimeFormatter)
ZonedDateTime.format(DateTimeFormatter)
Indirect Parsing None [unless you use deprecated Date.parse(String) or Groovy's Date.parse(String, String) extension] LocalDate.parse(CharSequence, DateTimeFormatter)
LocalTime.parse(CharSequence, DateTimeFormatter)
LocalDateTime.parse(CharSequence, DateTimeFormatter)
OffsetTime.parse(CharSequence, DateTimeFormatter)
OffsetDateTime.parse(CharSequence, DateTimeFormatter)
ZonedDateTime.parse(CharSequence, DateTimeFormatter)
Internationalization java.util.Locale java.util.Locale
Time Zone java.util.TimeZone java.time.ZoneId
java.time.ZoneOffset
Predefined Formatters None, but does offer static convenience methods for common instances:
getDateInstance()
getDateInstance(int)
getDateInstance(int, Locale)
getDateTimeInstance()
getDateTimeInstance(int, int)
getDateTimeInstance(int, int, Locale)
getInstance()
getTimeInstance()
getTimeInstance(int)
getTimeInstance(int, Locale)
ISO_LOCAL_DATE
ISO_LOCAL_TIME
ISO_LOCAL_DATE_TIME
ISO_OFFSET_DATE
ISO_OFFSET_TIME
ISO_OFFSET_DATE_TIME
ISO_ZONED_DATE_TIME
BASIC_ISO_DATE
ISO_DATE
ISO_DATE_TIME
ISO_ORDINAL_DATE
ISO_INSTANTISO_WEEK_DATE
RFC_1123_DATE_TIME

The remainder of this post uses examples to demonstrate formatting and parsing dates in Java 8 with the java.time constructs. The examples will use the following string patterns and instances.

/** Pattern to use for String representation of Dates/Times. */
private final String dateTimeFormatPattern = "yyyy/MM/dd HH:mm:ss z";

/**
 * java.util.Date instance representing now that can
 * be formatted using SimpleDateFormat based on my
 * dateTimeFormatPattern field.
 */
private final Date now = new Date();

/**
 * java.time.ZonedDateTime instance representing now that can
 * be formatted using DateTimeFormatter based on my
 * dateTimeFormatPattern field.
 *
 * Note that ZonedDateTime needed to be used in this example
 * instead of java.time.LocalDateTime or java.time.OffsetDateTime
 * because there is zone information in the format provided by
 * my dateTimeFormatPattern field and attempting to have
 * DateTimeFormatter.format(TemporalAccessor) instantiated
 * with a format pattern that includes time zone details
 * will lead to DateTimeException for instances of
 * TemporalAccessor that do not have time zone information
 * (such as LocalDateTime and OffsetDateTime).
 */
private final ZonedDateTime now8 = ZonedDateTime.now();


/**
 * String that can be used by both SimpleDateFormat and
 * DateTimeFormatter to parse respective date/time instances
 * from this String.
 */
private final String dateTimeString = "2014/09/03 13:59:50 MDT";

Before Java 8, the standard Java approach for dates and times was via the Date and Calendar classes and the standard approach to parsing and formatting dates was via DateFormat and SimpleDateFormat. The next code listing demonstrates these classical approaches.

Formatting and Parsing Java Dates with SimpleDateFormat
/**
 * Demonstrate presenting java.util.Date as String matching
 * provided pattern via use of SimpleDateFormat.
 */
public void demonstrateSimpleDateFormatFormatting()
{
   final DateFormat format = new SimpleDateFormat(dateTimeFormatPattern);
   final String nowString = format.format(now);
   out.println(
        "Date '" + now + "' formatted with SimpleDateFormat and '"
      + dateTimeFormatPattern + "': " + nowString);
}

/**
 * Demonstrate parsing a java.util.Date from a String
 * via SimpleDateFormat.
 */
public void demonstrateSimpleDateFormatParsing()
{
   final DateFormat format = new SimpleDateFormat(dateTimeFormatPattern);
   try
   {
      final Date parsedDate = format.parse(dateTimeString);
      out.println("'" + dateTimeString + "' is parsed with SimpleDateFormat as " + parsedDate);
   }
   // DateFormat.parse(String) throws a checked exception
   catch (ParseException parseException)
   {
      out.println(
           "ERROR: Unable to parse date/time String '"
         + dateTimeString + "' with pattern '"
         + dateTimeFormatPattern + "'.");
   }
}

With Java 8, the preferred date/time classes are no longer in the java.util package and the preferred date/time handling classes are now in the java.time package. Similarly, the preferred date/time formatting/parsing classes are no longer in the java.text package, but instead come from the java.time.format package.

The java.time package offers numerous classes for modeling dates and/or times. These include classes that model dates only (no time information), classes that model times only (no date information), classes that model date and time information, classes that use timezone information, and classes that do not incorporate time zone information. The approach for formatting and parsing these is generally similar, though the characteristics of the class (whether it supports date or time or timezone information, for example) affects which patterns that can be applied. In this post, I use the ZonedDateTime class for my examples. The reason for this choice is that it includes date, time, and time zone information and allows me to use a matching pattern that involves all three of those characteristics like a Date or Calendar instance does. This makes it easier to compare the old and new approaches.

The DateTimeFormatter class provides ofPattern methods to provide an instance of DateTimeFormatter based on the provided date/time pattern String. One of the format methods can then be called on that instance of DateTimeFormatter to get the date and/or time information formatted as a String matching the provided pattern. The next code listing illustrates this approach to formatting a String from a ZonedDateTime based on the provided pattern.

Formatting ZonedDateTime as String
/**
 * Demonstrate presenting ZonedDateTime as a String matching
 * provided pattern via DateTimeFormatter and its
 * ofPattern(String) method.
 */
public void demonstrateDateTimeFormatFormatting()
{
   final DateTimeFormatter formatter =
      DateTimeFormatter.ofPattern(dateTimeFormatPattern);
   final String nowString = formatter.format(now8);
   out.println(
        now8 + " formatted with DateTimeFormatter and '"
      + dateTimeFormatPattern + "': " + nowString);
}

Parsing a date/time class from a String based on a pattern is easily accomplished. There are a couple ways this can be accomplished. One approach is to pass the instance of DateTimeFormatter to the static ZonedDateTime.parse(CharSequence, DateTimeFormatter) method, which returns an instance of ZonedDateTime derived from the provided character sequence and based on the provided pattern. This is illustrated in the next code listing.

Parsing ZonedDateTime from String Using Static ZonedDateTime.parse Method
/**
 * Demonstrate parsing ZonedDateTime from provided String
 * via ZonedDateTime's parse(String, DateTimeFormatter) method.
 */
public void demonstrateDateTimeFormatParsingTemporalStaticMethod()
{
   final DateTimeFormatter formatter =
      DateTimeFormatter.ofPattern(dateTimeFormatPattern);
   final ZonedDateTime zonedDateTime = ZonedDateTime.parse(dateTimeString, formatter);
   out.println(
        "'" + dateTimeString
      + "' is parsed with DateTimeFormatter and ZonedDateTime.parse as "
      + zonedDateTime);
}

A second approach to parsing ZonedDateTime from a String is via DateTimeFormatter's parse(CharSequence, TemporalQuery<T>) method. This is illustrated in the next code listing which also provides an opportunity to demonstrate use of a Java 8 method reference (see ZonedDateTime::from).

Parsing ZonedDateTime from String Using DateTimeFormatter.parse Method
/**
 * Demonstrate parsing ZonedDateTime from String
 * via DateTimeFormatter.parse(String, TemporaryQuery)
 * with the Temple Query in this case being ZonedDateTime's
 * from(TemporalAccessor) used as a Java 8 method reference.
 */
public void demonstrateDateTimeFormatParsingMethodReference()
{
   final DateTimeFormatter formatter =
      DateTimeFormatter.ofPattern(dateTimeFormatPattern);
   final ZonedDateTime zonedDateTime = formatter.parse(dateTimeString, ZonedDateTime::from);
   out.println(
        "'" + dateTimeString
      + "' is parsed with DateTimeFormatter and ZoneDateTime::from as "
      + zonedDateTime);
}

Very few projects have the luxury of being a greenfield project that can start with Java 8. Therefore, it's helpful that there are classes that connect the pre-JDK 8 date/time classes with the new date/time classes introduced in JDK 8. One example of this is the ability of JDK 8's DateTimeFormatter to provide a descending instance of the pre-JDK 8 abstract Format class via the DateTimeFormatter.toFormat() method. This is demonstrated in the next code listing.

Accessing Pre-JDK 8 Format from JDK 8's DateTimeFormatter
/**
 * Demonstrate formatting ZonedDateTime via DateTimeFormatter,
 * but using implementation of Format.
 */
public void demonstrateDateTimeFormatAndFormatFormatting()
{
   final DateTimeFormatter formatter =
      DateTimeFormatter.ofPattern(dateTimeFormatPattern);
   final Format format = formatter.toFormat();
   final String nowString = format.format(now8);
   out.println(
        "ZonedDateTime " + now + " formatted with DateTimeFormatter/Format (and "
      + format.getClass().getCanonicalName() + ") and '"
      + dateTimeFormatPattern + "': " + nowString);
}

The Instant class is especially important when working with both pre-JDK 8 Date and Calendar classes in conjunction with the new date and time classes introduced with JDK 8. The reason Instant is so important is that java.util.Date has methods from(Instant) and toInstant() for getting a Date from an Instant and getting an Instant from a Date respectively. Because Instant is so important in migrating pre-Java 8 date/time handling to Java 8 baselines, the next code listing demonstrates formatting and parsing instances of Instant.

Formatting and Parsing Instances of Instant
/**
 * Demonstrate formatting and parsing an instance of Instant.
 */
public void demonstrateDateTimeFormatFormattingAndParsingInstant()
{
   // Instant instances don't have timezone information
   final Instant instant = now.toInstant();
   final DateTimeFormatter formatter =
      DateTimeFormatter.ofPattern(
         dateTimeFormatPattern).withZone(ZoneId.systemDefault());
   final String formattedInstance = formatter.format(instant);
   out.println(
        "Instant " + instant + " formatted with DateTimeFormatter and '"
      + dateTimeFormatPattern + "' to '" + formattedInstance + "'");
   final Instant instant2 =
      formatter.parse(formattedInstance, ZonedDateTime::from).toInstant();
      out.println(formattedInstance + " parsed back to " + instant2);
}

All of the above examples come from the sample class shown in the next code listing for completeness.

DateFormatDemo.java
package dustin.examples.numberformatdemo;

import static java.lang.System.out;

import java.text.DateFormat;
import java.text.Format;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.time.Instant;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Date;

/**
 * Demonstrates formatting dates as strings and parsing strings
 * into dates and times using pre-Java 8 (java.text.SimpleDateFormat)
 * and Java 8 (java.time.format.DateTimeFormatter) mechanisms.
 */
public class DateFormatDemo
{
   /** Pattern to use for String representation of Dates/Times. */
   private final String dateTimeFormatPattern = "yyyy/MM/dd HH:mm:ss z";

   /**
    * java.util.Date instance representing now that can
    * be formatted using SimpleDateFormat based on my
    * dateTimeFormatPattern field.
    */
   private final Date now = new Date();

   /**
    * java.time.ZonedDateTime instance representing now that can
    * be formatted using DateTimeFormatter based on my
    * dateTimeFormatPattern field.
    *
    * Note that ZonedDateTime needed to be used in this example
    * instead of java.time.LocalDateTime or java.time.OffsetDateTime
    * because there is zone information in the format provided by
    * my dateTimeFormatPattern field and attempting to have
    * DateTimeFormatter.format(TemporalAccessor) instantiated
    * with a format pattern that includes time zone details
    * will lead to DateTimeException for instances of
    * TemporalAccessor that do not have time zone information
    * (such as LocalDateTime and OffsetDateTime).
    */
   private final ZonedDateTime now8 = ZonedDateTime.now();


   /**
    * String that can be used by both SimpleDateFormat and
    * DateTimeFormatter to parse respective date/time instances
    * from this String.
    */
   private final String dateTimeString = "2014/09/03 13:59:50 MDT";

   /**
    * Demonstrate presenting java.util.Date as String matching
    * provided pattern via use of SimpleDateFormat.
    */
   public void demonstrateSimpleDateFormatFormatting()
   {
      final DateFormat format = new SimpleDateFormat(dateTimeFormatPattern);
      final String nowString = format.format(now);
      out.println(
           "Date '" + now + "' formatted with SimpleDateFormat and '"
         + dateTimeFormatPattern + "': " + nowString);
   }

   /**
    * Demonstrate parsing a java.util.Date from a String
    * via SimpleDateFormat.
    */
   public void demonstrateSimpleDateFormatParsing()
   {
      final DateFormat format = new SimpleDateFormat(dateTimeFormatPattern);
      try
      {
         final Date parsedDate = format.parse(dateTimeString);
         out.println("'" + dateTimeString + "' is parsed with SimpleDateFormat as " + parsedDate);
      }
      // DateFormat.parse(String) throws a checked exception
      catch (ParseException parseException)
      {
         out.println(
              "ERROR: Unable to parse date/time String '"
            + dateTimeString + "' with pattern '"
            + dateTimeFormatPattern + "'.");
      }
   }

   /**
    * Demonstrate presenting ZonedDateTime as a String matching
    * provided pattern via DateTimeFormatter and its
    * ofPattern(String) method.
    */
   public void demonstrateDateTimeFormatFormatting()
   {
      final DateTimeFormatter formatter =
         DateTimeFormatter.ofPattern(dateTimeFormatPattern);
      final String nowString = formatter.format(now8);
      out.println(
           now8 + " formatted with DateTimeFormatter and '"
         + dateTimeFormatPattern + "': " + nowString);
   }

   /**
    * Demonstrate parsing ZonedDateTime from provided String
    * via ZonedDateTime's parse(String, DateTimeFormatter) method.
    */
   public void demonstrateDateTimeFormatParsingTemporalStaticMethod()
   {
      final DateTimeFormatter formatter =
         DateTimeFormatter.ofPattern(dateTimeFormatPattern);
      final ZonedDateTime zonedDateTime = ZonedDateTime.parse(dateTimeString, formatter);
      out.println(
           "'" + dateTimeString
         + "' is parsed with DateTimeFormatter and ZonedDateTime.parse as "
         + zonedDateTime);
   }

   /**
    * Demonstrate parsing ZonedDateTime from String
    * via DateTimeFormatter.parse(String, TemporaryQuery)
    * with the Temple Query in this case being ZonedDateTime's
    * from(TemporalAccessor) used as a Java 8 method reference.
    */
   public void demonstrateDateTimeFormatParsingMethodReference()
   {
      final DateTimeFormatter formatter =
         DateTimeFormatter.ofPattern(dateTimeFormatPattern);
      final ZonedDateTime zonedDateTime = formatter.parse(dateTimeString, ZonedDateTime::from);
      out.println(
           "'" + dateTimeString
         + "' is parsed with DateTimeFormatter and ZoneDateTime::from as "
         + zonedDateTime);
   }

   /**
    * Demonstrate formatting ZonedDateTime via DateTimeFormatter,
    * but using implementation of Format.
    */
   public void demonstrateDateTimeFormatAndFormatFormatting()
   {
      final DateTimeFormatter formatter =
         DateTimeFormatter.ofPattern(dateTimeFormatPattern);
      final Format format = formatter.toFormat();
      final String nowString = format.format(now8);
      out.println(
           "ZonedDateTime " + now + " formatted with DateTimeFormatter/Format (and "
         + format.getClass().getCanonicalName() + ") and '"
         + dateTimeFormatPattern + "': " + nowString);
   }

   /**
    * Demonstrate formatting and parsing an instance of Instant.
    */
   public void demonstrateDateTimeFormatFormattingAndParsingInstant()
   {
      // Instant instances don't have timezone information
      final Instant instant = now.toInstant();
      final DateTimeFormatter formatter =
         DateTimeFormatter.ofPattern(
            dateTimeFormatPattern).withZone(ZoneId.systemDefault());
      final String formattedInstance = formatter.format(instant);
      out.println(
           "Instant " + instant + " formatted with DateTimeFormatter and '"
         + dateTimeFormatPattern + "' to '" + formattedInstance + "'");
      final Instant instant2 =
         formatter.parse(formattedInstance, ZonedDateTime::from).toInstant();
      out.println(formattedInstance + " parsed back to " + instant2);
   }

   /**
    * Demonstrate java.text.SimpleDateFormat and
    * java.time.format.DateTimeFormatter.
    *
    * @param arguments Command-line arguments; none anticipated.
    */
   public static void main(final String[] arguments)
   {
      final DateFormatDemo demo = new DateFormatDemo();
      out.print("\n1: ");
      demo.demonstrateSimpleDateFormatFormatting();
      out.print("\n2: ");
      demo.demonstrateSimpleDateFormatParsing();
      out.print("\n3: ");
      demo.demonstrateDateTimeFormatFormatting();
      out.print("\n4: ");
      demo.demonstrateDateTimeFormatParsingTemporalStaticMethod();
      out.print("\n5: ");
      demo.demonstrateDateTimeFormatParsingMethodReference();
      out.print("\n6: ");
      demo.demonstrateDateTimeFormatAndFormatFormatting();
      out.print("\n7: ");
      demo.demonstrateDateTimeFormatFormattingAndParsingInstant();
   }
}

The output from running the above demonstration is shown in the next screen snapshot.

Conclusion

The JDK 8 date/time classes and related formatting and parsing classes are much more straightforward to use than their pre-JDK 8 counterparts. This post has attempted to demonstrate how to apply these new classes and to take advantage of some of their benefits.

Saturday, August 30, 2014

Big Java News in Late Summer 2014

As is typical when JavaOne is imminent, there has been much big news in the Java community recently. This post briefly references three of these items (Java SE 8 updates, Java SE 9, and Java EE 8) and a "bonus" reference to a post I found to be one of the clearer ones I have seen on classpath/classloader issues.

String Deduplication in Oracle Java 8 JVM

In String Deduplication – A new feature in Java 8 Update 20, Fabian Lange introduces String Deduplication for the G1 Garbage Collector using the JVM option -XX:+UseStringDeduplication that was introduced with JDK 8 Update 20. The tools page for the Java launcher has been updated to mention the JVM options -XX:+UseStringDeduplication, -XX:+PrintStringDeduplicationStatistics, and -XX:StringDeduplicationAgeThreshold. More details on JDK 8 Update 20 are available in the blog post Release: Oracle Java Development Kit 8, Update 20. The Lange post has also sparked discussion on this and related JVM options on the Java subreddit.

Java 9 Features

Java 9 has been the hot topic of discussion in the Java community since the OpenJDK JDK 9 Project was announced. Long-awaited Java modularity (Project Jigsaw, which was booted from JDK 8) is probably the largest new feature anticipated for Java 9. Paul Krill writes in Why developers should get excited about Java 9 that "Jigsaw isn't the only new addition slated for Java 9. Support for the popular JSON (JavaScript Object Notation) data interchange format is key feature as well, along with process API, code cache, and locking improvements. The six JEPs currently proposed on that OpenJDK JDK 9 page are 102 (Process API Updates), 143 (Improve Contended Locking), 197 (Segmented Code Cache), 198 (Light-Weight JSON API), 199 (Smart Java Compilation, Phase 2), and 201 (Modular Source Code).

In the blog post Java 9 is coming with money api, otaviojava introduces JSR 354 ("JSR 354: Money and Currency API"), describes why it is needed, covers how it might be implemented, and concludes, "this API is expected to [be in] Java 9."

Java EE 8

Reza Rahman's post Java EE 8 Takes Off! talks about JSR 366 (Java EE 8 Specification) being kicked off. This post lists some of the anticipated high-level content for Java EE along with links to related JSRs.

Demystifying the Java Classpath

Java classpath issues are definitely one of the more difficult challenges that Java developers can face. The post Jar Hell made Easy - Demystifying the classpath with jHades provides a nice overview of some of the most common issues related to classpath and classloaders with concise and simple explanations of why these occur. I have not used jHades, but the quality of this post has definitely sparked my interest in that tool.

Conclusion

"Java" (SE, EE, JVM, etc.) keeps advancing and bringing us new language features, libraries, and tools. This post has referenced posts that highlight recent developments in JDK 8, JDK 9, and Java EE 8.

Friday, August 22, 2014

Java Numeric Formatting

I can think of numerous times when I have seen others write unnecessary Java code and I have written unnecessary Java code because of lack of awareness of a JDK class that already provides the desired functionality. One example of this is the writing of time-related constants using hard-coded values such as 60, 24, 1440, and 86400 when TimeUnit provides a better, standardized approach. In this post, I look at another example of a class that provides the functionality I have seen developers often implement on their one: NumberFormat.

The NumberFormat class is part of the java.text package, which also includes the frequently used DateFormat and SimpleDateFormat classes. NumberFormat is an abstract class (no public constructor) and instances of its descendants are obtained via overloaded static methods with names such as getInstance(), getCurrencyInstance(), and getPercentInstance().

Currency

The next code listing demonstrates calling NumberFormat.getCurrencyInstance(Locale) to get an instance of NumberFormat that presents numbers in a currency-friendly format.

Demonstrating NumberFormat's Currency Support
/**
 * Demonstrate use of a Currency Instance of NumberFormat.
 */
public void demonstrateCurrency()
{
   writeHeaderToStandardOutput("Currency NumberFormat Examples");
   final NumberFormat currencyFormat = NumberFormat.getCurrencyInstance(Locale.US);
   out.println("15.5      -> " + currencyFormat.format(15.5));
   out.println("15.54     -> " + currencyFormat.format(15.54));
   out.println("15.345    -> " + currencyFormat.format(15.345));  // rounds to two decimal places
   printCurrencyDetails(currencyFormat.getCurrency());
}

/**
 * Print out details of provided instance of Currency.
 *
 * @param currency Instance of Currency from which details
 *    will be written to standard output.
 */
public void printCurrencyDetails(final Currency currency)
{
   out.println("Concurrency: " + currency);
   out.println("\tISO 4217 Currency Code:           " + currency.getCurrencyCode());
   out.println("\tISO 4217 Numeric Code:            " + currency.getNumericCode());
   out.println("\tCurrency Display Name:            " + currency.getDisplayName(Locale.US));
   out.println("\tCurrency Symbol:                  " + currency.getSymbol(Locale.US));
   out.println("\tCurrency Default Fraction Digits: " + currency.getDefaultFractionDigits());
}

When the above code is executed, the results are as shown next:

==================================================================================
= Currency NumberFormat Examples
==================================================================================
15.5      -> $15.50
15.54     -> $15.54
15.345    -> $15.35
Concurrency: USD
 ISO 4217 Currency Code:           USD
 ISO 4217 Numeric Code:            840
 Currency Display Name:            US Dollar
 Currency Symbol:                  $
 Currency Default Fraction Digits: 2

The above code and associated output demonstrate that the NumberFormat instance used for currency (actually a DecimalFormat), automatically applies the appropriate number of digits and appropriate currency symbol based on the locale.

Percentages

The next code listings and associated output demonstrate use of NumberFormat to present numbers in percentage-friendly format.

Demonstrating NumberFormat's Percent Format
/**
 * Demonstrate use of a Percent Instance of NumberFormat.
 */
public void demonstratePercentage()
{
   writeHeaderToStandardOutput("Percentage NumberFormat Examples");
   final NumberFormat percentageFormat = NumberFormat.getPercentInstance(Locale.US);
   out.println("Instance of: " + percentageFormat.getClass().getCanonicalName());
   out.println("1        -> " + percentageFormat.format(1));
   // will be 0 because truncated to Integer by Integer division
   out.println("75/100   -> " + percentageFormat.format(75/100));
   out.println(".75      -> " + percentageFormat.format(.75));
   out.println("75.0/100 -> " + percentageFormat.format(75.0/100));
   // will be 0 because truncated to Integer by Integer division
   out.println("83/93    -> " + percentageFormat.format((83/93)));
   out.println("93/83    -> " + percentageFormat.format(93/83));
   out.println(".5       -> " + percentageFormat.format(.5));
   out.println(".912     -> " + percentageFormat.format(.912));
   out.println("---- Setting Minimum Fraction Digits to 1:");
   percentageFormat.setMinimumFractionDigits(1);
   out.println("1        -> " + percentageFormat.format(1));
   out.println(".75      -> " + percentageFormat.format(.75));
   out.println("75.0/100 -> " + percentageFormat.format(75.0/100));
   out.println(".912     -> " + percentageFormat.format(.912));
}
==================================================================================
= Percentage NumberFormat Examples
==================================================================================
1        -> 100%
75/100   -> 0%
.75      -> 75%
75.0/100 -> 75%
83/93    -> 0%
93/83    -> 100%
.5       -> 50%
.912     -> 91%
---- Setting Minimum Fraction Digits to 1:
1        -> 100.0%
.75      -> 75.0%
75.0/100 -> 75.0%
.912     -> 91.2%

The code and output of the percent NumberFormat usage demonstrate that by default the instance of NumberFormat (actually a DecimalFormat in this case) returned by NumberFormat.getPercentInstance(Locale) method has no fractional digits, multiplies the provided number by 100 (assumes that it is the decimal equivalent of a percentage when provided), and adds a percentage sign (%).

Integers

The small amount of code shown next and its associated output demonstrate use of NumberFormat to present numbers in integral format.

Demonstrating NumberFormat's Integer Format
/**
 * Demonstrate use of an Integer Instance of NumberFormat.
 */
public void demonstrateInteger()
{
   writeHeaderToStandardOutput("Integer NumberFormat Examples");
   final NumberFormat integerFormat = NumberFormat.getIntegerInstance(Locale.US);
   out.println("7.65   -> " + integerFormat.format(7.65));
   out.println("7.5    -> " + integerFormat.format(7.5));
   out.println("7.49   -> " + integerFormat.format(7.49));
   out.println("-23.23 -> " + integerFormat.format(-23.23));
}
==================================================================================
= Integer NumberFormat Examples
==================================================================================
7.65   -> 8
7.5    -> 8
7.49   -> 7
-23.23 -> -23

As demonstrated in the above code and associated output, the NumberFormat method getIntegerInstance(Locale) returns an instance that presents provided numerals as integers.

Fixed Digits

The next code listing and associated output demonstrate using NumberFormat to print fixed-point representation of floating-point numbers. In other words, this use of NumberFormat allows one to represent a number with an exactly prescribed number of digits to the left of the decimal point ("integer" digits) and to the right of the decimal point ("fraction" digits).

Demonstrating NumberFormat for Fixed-Point Numbers
/**
 * Demonstrate generic NumberFormat instance with rounding mode,
 * maximum fraction digits, and minimum integer digits specified.
 */
public void demonstrateNumberFormat()
{
   writeHeaderToStandardOutput("NumberFormat Fixed-Point Examples");
   final NumberFormat numberFormat = NumberFormat.getNumberInstance();
   numberFormat.setRoundingMode(RoundingMode.HALF_UP);
   numberFormat.setMaximumFractionDigits(2);
   numberFormat.setMinimumIntegerDigits(1);
   out.println("234.234567 --> " + numberFormat.format(234.234567));
   out.println("1          --> " + numberFormat.format(1));
   out.println(".234567    --> " + numberFormat.format(.234567));
   out.println(".349       --> " + numberFormat.format(.349));
   out.println(".3499      --> " + numberFormat.format(.3499));
   out.println("0.9999     --> " + numberFormat.format(0.9999));
}
==================================================================================
= NumberFormat Fixed-Point Examples
==================================================================================
234.234567 --> 234.23
1          --> 1
.234567    --> 0.23
.349       --> 0.34
.3499      --> 0.35
0.9999     --> 1

The above code and associated output demonstrate the fine-grain control of the minimum number of "integer" digits to represent to the left of the decimal place (at least one, so zero shows up when applicable) and the maximum number of "fraction" digits to the right of the decimal point. Although not shown, the maximum number of integer digits and minimum number of fraction digits can also be specified.

Conclusion

I have used this post to look at how NumberFormat can be used to present numbers in different ways (currency, percentage, integer, fixed number of decimal points, etc.) and often means no or reduced code need be written to massage numbers into these formats. When I first began writing this post, I envisioned including examples and discussion on the direct descendants of NumberFormat (DecimalFormat and ChoiceFormat), but have decided this post is already sufficiently lengthy. I may write about these descendants of NumberFormat in future blog posts.

Tuesday, August 19, 2014

jinfo: Command-line Peeking at JVM Runtime Configuration

In several recent blogs (in my reviews of the books Java EE 7 Performance Tuning and Optimization and WildFly Performance Tuning in particular), I have referenced my own past blog posts on certain Oracle JDK command-line tools. I was aghast to discover that I had never exclusively addressed the nifty jinfo tool and this post sets to rectify that troubling situation. I suspect that the reasons I chose not to write about jinfo previously include limitations related to jinfo discussed in my post VisualVM: jinfo and So Much More.

In the Java SE 8 version of jinfo running on my machine, the primary limitation of jinfo on Windows that I discussed in the post Acquiring JVM Runtime Information has been addressed. In particular, I noted in that post that the -flags option was not supported on Windows version of jinfo at that time. As the next screen snapshot proves, that is no longer the case (note the use of jps to acquire the Java process ID to instruct jinfo to query).

As the above screen snapshot demonstrates, the jinfo -flags command and option show the flags the explicitly specified JVM options of the Java process being monitored.

If I want to find out about other JVM flags that are in effect implicitly (automatically), I can run java -XX:+PrintFlagsFinal to see all default JVM options. I can then query for any one of these against a running JVM process to find out what that particular JVM is using (same default or overridden different value). The next screen snapshot demonstrates how a small portion of the output provided from running java -XX:+PrintFlagsFinal.

Let's suppose I notice a flag called PrintHeapAtGC in the above output and want to know if it's set in my particular Java application (-XX:+PrintHeapAtGC means it's set and -XX:-PrintHeapAtGC means it's not set). I can have jinfo tell me what its setting is (note my choice to use jcmd instead of jps in this case to determine the Java process ID):

Because of the subtraction sign (-) instead of an addition sign (+) after the colon and before "PrintHeapAtGC", we know this is turned off for the Java process with the specified ID. It turns out that jinfo does more than let us look; it also let's us touch. The next screen snapshot shows changing this option using jinfo.

As the previous screen snapshot indicates, I can turn off and on the boolean-style JVM options by simply using the same command to view the flag's setting but preceding the flag's name with the addition sign (+) to turn it on or with the substraction sign (-) to turn it off. In the example just shown, I turned off the PrintGCDateStamps, turned it back on again, and monitored its setting between those changes. Not all JVM options are boolean conditions. In those cases, their new values are assigned to them by concatenating the equals sign (=) and new value after the flag name. It's also important to note that the target JVM (the one you're trying to peek at and touch with jinfo will not allow you to change all its JVM option settings). In such cases, you'll likely see a stack trace with message "Command failed in target VM."

In addition to displaying a currently running JVM's options and allowing the changing of some of these, jinfo also allows one to see system properties used by that JVM as name/value pairs. This is demonstrated in the next screen snapshot with a small fraction of the output shown.

Perhaps the easiest way to run jinfo is to simply provide no arguments other than the PID of the Java process in question and have both JVM options (non-default and command-line) and system properties displayed. Running jinfo -help provides brief usage details. Other important details are found in the Oracle documentation on the jinfo tool. These details includes the common (when it comes to these tools) reminder that this tool is "experimental and unsupported" and "might not be available in future releases of the JDK." We are also warned that jinfo on Windows requires availability of dbgeng.dll or installed Debugging Tools For Windows.

Although I have referenced the handy jinfo command line tool previously in posts VisualVM: jinfo and So Much More and Acquiring JVM Runtime Information, it is a handy enough tool to justify a post of its very own. As a command-line tool, it enjoys benefits commonly associated with command-line tools such as being relatively lightweight, working well with scripts, and working in headless environments.

Friday, August 15, 2014

Autoboxing, Unboxing, and NoSuchMethodError

J2SE 5 introduced numerous features to the Java programming language. One of these features is autoboxing and unboxing, a feature that I use almost daily without even thinking about it. It is often convenient (especially when used with collections), but every once in a while it leads to some nasty surprises, "weirdness," and "madness." In this blog post, I look at a rare (but interesting to me) case of NoSuchMethodError resulting from mixing classes compiled with Java versions before autoboxing/unboxing with classes compiled with Java versions that include autoboxing/unboxing.

The next code listing shows a simple Sum class that could have been written before J2SE 5. It has overloaded "add" methods that accept different primitive numeric data types and each instance of Sum> simply adds all types of numbers provided to it via any of its overloaded "add" methods.

Sum.java (pre-J2SE 5 Version)
import java.util.ArrayList;

public class Sum
{
   private double sum = 0;

   public void add(short newShort)
   {
      sum += newShort;
   }

   public void add(int newInteger)
   {
      sum += newInteger;
   }

   public void add(long newLong)
   {
      sum += newLong;
   }

   public void add(float newFloat)
   {
      sum += newFloat;
   }

   public void add(double newDouble)
   {
      sum += newDouble;
   }

   public String toString()
   {
      return String.valueOf(sum);
   }
}

Before unboxing was available, any clients of the above Sum class would need to provide primitives to these "add" methods or, if they had reference equivalents of the primitives, would need to convert the references to their primitive counterparts before calling one of the "add" methods. The onus was on the client code to do this conversion from reference type to corresponding primitive type before calling these methods. Examples of how this might be accomplished are shown in the next code listing.

No Unboxing: Client Converting References to Primitives
private static String sumReferences(
   final Long longValue, final Integer intValue, final Short shortValue)
{
   final Sum sum = new Sum();
   if (longValue != null)
   {
      sum.add(longValue.longValue());
   }
   if (intValue != null)
   {
      sum.add(intValue.intValue());
   }
   if (shortValue != null)
   {
      sum.add(shortValue.shortValue());
   }
   return sum.toString();
}

J2SE 5's autoboxing and unboxing feature was intended to address this extraneous effort required in a case like this. With unboxing, client code could call the above "add" methods with references types corresponding to the expected primitive types and the references would be automatically "unboxed" to the primitive form so that the appropriate "add" methods could be invoked. Section 5.1.8 ("Unboxing Conversion") of The Java Language Specification explains which primitives the supplied numeric reference types are converted to in unboxing and Section 5.1.7 ("Boxing Conversion") of that same specification lists the references types that are autoboxed from each primitive in autoboxing.

In this example, unboxing reduced effort on the client's part in terms of converting reference types to their corresponding primitive counterparts before calling Sum's "add" methods, but it did not completely free the client from needing to process the number values before providing them. Because reference types can be null, it is possible for a client to provide a null reference to one of Sum's "add" methods and, when Java attempts to automatically unbox that null to its corresponding primitive, a NullPointerException is thrown. The next code listing adapts that from above to indicate how the conversion of reference to primitive is no longer necessary on the client side but checking for null is still necessary to avoid the NullPointerException.

Unboxing Automatically Coverts Reference to Primitive: Still Must Check for Null
private static String sumReferences(
   final Long longValue, final Integer intValue, final Short shortValue)
{
   final Sum sum = new Sum();
   if (longValue != null)
   {
      sum.add(longValue);
   }
   if (intValue != null)
   {
      sum.add(intValue);
   }
   if (shortValue != null)
   {
      sum.add(shortValue);
   }
   return sum.toString();
}

Requiring client code to check their references for null before calling the "add" methods on Sum may be something we want to avoid when designing our API. One way to remove that need is to change the "add" methods to explicitly accept the reference types rather than the primitive types. Then, the Sum class could check for null before explicitly or implicitly (unboxing) dereferencing it. The revised Sum class with this changed and more client-friendly API is shown next.

Sum Class with "add" Methods Expecting References Rather than Primitives
import java.util.ArrayList;

public class Sum
{
   private double sum = 0;

   public void add(Short newShort)
   {
      if (newShort != null)
      {
         sum += newShort;
      }
   }

   public void add(Integer newInteger)
   {
      if (newInteger != null)
      {
         sum += newInteger;
      }
   }

   public void add(Long newLong)
   {
      if (newLong != null)
      {
         sum += newLong;
      }
   }

   public void add(Float newFloat)
   {
      if (newFloat != null)
      {
         sum += newFloat;
      }
   }

   public void add(Double newDouble)
   {
      if (newDouble != null)
      {
         sum += newDouble;
      }
   }

   public String toString()
   {
      return String.valueOf(sum);
   }
}

The revised Sum class is more client-friendly because it allows the client to pass a reference to any of its "add" methods without concern for whether the passed-in reference is null or not. However, the change of the Sum class's API like this can lead to NoSuchMethodErrors if either class involved (the client class or one of the versions of the Sum class) is compiled with different versions of Java. In particular, if the client code uses primitives and is compiled with JDK 1.4 or earlier and the Sum class is the latest version shown (expecting references instead of primitives) and is compiled with J2SE 5 or later, a NoSuchMethodError like the following will be encountered (the "S" indicates it was the "add" method expecting a primitive short and the "V" indicates that method returned void).

Exception in thread "main" java.lang.NoSuchMethodError: Sum.add(S)V
 at Main.main(Main.java:9)

On the other hand, if the client is compiled with J2SE 5 or later and with primitive values being supplied to Sum as in the first example (pre-unboxing) and the Sum class is compiled in JDK 1.4 or earlier with "add" methods expecting primitives, a different version of the NoSuchMethodError is encountered. Note that the Short reference is cited here.

Exception in thread "main" java.lang.NoSuchMethodError: Sum.add(Ljava/lang/Short;)V
 at Main.main(Main.java:9)

There are several observations and reminders to Java developers that come from this.

  • Classpaths are important:
    • Java .class files compiled with the same version of Java (same -source and -target) would have avoided the particular problem in this post.
    • Classpaths should be as lean as possible to reduce/avoid possibility of getting stray "old" class definitions.
    • Build "clean" targets and other build operations should be sure to clean past artifacts thoroughly and builds should rebuild all necessary application classes.
  • Autoboxing and Unboxing are well-intentioned and often highly convenient, but can lead to surprising issues if not kept in mind to some degree. In this post, the need to still check for null (or know that the object is non-null) is necessary remains in situations when implicit dereferencing will take place as a result of unboxing.
  • It's a matter of API style taste whether to allow clients to pass nulls and have the serving class check for null on their behalf. In an industrial application, I would have stated whether null was allowed or not for each "add" method parameter with @param in each method's Javadoc comment. In other situations, one might want to leave it the responsibility of the caller to ensure any passed-in reference is non-null and would be content throwing a NullPointerException if the caller did not obey that contract (which should also be specified in the method's Javadoc).
  • Although we typically see NoSuchMethodError when a method is completely removed or when we access an old class before that method was available or when a method's API has changed in terms of types or number of types. In a day when Java autoboxing and unboxing are largely taken for granted, it can be easy to think that changing a method from taking a primitive to taking the corresponding reference type won't affect anything, but even that change can lead to an exception if not all classes involved are built on a version of Java supporting autoboxing and unboxing.
  • One way to determine the version of Java against which a particular .class file was compiled is to use javap -verbose and to look in the javap output for the "major version:". In the classes I used in my examples in this post (compiled against JDK 1.4 and Java SE 8), the "major version" entries were 48 and 52 respectively (the General Layout section of the Wikipedia entry on Java class file lists the major versions).

Fortunately, the issue demonstrated with examples and text in this post is not that common thanks to builds typically cleaning all artifacts and rebuilding code on a relatively continuous basis. However, there are cases where this could occur and one of the most likely such situations is when using an old JAR file accidentally because it lies in wait on the runtime classpath.

Saturday, August 9, 2014

Book Review: Java EE 7 Performance Tuning and Optimization

Packt Publishing recently published 's Osama Oransa's Java EE 7 Performance Tuning and Optimization. This post is my review of this book of twelve chapters and approximately 430 pages of substantive content.

Preface

The Preface of Java EE 7 Performance Tuning and Optimization provides brief summaries of all 12 chapters. The Preface also lists the main tools used throughout the book: JDK 7 Update 45, NetBeans 7.4, GlassFish 4.0, MySQL 5.5, Eclipse with Eclipse Test   Performance Tools Platform Project, Apache JMeter 2.10, and JProfiler 8. The Preface states that Java EE 7 Performance Tuning and Optimization is a "strong entry point for the persons without any performance tuning experience" and that the book is for "experienced Java developers, architects, team leaders, consultants, support engineers, and all people working in the performance tuning in the Java applications, and particularly in Java Enterprise applications."

Chapter 1: Getting Started with Performance Tuning

The first chapter of Java EE 7 Performance Tuning and Optimization is a nearly 30 pages long high-level discussion on "the art of performance tuning." The chapter discusses "Java EE performance tuning art" before looking at identifying performance issues and classifying these performance issues by waterfall development lifecycle discovery phase (when the issue was identified) and by root phase (when the issue was introduced).

Chapter 1 spends several pages discussing high-level "soft skills" one needs as a leader of a performance tuning team and the very-high-level approach used to narrow down causes of performance issues. There are several other high-level discussions covered in this chapter such as descriptions of the different common architectures and the different application layers to be performance tuned.

Java EE 7 Performance Tuning and Optimization's initial chapter has a few things for experienced Java developers, but is really better suited for those entirely new to Java performance tuning considerations. It might also be better suited to the readers who are team leaders and consultants than those readers who are developers. The most beginning Java developers might benefit from the brief, very high level descriptions of application layers, but most of this information should be something most developers already know.

Chapter 2: Understanding Java Fundamentals

Chapter 2 of Java EE 7 Performance Tuning and Optimization provides lower-level details than the first chapter. However, much of this chapter is identical to online resources upon which it is based. One of the first sentences in Chapter 2 states, "Most of the content of this chapter adheres to Oracle Java documentations as the source of Java specifications and features." I didn't know what this meant when I first read it, but I believe it is the author's way of saying that much of the material in the chapter comes directly from Oracle's Java-related documentation.

The second chapter begins with a section on new Java EE 7 features. Each new Java EE 7 feature is briefly described, illustrated with pseudo code, and has its underlying specification referenced with a link. I agree with the author's recommendation that anyone with awareness of the new Java EE 7 features should skip this section.

Chapter 2's section "Understanding memory structure in the JVM" is good, but much of it is word-for-word the same as the text provided in Chapter 2 ("The Structure of the Java Virtual Machine") of the Java Virtual Machine specification and in Memory Management in the HotSpot JVM. In fact, many of the diagrams are the same or nearly the same except that the original sources have colors included in the diagrams, which are grayscale in the PDF version of Java EE 7 Performance Tuning and Optimization that I reviewed.

I also liked the "Understanding concurrency in Java" section of Java EE 7 Performance Tuning and Optimization, but it seemed to be much the same content (including several passages that are word-for-word the same) as the "Concurrency Basics" section of the Java EE 7 Tutorial and Chapter 17 ("Threads and Locks") of the Java Virtual Machine specification.

Chapter 2 also includes brief discussion of JSP/servlet application scopes and more focus on Java EE 7 concurrency features. There is some good information in this chapter, but the downsides are that some of its content seems a little out of order and much of the content relies heavily, even directly, on Oracle's documentation.

Chapter 3: Getting Familiar with Performance Testing

The third chapter of Java EE 7 Performance Tuning and Optimization opens with a definition of performance testing and contrasts performance with the more common types of testing done to test functionality. The chapter then lists and briefly discusses each "important aspect of the performance testing process." This section contains useful definitions and terminology for those new to performance testing and performance optimization and is largely at a higher or more general level than Java EE. Someone wanting to know the different between load testing and stress testing, for example, would find this section very useful.

The third chapter provides a list of some tools that can used in performance testing, but its focus is on JMeter, which it devotes numerous pages to. The section on JMeter could have likely been a chapter of its own and provides quite a bit of detail about setting up and applying JMeter in performance testing of a web service, of a web application, and of JDBC code.

Chapter 4: Monitoring Java Applications

Chapter 4 of Java EE 7 Performance Tuning and Optimization provides a survey of some of the monitoring tools available for monitoring an application at different layers. The section on monitoring operating systems discusses and provides illustrations of Windows-based monitoring tools such as Windows 7 Task Manager, Resource Monitor, netstat, and Typeperf. The portion covering Linux monitoring tools is shorter (about half of a page), but lists tools such as ps, pgrep, pstree, top, vmstat, iostat, sar, and tcpdump, and refers the reader to these respective tools' man pages for additional details. The section on operating system monitoring concludes with an example of applying Windows and Linux tools to diagnose heavily contended CPUs.

Chapter 4's coverage of "The Java monitoring tools" talks about "the JDK monitoring tools" that are specifically delivered with the Oracle/Sun JDK such as JConsole, VisualVM, and Java Mission Control. The section on "Java monitoring tools" also mentions IDEs (and monitoring plug-ins for the IDEs) and application server-specific monitoring tools.

The JVM Tool Interface (JVM TI) is introduced in Chapter 4 as a replacement for the deprecated JVMPI/JVMDI. This section is essentially a condensed version of the Introduction portion of the JVM Tool Interface 1.2 specification.

The Chapter 4 section "Understanding the different JDK tools" discusses tools provided with the Oracle/Sun JDK for monitoring (jps, jstat, jstatd), profiling (JConsole, VisualVM, Java Mission Control), and troubleshooting (jhat, jstack, jinfo, jmap, jsadebugd). The overviews of each of these tools and their respective syntax seem to be borrowed directly from the Oracle tools pages (online and similar to man pages), but the section does include some specifics about using these tools in the book's examples at the end of each tool's discussion. Even the categorization of the tools is the same as the main Oracle tools page.

Chapter 4 barely scratches the surface of using JConsole, but provides more detailed and nicely illustrated examples of using VisualVM, Oracle Java Mission Control, and the NetBeans Profiler. The chapter introduces Eclipse plug-ins and provides some focus on JVM Monitor and Test and Performance Tools Platform (Eclipse Memory Analyzer Tool is covered in a later chapter). Chapter 4 concludes by introducing and significantly covering JProfiler.

Chapter 5: Recognizing Common Performance Issues

Java EE 7 Performance Tuning and Optimization's fifth chapter outlines categories of performance issues, briefly describes each category, and then looks in more detail at a subset of the individual categorized performance issues. The sections on "Performance Symptoms" of each performance issue are probably the most interesting to me. For developers new to performance tuning, it could be helpful to have "performance symptoms" collected and associated with particular likely causes of the symptoms. The discussion of performance issues also hits on some commonly accepted practices to adopt and some commonly accepted practices to avoid.

The second major part of Chapter 5 deals with "client-side performance issues." In this case, "client-side" appears to be synonymous with (assumes) a web application and web browser. This portion of Chapter 5 introduces and illustrates with screen snapshots several browser-based tools such as Chrome Developer Tools, Chrome plugin Speed Tracer, Internet Explorer Developer Tools, and FireFox Developer Tools.

Chapter 6: CPU Time Profiling

Chapter 6 of Java EE 7 Performance Tuning and Optimization is the first of three chapters to focus on use of Java profilers. This chapter focuses on profiling CPU usage and the subsequent two chapters focus on profiling threading and memory usage. This sixth chapter introduces three Java profiling tools that feature CPU profiling capabilities: NetBeans Profiler, Java Mission Control (JMC), and JProfiler. These sections on using these three tools provide sharp screen snapshots (full color in electronic version) to demonstrate all three graphically-oriented tools in action.

After introducing the three profiling tools in action profiling CPU usage and method execution times, Chapter 6 moves onto using the results provided by those tools to identify and categorize potential causes of the most significant time-consuming operations. This includes high level discussion of how to resolve many of these types of issues and a little more detailed coverage of addressing issues related to inappropriate algorithms.

Chapter 7: Thread Profiling

Chapter 7 demonstrates using the same three Java profiling tools demonstrated in Chapter 6 (NetBeans Profiler, Java Mission Control, JProfiler), but this time used for profiling threads. The demonstration of these three profiling tools used for profiling thread usage is followed by some discussion on what the results might indicate.

The seventh chapter of Java EE 7 Performance Tuning and Optimization also looks at "options from the operating system to produce a thread dump for the executing Java process" (CTRL+Break and sending SIGQUIT signal [kill -3]). There is also a section on using the Oracle JDK tools jstack, JVisualVM, and Java Mission Control to dump heaps. The section on "Understanding the thread dump structure" provides a highly useful overview of the data included in a thread dump and how to use it. The chapter also introduces the Thread Dump Analyzer.

Chapter 7 includes very high level discussion of potential threading performance issues. It also points out that -XX:+PrintFlagsFinal and jinfo can be used with the Oracle JDK to determine which JVM flags are currently set. High level discussion on "detecting the root cause of a hung application" is also in Chapter 7. There is a very brief overview of dealing with thread-related issues at the end of the chapter.

Chapter 8: Memory Profiling

The eighth chapter of Java EE 7 Performance Tuning and Optimization is the concluding chapter of the three-chapter series on applying Java profiling tools and this chapter specifically covers memory profiling. The chapter illustrates using NetBeans Profiler and JProfiler to perform memory profiling. This chapter also describes what is contained in a garbage collection log before briefly demonstrating use of JVisualVM's plugin called Visual GC (the plugin version of the Oracle visualgc command-line tool that is downloaded separately from the JDK command-line tools) to see memory and garbage collection activities in the different heap memory pools.

Chapter 8 provides an overview of heap dumps and their content. It explains approaches that can be used to obtain a heap dump such as use of JVM options (-XX:+HeapDumpOnOutOfMemoryError and -XX:HeapDumpPath), Oracle JDK tools (jmap, JVisualVM, and Java Mission Control), and with Java profiler tools (NetBeans, Eclipse Memory Analyzer Tool, and JProfiler). This chapter also describes Object Query Language (OQL) and provides examples of OQL similar to those provided by the OQL Help that can be seen at http://localhost:7000/oqlhelp/ when running jhat against a heap dump on the local machine.

Chapter 8 wraps up with a nice high-level overview of the types of things that commonly lead to memory issues and OutOfMemoryError.

Chapter 9: Tuning an Application's Environment

Chapter 9 of Java EE 7 Performance Tuning and Optimization opens with a discussion of JVM tuning using the standard and non-standard Oracle JVM options. This includes high-level discussion of using JVM options related to garbage collection and different garbage collectors. There is a similar section on JRockit VM tuning options.

After covering JVM tuning, Chapter 9 moves onto application server tuning with general application server tuning discussion illustrated with examples specific to GlassFish and WebLogic application servers. The section "Tuning the Oracle GlassFish application server" states that it goes "through the configuration as specified in the GlassFish 4 performance tuning guide." In fact, this section is very close to a subset of what is in the GlassFish 4 Performance Tuning Guide but does offer a small number of screen snapshots not available in the guide. The chapter also includes brief, high-level overviews of "tuning the Oracle Weblogic application server" and "tuning web servers (HTTP servers)" (including Apache HTTP Server and Oracle Web Server).

"Tuning the operating system and hardware" rounds out Chapter 9 and is a very brief discussion of issues to consider and strategies for tuning operating system and hardware.

Chapter 10: Designing High-performance Enterprise Applications

The tenth chapter of Java EE 7 Performance Tuning and Optimization begins with the assertion, "Design is our key to a good performing enterprise application" and then mentions the objective of the chapter to "cover some of the design areas that we should consider while designing our enterprise applications." The first section in this chapter provides high level discussion on things to think about when designing and how there are trade-offs between different design objectives. This high level discussion includes comparing "potential performance impacts" of various considerations (such as security and framework choices) on performance.

Chapter 10 devotes a couple pages to high-level discussion of "performance anti-patterns." This is followed by sections on Service-Oriented Architecture (SOA) performance aspects and Resource-Oriented Architecture (ROA) performance aspects. The chapter concludes with coverage of data caching performance aspects.

Chapter 11: Performance Tuning Tips

Java EE 7 Performance Tuning and Optimization's Chapter 11 begins with a discussion on performance considerations during Agile software development. This discussion addresses when in the agile development process performance considerations should be made. It also introduces Test-Driven Development (TDD) and code reviews (manual and automated) as software approaches to improve performance tuning ability. The section on automated code reviews lists several code analysis tools as standalone tools and as NetBeans tools.plugins (such as PMD).

The "Java EE Performance Tuning Tips" section of Chapter 11 provides very brief, high-level descriptions of Java EE performance tuning tips categorized by Java EE component (web services, Enterprise JavaBeans, servlets/JavaServer Pages, JavaServer Faces, and Java Persistence API). The "Java Performance Tuning Tips" section provides a high-level overview of some common issues in Java SE code that lead to performance issues and mentions tips (coding and JVM options) to reduce or avoid those issues. There is also discussion of logging and exception handling best practices that can help performance.

Chapter 11's section "Using the javap tool to understand micro-optimizations" provides an overview of one of my favorite Oracle JDK command-line tools, javap, to determine how changes are impacting the byte code. The "Database performance tuning tips" section of Chapter 11 discusses a few database performance tips such as use of SQL optimizer hints and batch processing. Chapter 11 ends with a section on "Client-side optimization" that addresses "presentation layer performance." The author introduces and briefly illustrates PageSpeed Insights as part of this client-size optimization discussion.

Chapter 12: Tuning a Sample Application

The objective of the final chapter of Java EE 7 Performance Tuning and Optimization is "to demonstrate the performance tuning activities of a small application so that we can practice some of the activities that we have learned in this book." The example used here is called ExcellentSurvey and numerous pages are devoted to identifying and fixing performance-related issues in this ExcellentSurvey sample application. The chapter's summary also summarizes the book as a whole.

General Observations
  • Java EE 7 Performance Tuning and Optimization covers a lot of ground. It is mostly at an introductory level. Much of the content in this book is available online, but the book brings all of this content together in a single source. Readers likely to benefit most from reading this book are those with minimal Java performance tuning experience. The book provides a wide overview of available tools, tips, techniques, and best practices related to Java performance tuning. This book is not as deep as a book like Java Performance, but is instead targeted at higher level performance tuning concepts and approaches.
  • Although Java EE 7 Performance Tuning and Optimization does cover performance concepts related to Java EE application servers and Java EE, much of what it covers can be applied even more generally to all types of Java-based/JVM-hosted applications.
  • Screen snapshots are generally sharp and readable. They tend to be full color in the electronic version (PDF) of the book that I reviewed.
  • Code listings are in black font on white background (no color syntax) and there are no line numbers included in the code listings.
  • A flowchart for troubleshooting Java performance issues is built as the book progresses and this is a good starting point for someone wanting a systematic approach to resolving performance issues.
  • Java EE 7 Performance Tuning and Optimization is lengthier than it needs to be.
    • Content reproduced from Oracle's sites and other sites could have been summarized and referenced rather than being reproduced. The section on Linux monitoring tools used the better approach of listing the tools and referencing where to get the details on those tools. I think it would have been better if other parts of the book had followed that same tactic instead of merely reproducing content from those other sources.
    • Some of the overview and high-level concepts chapters seemed overly verbose to me, but that could be more a matter of taste. I prefer books to dive right into technical detail with only a little description and background; others prefer more background and discussion.
  • Some portions of Java EE 7 Performance Tuning and Optimization were difficult to understand due to awkward sentences or imprecise wording.
    • For example, the fourth chapter and sixth chapters discuss use of profiling tools to "catch HotSpot areas." At first, I was trying to figure out how the paragraph in Chapter 4 was specific to the HotSpot virtual machine because it was referenced with the same trademarked-style "HotSpot." Only after its second use in the same section, where "HotSpot areas" were described as "underperforming areas," did I confirm my suspicion that these references were intended to be to "hot spots" rather than to "HotSpots. Once I realized this, I was able to use context to distinguish when the author was talking about the HotSpot JVM versus talking about "Hot Spots" spots where code execution times are longer or more memory is being consumed.
    • A second example is this phrase: "Missed cache hits are very costive..."
    • An otherwise interesting brief discussion on potential memory issues due to inconsistencies between equals(Object) and hashCode() is a bit marred by a typo (uses "mush" rather than "must") "In Java, if any two objects are equal, then their hash code values mush be equal as well."
    • There are numerous more examples like these and I feel that this book could have used a round or two more of text editing. More editing would have probably helped tighten the book up a bit as well.
  • Java EE 7 Performance Tuning and Optimization makes heavy use (sometimes word for word identical text) of some significant online resources on JVM and application server performance tuning. These excellent online resources include:
  • Although Java EE 7 Performance Tuning and Optimization generally provides broad coverage of performance tuning Java EE 7 applications, it is important to note in a book review the things it doesn't cover or only covers very briefly. These include:
    • Specific Java EE implementations (specific application servers) - this is understandable as entire books and guides can be written to each particular Java EE application server implementation.
    • Java Management Extensions (JMX) - JMX is briefly referenced in a few locations, but its direct use is relatively ignored (it is indirectly discussed in terms of much of the coverage of VisualVM and other JMX-powered tools).
    • Virtual Machine Environments - Although operating system level performance tuning is covered multiple times in this book, running Java applications in virtual machines is not discussed in any level of significant detail.
  • Readers of Java EE 7 Performance Tuning and Optimization will likely want to download the associated code samples. In particular, the code examples that intentionally have problems that can be diagnosed with the described tools can provide practice on applying those tools.
  • Because different reviewers have different tastes, different background, different experiences, and different preferences, I think it's important to read book reviews from multiple sources. A different perspective on Java EE 7 Performance Tuning and Optimization is available in Abhishek's post Book Review: Java EE 7 Performance Tuning and Optimization.
Conclusion

Java EE 7 Performance Tuning and Optimization is a lengthy book that at times felt lengthier to me than it needed to be. However, developers new to Java performance (both EE and SE) may appreciate this extra background. Significant portions of some of the earlier chapters are nearly word-for-word the same as online Oracle Java documentation, but some readers new to these tools may find this more convenient than reading the original sources. Experienced Java EE developers who have had to do basic tuning of their applications in the past are likely to find a few new ideas or concepts in this book, but the people most likely to benefit are those with relatively little or no Java performance experience. For them, this book provides an overall view of the landscape of tools and concepts related to Java EE (and really general JVM) performance tuning.

The book's strengths include clear screen snapshots that are especially useful in demonstrating graphical tools and discussion and demonstration of applying tools to code examples with performance issues. Its weaknesses include numerous grammar issues and typos that make the text less readable than it otherwise would have been and stating the same things too many times and being lengthier than it needs to be. On one hand, the repetition seems monotonous to someone familiar with the concepts, but, on the other hand, this repetition might benefit those new to the concepts.