Monday, July 9, 2018

Deferred Execution with Java's Predicate

In the previous posts "Deferred Execution with Java's Supplier" and "Deferred Execution with Java's Consumer", I looked at easily deferring execution in Java via standard Java APIs that accept, respectively, Suppliers and Consumers. In this post, I take a similar look at how standard JDK-provided APIs allow for deferred execution via the standard functional interface Predicate. The Predicate is described in its Javadoc, "Represents a predicate (boolean-valued function) of one argument." In other words, a Predicate is like a JDK-supplied Function, but with its return value limited to either true or false.

Perhaps the most common application of Predicate in the standard Java APIs is in the context of filters. Several of the examples in this post will demonstrate use of Predicate in conjunction with filtering methods on instances of Optional and on instances of Stream.

Optional.filter(Predicate)

The behavior of the Optional class's filter(Predicate) method is described this way by its Javadoc documentation, "If a value is present, and the value matches the given predicate, returns an Optional describing the value, otherwise returns an empty Optional." In other words, Optional.filter(Predicate) returns an Optional that will be empty if either the original Optional was empty or if the Predicate applied to the original and present Optional resolves to false. Otherwise, if the original Optional does have a "present" value and the Predicate applied to that value returns true, the returned Optional will also have the same "present" value. This is illustrated in the next code listing (full source is available on GitHub).

Optional.filter(Predicate) Demonstrated

/**
 * Demonstrate use of {@code Optional.filter(Predicate)} on an
 * {@code Optional<Boolean>}.
 */
public static void demonstrateOptionalFilterOnBoolean()
{
   out.print("\nfalse: ");
   getOptionalBoolean(false).filter(b -> b).ifPresent(out::print);
   out.print("\ntrue:  ");
   getOptionalBoolean(true).filter(b -> b).ifPresent(out::print);
   out.print("\nnull:  ");
   getOptionalBoolean(null).filter(b -> b).ifPresent(out::print);
}

/**
 * Demonstrate use of {@code Optional.filter(Predicate)} on an
 * {@code Optional<Float>}.
 */
public static void demonstrateOptionalFilterOnFloat()
{
   out.print("\n3.14: ");
   getOptionalFloat(3.14f).filter(f -> f > 0.0).ifPresent(out::print);
   out.print("\n-2.5: ");
   getOptionalFloat(-2.5f).filter(f -> f > 0.0).ifPresent(out::print);
   out.print("\nnull: ");
   getOptionalFloat(null).filter(f -> f > 0.0).ifPresent(out::print);
}

The two methods in the above code listing demonstrate use of Optional.filter(Predicate) on a lambda expression that results in a direct boolean result and on a lambda expression that results in a boolean result based on numerical comparison. In one case, the Predicate is the boolean and in the other case the Predicate is the numeric comparison.

Stream.filter(Predicate)

The Stream interface's method filter(Predicate) works similarly to the Optional class's method of the same name. The next code listing demonstrates application of Stream.filter(Predicate).

Stream.filter(Predicate) Demonstrated

/**
 * Demonstrates use of {@code Stream.filter(Predicate}}.
 */
public static void demonstrateStreamFilter()
{
   final int maximum = 100;
   out.println("\nThe probable prime numbers between 1 and " + maximum + " are:");
   final Stream<BigInteger> bigIntegers = getConsecutiveBigIntegers(maximum);
   bigIntegers.filter(bi -> bi.isProbablePrime(100)).forEach(pp -> out.println(" " + pp));
}

The above code listing is not intended to demonstrate the best approach to identifying prime numbers in Java. Instead, it's intended to demonstrate how filter(Predicate) can be invoked on a Stream to narrow down elements of that Stream to only those matching the Predicate.

For my next illustration of Stream.filter(Predicate), I use the Pattern class's convenient method asPredicate() to supply the instance of the Predicate to be supplied to both examples using Stream.filter(Predicate).

Stream.filter(Predicate) with Pattern.asPredicate() Demonstrated

/**
 * Demonstrates use of {@code Pattern.asPredicate()} to provide
 * a {@code Predicate} that can be used with {@code Stream.filter()}.
 */
public static void demonstratePatternAsPredicateInFilter()
{
   final long count
      = getPotentialTelephoneNumbers().stream()
         .filter(PATTERN.asPredicate())
         .peek(out::println)
         .count();
   out.println(count + " valid telephone numbers.");
}

Collection.removeIf(Predicate)

The Collection interface specifies (and implements as a default method) the useful method removeIf(Predicate). There are also multiple implementations of Collection that implement their own overridden versions of removeIf(Predicate) that include ArrayDeque.removeIf(Predicate), ArrayList.removeIf(Predicate), and Vector.removeIf(Predicate).

The next code listing demonstrates two examples of Collection.removeIf(Predicate) in action. The first example uses the method Predicate.negate() to negate the expected regular expression pattern so that the elements removed from the collection are those that do NOT match the regular expression. The second example performs similar functionality, but takes advantage of the JDK 11-introduced 'not' method to perform this negation.

Collection.removeIf(Predicate) with Negated Pattern.asPredicate() Demonstrated

/**
 * Demonstrates use of {@code Collection.removeIf(Predicate)}
 * in conjunction with {@code Predicate.negate()}.
 */
public static void demonstrateCollectionRemoveIf()
{
   final Set<String> telephoneNumbers = new HashSet<>(getPotentialTelephoneNumbers());
   telephoneNumbers.removeIf(PATTERN.asPredicate().negate());
   out.println(telephoneNumbers);
}

/**
 * Demonstrates use of {@code Collection.removeIf(Predicate)}
 * in conjunction with JDK 11-introduced {@code Predicate.not()}.
 */
public static void demonstrateCollectionRemoveIfWithJdk11Not()
{
   final Set<String> telephoneNumbers = new HashSet<>(getPotentialTelephoneNumbers());
   telephoneNumbers.removeIf(not(PATTERN.asPredicate()));
   out.println(telephoneNumbers);
}

Stream.allMatch(Predicate)

The Stream interface's method allMatch(Predicate) returns true if every single element in the stream matches the provided Predicate. If even a single element does not match the Predicate, the method returns false.

Stream.allMatch(Predicate) Demonstrated

/**
 * Demonstrate use of {@code Stream.allMatch(Predicate)}.
 */
public static void demonstrateStreamAllMatch()
{
   final Set<String> names = getNames();
   final boolean allNamesSixDigits = names.stream()
      .allMatch(name -> name.length() == 6);
   out.println("Are all names " + names + " six digits? " + allNamesSixDigits);
}

Stream.anyMatch(Predicate)

The Stream.anyMatch(Predicate) method returns true if at least one of its elements matches the Predicate and returns false if none of its elements match the Predicate.

Stream.anyMatch(Predicate) Demonstrated

/**
 * Demonstrate use of {@code Stream.anyMatch(Predicate)}.
 */
public static void demonstrateStreamAnyMatch()
{
   final Set<String> names = getNames();
   final boolean anyNamesSixDigits = names.stream()
      .anyMatch(name -> name.length() == 6);
   out.println("Are any names " + names + " six digits? " + anyNamesSixDigits);
}

Stream.noneMatch(Predicate)

The Stream.noneMatch(Predicate) method returns true when no elements in stream match the Predicate and returns false if at least one element in the stream DOES match the Predicate.

Stream.noneMatch(Predicate) Demonstrated

/**
 * Demonstrate use of {@code Stream.noneMatch(Predicate)}.
 */
public static void demonstrateStreamNoneMatch()
{
   final Set<String> names = getNames();
   final boolean noNamesSixDigits = names.stream()
      .noneMatch(name -> name.length() == 6);
   out.println("Are no names " + names + " six digits? " + noNamesSixDigits);
   final boolean noNamesFourDigits = names.stream()
      .noneMatch(name -> name.length() == 4);
   out.println("Are no names " + names + " four digits? " + noNamesFourDigits);
}

Collectors.partitioningBy(Predicate)

Although there are several more JDK APIs that use Predicate, I wrap this post up with discussion of and an example of using Collectors.partitioningBy(Predicate). This interesting method divides up all elements in the stream it is invoked upon into two groups with one group associated with key Boolean.TRUE (elements that match the Predicate) and with one group associated with the key Boolean.FALSE (those elements that did not match the Predicate). The next code listing takes advantage of this to divide integers up into even numbers and odd numbers.

Collectors.partitioningBy(Predicate) Demonstrated

/**
 * Demonstrate use of {@code Collectors.partitioningBy(Predicate)}.
 */
public static void demonstrateCollectorsPartitioningBy()
{
   final Map<Boolean, List<Integer>> evensAndOdds
      = getConsecutiveIntegers(100)
         .collect(Collectors.partitioningBy(integer -> integer % 2 == 0));
   out.println("Evens: " + evensAndOdds.get(Boolean.TRUE));
   out.println("Odds:  " + evensAndOdds.get(Boolean.FALSE));
}

I used several "helper" methods in the above code examples that are not shown in this post. These "helper" methods and all of the examples shown in this post are available on GitHub.

Java's standard functional interface Predicate is a specialized version of fellow built-in Java functional interface Function that arguably deserves its own specialization because the true/false return status is so commonly useful for representing the conditions in which certain functionality applies or does not apply. This post has demonstrated several instances in the JDK where Predicate is used to determine which stream elements apply, whether or not an Optional applies, and to divide up stream elements into those that satisfy the predicate and those that do not. Along the way, convenience methods such as Pattern.asPredicate() and Predicate.not() were also demonstrated.

Thursday, July 5, 2018

Applying New JDK 11 String Methods

In the posts "New Methods on Java String with JDK 11" and "String#repeat Coming to Java?", I discussed six new methods coming to the Java String with JDK 11. The available early access JDK 11 builds already include these new methods and I use one of those early access builds to demonstrate them in this post.

I am using OpenJDK JDK 11 Early Access Build 20 for compiling and running the examples shown in this post.

The six methods added to String for JDK 11 that are demonstrated in this post via the OpenJDK JDK 11 Early Access Build 20 are:

  • String.repeat(int)
  • String.lines()
  • String.strip()
  • String.stripLeading()
  • String.stripTrailing()
  • String.isBlank()

The source code for the examples demonstrated in this post is available on GitHub.

String.repeat(int)

The String.repeat(int) method provides handy functionality that I've wished to see in Java since experiencing this functionality in Groovy. As its name suggests, this method repeats the String it is run against as many times as provided by the int parameter. I will likely use this method frequently in the future when generating simple demonstrations and use it for this post's examples. The next code listing demonstrates use of String.repeat(int) to easily generate header separators for the demonstration output.

Use of String.repeat(int)

/**
 * Write provided {@code String} in header. Note that this
 * implementation uses {@code String.repeat(int)}.
 *
 * @param headerText Title of header.
 */
private static void writeHeader(final String headerText)
{
   final String headerSeparator = "=".repeat(headerText.length()+4);
   out.println("\n" + headerSeparator);
   out.println("= " + headerText + " =");
   out.println(headerSeparator);
}

The writeHeader(String) method uses String.repeat(int) to easily generate "header separator" lines from the "=" character enough times to cover the provided headerText length plus 4 additional characters to allow for an extra "=" and extra space on each side of the "header text". The writeHeader(String) method is used by all the other demonstration examples in this post and so will be demonstrated via those examples.

String.lines()

The String.lines() method splits the String upon which it is called by its line terminators and returns a Stream of Strings as demarcated by those line terminators.

Use of String.lines()

/**
 * Demonstrate method {@code String.lines()} added with JDK 11.
 */
public static void demonstrateStringLines()
{
   final String originalString = prepareStringWithLineTerminators();
   final String stringWithoutLineSeparators
      = originalString.replaceAll("\\n", "\\\\n");
   writeHeader("String.lines() on '"  + stringWithoutLineSeparators  + "'");
   final Stream<String> strings = originalString.lines();
   strings.forEach(out::println);
}

Sample output is shown in the next screen snapshot.

String.strip() / String.stripLeading() / String.stripTrailing()

The String.strip(), String.stripLeading(), and String.stripTrailing() methods trim white space [as determined by Character.isWhiteSpace()] off either the front, back, or both front and back of the targeted String.

Use of String.strip() / String.stripLeading() / String.stripTrailing()

/**
 * Demonstrate method {@code String.strip()} added with JDK 11.
 */
public static void demonstrateStringStrip()
{
   final String originalString = prepareStringSurroundedBySpaces();
   writeHeader("String.strip() on '" + originalString + "'");
   out.println("'" + originalString.strip() + "'");
}

/**
 * Demonstrate method {@code String.stripLeading()} added with JDK 11.
 */
public static void demonstrateStringStripLeading()
{
   final String originalString = prepareStringSurroundedBySpaces();
   writeHeader("String.stripLeading() on '" + originalString + "'");
   out.println("'" + originalString.stripLeading() + "'");
}

/**
 * Demonstrate method {@code String.stripTrailing()} added with JDK 11.
 */
public static void demonstrateStringStripTrailing()
{
   final String originalString = prepareStringSurroundedBySpaces();
   writeHeader("String.stripTrailing() on '" + originalString + "'");
   out.println("'" + originalString.stripTrailing() + "'");
}

When the above code is executed, the output looks like that shown in the next screen snapshot.

String.isBlank()

The String.isBlank() method indicates if the targeted String is empty or contains only white space characters as determined by Character.isWhitespace(int).

Use of String.isBlank()

/**
 * Demonstrate method {@code String.isBlank()} added with JDK 11.
 */
public static void demonstrateStringIsBlank()
{
   writeHeader("String.isBlank()");
   final String emptyString = "";
   out.println("Empty String -> " + emptyString.isBlank());
   final String onlyLineSeparator = System.getProperty("line.separator");
   out.println("Line Separator Only -> " + onlyLineSeparator.isBlank());
   final String tabOnly = "\t";
   out.println("Tab Only -> " + tabOnly.isBlank());
   final String spacesOnly = "   ";
   out.println("Spaces Only -> " + spacesOnly.isBlank());
}

An example of executing this code is shown in the next screen snapshot.

Some of the methods whose code is shown above call "helper" methods that can be seen on GitHub.

The methods added to JDK 11's String are small additions, but will make certain "presentation" tasks related to Java Strings easier than in the past and reduce the need for third-party libraries.

Wednesday, June 27, 2018

Looking into Java's Future: Empty Value Types

Although the promising Java value types are not imminent, I still enjoy nosing around occasionally in the OpenJDK valhalla-dev mailing list to see how things appear to be progressing and to get an idea of what is to come. Admittedly, there are some discussions that are beyond my understanding given my limited exposure to the terms used and the low-level specifics of some of these messages. However, there are occasionally messages and threads that I understand well what is being written and find interesting. A recent example of this is the "Empty value type ?" thread.

Rémi Forax starts the thread by asking "Is empty value type targeted for LW1?" The example error message included with that question shows a LinkageError and ClassFormatError with the message "Value Types do not support zero instance size yet". The response to this question from Tobias Hartmann answers, "No, empty value types are not planned to be supported for LW1."

Before moving onto the rest of the thread [which is the part that interested me the most], I'll quickly discuss "LW1." In a message on that same OpenJDK mailing list called "[lworld] LW1 - 'Minimal LWorld'", David Simms states, "we are approaching something 'usable' in terms of 'minimal L World' (LW1)" and "we will be moving of prototyping to milestone stabilization." That same message states that the "label" is "lw1" and the affected-version and fixed-version are both "repo-valhalla". In other words, "LW1" is the label used to track bugs and issues related to work on the "minimal L world" implementation. You can reference John Rose's 19 November 2017 message "abandon all U-types, welcome to L-world (or, what I learned in Burlington)" for an introduction to the "L World" term and what it means in terms of value types.

Returning to the "Empty value type?" thread, Kirk Pepperdine asked a question that also occurred to me, "How can a value type be empty?" He added, "What is an empty integer? An empty string?" He said he was "just curious" and now I was too. Here is a summary of the informative responses:

  • Rémi Forax: "type [that] represents the absence of value like unit, void or bottom"
  • Rémi Forax: "type that represents the result of a throw"
  • Rémi Forax: "type that allows HashSet<E> to be defined as HashMap<E,Empty>"
    • Brian Goetz's message elaborates on the value of this: "Zero-length values can be quite useful, just not directly. Look at the current implementations of Set that delegate to HashMap; all that wasted space. When we have specialized generics, they can specialize to HashMap<T, empty>, and that space gets squeezed away to zero."
  • Rémi Forax: "transformative type like a marker type that separate arguments" (see message for example of this one)

I also liked the final (as of this writing) Kirk Pepperdine message on that thread where he summarizes, "Feels like a value type version of null."

Incidentally, there are some other interesting messages and threads in the June 2018 Archives of the valhalla-dev mailing list. Here are some of them:

I look forward to hopefully one day being able to apply value types in my everyday Java code. Until then, it is interesting to think of what might be and also to see how much work is going into making it so.

Tuesday, June 26, 2018

Deferred Execution with Java's Consumer

In an earlier blog post ("Deferred Execution with Java's Supplier") I referenced Cay Horstmann's statement in the book "Java SE8 for the Really Impatient" regarding lambda expressions, "The point of all lambdas is deferred execution." Horstmann wrote an article called "Lambda Expressions in Java 8" for Dr. Dobb's magazine in its final year in which he wrote a similar statement using different terminology, "A lambda expression is a block of code that you can pass around so it can be executed later, just once or multiple times."

In that earlier post, I looked at how the standard functional interface Supplier is used with lambda expressions in the JDK to support deferred execution for cases where a single value is "supplied only when necessary" and without any argument passed to it. In this post, I focus on JDK-provided examples of using the Consumer standard functional interface to "consume" or "process" a particular code block "only when necessary." Whereas the Supplier accepts no arguments and returns exactly one response, the Consumer accepts one or more arguments and returns no response. The method invoked on a Supplier is the get() method and it is the accept(T) method for a Consumer. By definition, the Consumer is expected to have "side effects" as it "consumes" the provided code block.

There are numerous Consumer-style standard functional interfaces supplied in the java.util.function package. None of these returns a result (that's why they're consumers!), but they differ in the number and types of arguments they accept (but they all accept at least one argument). These are listed here:

  • Consumer - General Consumer that accepts a single argument and will be the center of attention for most of this post's examples.
  • BiConsumer - Accepts two arguments instead of one ("two-arity specialization of Consumer")
  • DoubleConsumer - Specialized Consumer intended for primitive doubles
  • IntConsumer - Specialized consumer for primitive ints
  • LongConsumer - Specialized Consumer intended for primitive longs
  • ObjDoubleConsumer - Specialized Consumer that accepts two arguments, with the first of type Object and the second of type double
  • ObjIntConsumer - Specialized Consumer that accepts two arguments, with the first of type Object and the second of type int
  • ObjLongConsumer - Specialized Consumer that accepts two arguments, with the first of type Object and the second of type long

The remainder of this post will look at a subset of the JDK uses of Consumer and related classes to help demonstrate how and when they are useful.

Peeking at Flow of Stream Elements

In the blog post "Peeking Inside Java Streams with Stream.peek," I discussed the intermediate operation Stream.peek(Consumer) that can be used to view the flowing elements of a stream. This can be very useful for understanding what the various stream operations are doing to their respective stream elements. A common way to do this is to have the Consumer provided to the peek method be a call to System.out.println that prints the currently processed stream element to standard output (or log the element or print it to standard error). An example of this is provided in the Javadoc documentation for the Stream.peek(Consumer) method:

Stream.of("one", "two", "three", "four")
   .filter(e -> e.length() > 3)
   .peek(e -> System.out.println("Filtered value: " + e))
   .map(String::toUpperCase)
   .peek(e -> System.out.println("Mapped value: " + e))
   .collect(Collectors.toList());

Because the various overloaded versions of the println(-) method accept a parameter but do not return anything, they fit perfectly with the "Consumer" concept.

Specifying Action Upon Iterated Stream Elements

While Stream.peek(Consumer) is an intermediate operation, Stream provides two other methods that accept a Consumer that are both terminal operations and are both "for each" methods. The method Stream.forEach​(Consumer) is a method that performs the action specified by the provided Consumer in an "explicitly nondeterministic" manner on the stream's elements. The method Stream.forEachOrdered(Consumer) performs the action specified by the provided Consumer in "the encounter order" of the stream if that stream has an encounter order. In both methods' cases, the Consumer-based "action" should be "non-interfering." Both methods are demonstrated below.

Set.of("one", "two", "three", "four")
   .stream()
   .forEach(i -> out.println(i.toUpperCase()));

Stream.of("one", "two", "three", "four")
   .forEach(i -> out.println(i.toUpperCase()));

List.of("one", "two", "three", "four")
   .stream()
   .forEachOrdered(i -> out.println(i.toUpperCase()));

Stream.of("one", "two", "three", "four")
   .forEachOrdered(i -> out.println(i.toUpperCase()));

The above examples look and very similar. The most obvious situation in which forEach could lead to dramatically different results than forEachOrdered is when parallel stream processing is employed. In that case, it makes most sent to use forEach instead of forEachOrdered.

Specifying Action Upon Iterable Elements

The previous code examples showed using Stream.forEach(Consumer) methods to iterate a stream. The examples also demonstrated doing this against a Set and List by first calling stream() on these collections. There are convenience methods, however, that are defined by Iterable and implemented by these collection implementations which accept a Consumer and allow for iteration of that collection using the forEach method. Examples of this are shown in the next code listing.

Set.of("one", "two", "three", "four")
   .forEach(i -> out.println(i.toUpperCase()));
List.of("one", "two", "three", "four")
   .forEach(i -> out.println(i.toUpperCase()));

Although I used collections in my example above, anything that implements Iterable will generally support the forEach method (or be in violation of the interface's advertised contract).

Specifying Action Upon Iteration of Map Entries

Although Java's Map interface does not extend the Iterable interface like Set and List do, the Java Map was still provided with a similar capability to specify a consumer to "consume" each entry in the Map. Because a Map has two input arguments (key and value), its forEach method accepts a BiConsumer instead of the Consumer discussed so far in this post. A simple example is shown next.

Map.of("Denver", "Colorado",
       "Cheyenne", "Wyoming",
       "Salt Lake City", "Utah",
       "Boise", "Idaho")
   .forEach((c, s) -> out.println(c + " is the capital of " + s));

Walking the Stack

The StackWalker is a welcome addition to JDK 9 that provides a thread-safe approach to perusing a stack trace and is a significant improvement over the StackTraceElement approach. It's arguably more common for developers to use StackWalker.walk(Function), but this post is about Consumer and so the focus is on StackWalker.forEach(Consumer). This method is similar to the previously discussed Stream.forEach and Iterable.forEach methods and is demonstrated in the next code listing.

StackWalker.getInstance().forEach(out::println);

Although there are many more JDK uses of Consumer, of BiConsumer, and of the other types of standard Consumer-style functional interfaces, the last examples I'll cover in this post come from the Optional class.

Applying Only When Present

The methods Optional.ifPresent(Consumer) and Optional.ifPresentOrElse(Consumer) defer the execution of the provided Consumers such that the provided Consumer will only be invoked if the Optional is not "empty" (contains a non-null value). This is a simple but powerful concept and the simplistic and contrived examples show how they work.

public void demonstrateOptionalIfPresent()
{
   getMiddleName(true).ifPresent(n -> out.println("Middle Name: " + n));
}

public void demonstrateOptionalIfPresentOrElse()
{
   getMiddleName(false).ifPresentOrElse(
      n -> out.println("Middle Name: " + n),
      () -> displayMissingMiddleName());
}

private Optional<String> getMiddleName(final boolean present)
{
   return present ? Optional.of("Wayne") : Optional.empty();
}

private void displayMissingMiddleName()
{
   out.println("No middle name provided!");
}

As the above code listing demonstrates, both Optional.ifPresent and JDK 9-introduced Optional.ifPresentOrElse() only invoke the provided Consumer if the Optional is not empty. If the Optional is empty, the ifPresent method does nothing and the ifPresentOrElse invokes the second argument (a Runnable).

The standard Java functional interfaces that accept one or more arguments and return no result include the general Consumer as well as some specialized consumers. These are useful for deferring execution until a given condition occurs (such as being iterated upon or being determined to be present) and the behavior to apply when that condition occurs involves one or more input arguments and no need to provide a response. The source code examples shown in this post are available on GitHub.

Wednesday, June 20, 2018

Java's Ternary is Tricky with Autoboxing/Unboxing

The comments section of the DZone-syndicated version of my post "JDK 8 Versus JDK 10: Ternary/Unboxing Difference" had an interesting discussion regarding the "why" of the "fix" for how Java handles autoboxing/unboxing in conjunction with use of the ternary operator (AKA "conditional operator"). This post expands on that discussion with a few more details.

One of the points made in the discussion is that the logic for how primitives and reference types are handled in a ternary operator when autoboxing or unboxing is required can be less than intuitive. For compelling evidence of this, one only needs to look at the number of bugs written for perceived problems with Java's conditional operator's behavior when autoboxing and unboxing are involved:

  • JDK-6211553 : Unboxing in conditional operator might cause null pointer exception
    • The "EVALUATION" section states, "This is not a bug." It then explains that the observed behavior that motivated the writing of the bug "is very deliberate since it makes the type system compositional." That section also provides an example of a scenario that justifies this.
  • JDK-6303028 : Conditional operator + autoboxing throws NullPointerException
    • The "EVALUATION" section states, "This is not a bug." This section also provides this explanation:
      The type of the conditional operator 
      
      (s == null) ? (Long) null : Long.parseLong(s)
      
      is the primitive type long, not java.lang.Long.
      This follows from the JLS, 3rd ed, page 511:
      
      "Otherwise, binary numeric promotion (5.6.2) is applied to the operand
      types, and the type of the conditional expression is the promoted type of the
      second and third operands. Note that binary numeric promotion performs
      unboxing conversion (5.1.8) and value set conversion (5.1.13)."
      
      In particular, this means that (Long)null is subjected to unboxing conversion.
      This is the source of the null pointer exception.
      
  • JDK-8150614 : conditional operators, null argument only for return purpose, and nullpointerexception
    • The "Comments" section explains "The code is running afoul of the complicated rules for typing of the ?: operator" and references the pertinent section of the Java Language Specification for the current version at time of that writing (https://docs.oracle.com/javase/specs/jls/se8/html/jls-15.html#jls-15.25).
    • I like the explanation on this one as well: "The code in the bug has one branch of the ?: typed as an Integer (with the 'replace' variable") and the other branch typed as an int from Integer.parseInt. In that case, first an unboxing Integer -> int conversion will occur before a boxing to the final result, leading to the NPE. To avoid this, case the result of parseInt to Integer."
    • The "Comments" section concludes, "Closing as not a bug."
  • JDK-6777143 : NullPointerException occured at conditional operator
    • The "EVALUATION" section of this bug report provides interesting explanation with a historical perspective:
      It is because of NPEs that JLS 15.25 says 'Note that binary numeric promotion performs unboxing conversion'. The potential for NullPointerExceptions and OutOfMemoryErrors in 1.5 where they could never have occurred in 1.4 was well known to the JSR 201 Expert Group. It could have made unboxing conversion from the null type infer the target type from the context (and have the unboxed value be the default value for that type), but inference was not common before 1.5 expanded the type system and it's certainly not going to happen now.
  • JDK-6360739 : Tertiary operator throws NPE due to reduntant casting

It's no wonder it's not intuitive to many of us! Section 15.25 ("Conditional Operator ? :") of the Java Language Specification is the defining authority regarding the behavior of the ternary operator with regards to many influences, including autoboxing and unboxing. This is the section referenced in several of the bug reports cited above and in some of the other resources that I referenced in my original post. It's worth noting that this section of the PDF version of the Java SE 10 Language Specification is approximately 9 pages!

In the DZone comments on my original post, Peter Schuetze and Greg Brown reference Table 15.25-D from the Java Language Specification for the most concise explanation of the misbehavior in JDK 8 that was rectified in JDK 10. I agree with them that this table is easier to understand than the accompanying text illustrated by the table. That table shows the type of the overall ternary operation based on the types of the second expression and third expression (where second expression is the expression between the ? and : and the third expression is the expression following the : as shown next):

    first expression ? second expression : third expression

The table's rows represent the type of the second expression and the table's columns represent the type of the third expression. One can find where the types meet in the table to know the overall type of the ternary operation. When one finds the cell of the table that correlates to row of primitive double and column of reference Double, the cell indicates that the overall type is primitive double. This is why the example shown in my original post should throw a NullPointerException, but was in violation of the specification in JDK 8 when it did not do so.

I sometimes wonder if autoboxing and unboxing are a case of the "cure being worse than the disease." However, I have found autoboxing and unboxing to be less likely to lead to subtle errors if I'm careful about when and how I use those features. A J articulates it well in his comment on the DZone version of my post: "The practical takeaway I got from this article is: when presented with an incomprehensible error, if you see that you are relying on autoboxing in that area of code (i.e., automatic type conversion), do the type conversion yourself manually. Then you will be sure the conversion is being done right."

Saturday, June 16, 2018

JDK 11: Beginning of the End for Java Serialization?

In the blog post "Using Google's Protocol Buffers with Java," I quoted Josh Bloch's Third Edition of Effective Java, in which he wrote, "There is no reason to use Java serialization in any new system you write." Bloch recommends using "cross-platform structured-data representations" instead of Java's deserialization. The proposed JDK 11 API documentation will include a much stronger statement about use of Java deserialization and this is briefly covered in this post.

The second draft of the "Java SE 11 (18.9) (JSR 384)" specification includes an "A2 Annex" called "API Specification differences" that includes the changes coming to the Javadoc-based documentation for package java.io. The new java.io package documentation will include this high-level warning comment:

Warning: Deserialization of untrusted data is inherently dangerous and should be avoided. Untrusted data should be carefully validated according to the "Serialization and Deserialization" section of the Secure Coding Guidelines for Java SE.

At the time of the writing of this post, the referenced Secure Coding Guidelines for Java SE states that it is currently Version 6.0 and was "Updated for Java SE 9."

The intended package-level documentation for package java.io in JDK 11 will also provide links to the following additional references (but likely to be JDK 11-based references):

The former reference link to the "Java Object Serialization" (JDK 8) document will be removed from java.io's package documentation.

In addition to the java.io package documentation that is being updated in JDK 11 related to the dangers of Java deserialization, the java.io.Serializable interface's Javadoc comment is getting a similar high-level warning message.

These changes to the Javadoc-based documentation in JDK 11 are not surprising given various announcements over the past few years related to Java serialization and deserialization. "RFR 8197595: Serialization javadoc should link to security best practices" specifically spelled out the need to add this documentation. A recent InfoWorld article called "Oracle plans to dump risky Java serialization" and an ADT Magazine article called "Removing Serialization from Java Is a 'Long-Term Goal' at Oracle" quoted Mark Reinhold's statement at Devoxx UK 2018 that adding serialization to Java was a "horrible mistake in 1997."

There has been talk of removing Java serialization before, sometimes even in humor. JEP 154: Remove Serialization was created with the intent to "deprecate, disable, and ultimately remove the Java SE Platform's serialization facility." However, that JEP's status is now "Closed / Withdrawn" (April Fool's Day prank). Still, as talk of removing Java serialization picks up, it seems prudent to consider alternatives to Java serialization for all new systems, which is precisely what Bloch recommends in Effective Java's Third Edition. All this being stated, Apostolos Giannakidis has written in the blog post "Serialization is dead! Long live serialization!" that "deserialization vulnerabilities are not going away" because "Java's native serialization is not the only flawed serialization technology."

Additional References

Thursday, June 14, 2018

JDK 8 BigInteger Exact Narrowing Conversion Methods

In the blog post "Exact Conversion of Long to Int in Java," I discussed using Math.toIntExact(Long) to exactly convert a Long to an int or else throw an ArithmeticException if this narrowing conversion is not possible. That method was introduced with JDK 8, which also introduced similar narrowing conversion methods to the BigInteger class. Those BigInteger methods are the topic of this post.

BigInteger had four new "exact" methods added to it in JDK 8:

As described above, each of these four "exact" methods added to BigInteger with JDK 8 allow for the BigInteger's value to be narrowed to the data type in the method name, if that is possible. Because all of these types (byte, short, int, and long) have smaller ranges than BigInteger, it's possible in any of these cases to have a value in BigDecimal with a magnitude larger than that which can be represented by any of these four types. In such a case, all four of these "Exact" methods throw an ArithmeticException rather than quietly "forcing" the bigger value into the smaller representation (which is typically a nonsensical number for most contexts).

Examples of using these methods can be found on GitHub. When those examples are executed, the output looks like this:

===== Byte =====
125 => 125
126 => 126
127 => 127
128 => java.lang.ArithmeticException: BigInteger out of byte range
129 => java.lang.ArithmeticException: BigInteger out of byte range
===== Short =====
32765 => 32765
32766 => 32766
32767 => 32767
32768 => java.lang.ArithmeticException: BigInteger out of short range
32769 => java.lang.ArithmeticException: BigInteger out of short range
===== Int =====
2147483645 => 2147483645
2147483646 => 2147483646
2147483647 => 2147483647
2147483648 => java.lang.ArithmeticException: BigInteger out of int range
2147483649 => java.lang.ArithmeticException: BigInteger out of int range
===== Long =====
9223372036854775805 => 9223372036854775805
9223372036854775806 => 9223372036854775806
9223372036854775807 => 9223372036854775807
9223372036854775808 => java.lang.ArithmeticException: BigInteger out of long range
9223372036854775809 => java.lang.ArithmeticException: BigInteger out of long range

The addition of these "exact" methods to BigInteger with JDK 8 is a welcome one because errors associated with numeric narrowing and overflow can be subtle. It's nice to have an easy way to get an "exact" narrowing or else have the inability to do that narrowing exactly made obvious via an exception.