Saturday, November 2, 2019

Six More JEPs Proposed for JDK 14

A recent Mark Reinhold message announces six new "JEPs proposed to target JDK 14": JEP 345, JEP 361, JEP 363, JEP 364, JEP 365, and JEP 367. Assuming no objections are made by November 7, these JEPs will be targeted to JDK 14 along with previously targeted JEPs JEP 349 ("JFR Event Streaming"), JEP 352 ("Non-Volatile Mapped Byte Buffers"), and JEP 358 ("Helpful NullPointerExceptions"). This post summarizes each of these six JEPs proposed for targeting JDK 14.

JEP 345: NUMA-Aware Memory Allocation for G1

The succinct "Summary" of JEP 345 states, "Improve G1 performance on large machines by implementing NUMA-aware memory allocation." Two important "non-goals" of this JEP indicate that the JEP is only intended to add NUMA (non-uniform memory access) support to the G1 garbage collector and only for Linux. However, the JEP also points out that "the parallel collector, enabled by by -XX:+UseParallelGC, has been NUMA-aware for many years."

JEP 361: Switch Expressions (Standard)

JEP 361 moves switch expressions from their JEP 325 "preview" status to "standard" status. Along the way, JEP 354 (targeted at JDK 13) made this change to the preview switch expressions feature: "To yield a value from a switch expression, the break with value statement is dropped in favor of a yield statement." The "History" section of JEP 361 discusses switch expressions support in preview status in JDK 12 and JDK 13 and explains why it is proposed for standard status with JDK 14: "Feedback on JDK 13 suggests that this feature is now ready to be made final and permanent in JDK 14."

JEP 363: Remove the Concurrent Mark Sweep (CMS) Garbage Collector

JEP 363's concise "Summary" states, "Remove the Concurrent Mark Sweep (CMS) garbage collector." The Concurrent Mark Sweep (CMS) garbage collector was deprecated with JEP 291 (JDK 9) and this JEP aims to remove it altogether with JDK 14.

JEP 364: ZGC on macOS / JEP 365: ZGC on Windows

The goals of JEP 364 and JEP 365 are to "port the ZGC garbage collector" to macOS and Windows operating systems respectively.

JEP 364's "Motivation" section states, "While we expect users that require the scalability of ZGC to use Linux-based environments, it is not uncommon that developers use Macs for local development and testing, before deploying applications." Although JEP 365 lacks an explicit "Motivation" section, its motivation is probably similar to that for JEP 364.

JEP 365's "Non-Goals" section states, "It is not a goal to support Windows 10 and Windows Server older than version 1803, since older versions lack the required API for placeholder memory reservations."

JEP 367: Remove the Pack200 Tools and API

The "Summary" section of JEP 367 begins with this statement, "Remove the pack200 and unpack200 tools, and the Pack200 API in the java.util.jar package." This removal of these tools and APIs was advertised in Java SE 11 via JEP 336 when they were deprecated.

The "Motivation" sections of both the deprecation JEP 336 and the removal JEP 367 list three reasons for deprecating and removing Pack200 and conclude with this statement, "Overall, the cost of maintaining Pack200 is significant, and outweighs the benefit of including it in Java SE and the JDK."

The "Risks and Assumptions" section of JEP 336 includes this statement, "We assume that developers who use pack200 to shrink application JARs can switch to either the jlink tool or the jpackage tool to create application-specific runtimes with an optimized form factor." There is discussion regarding the suitability of jlink and jpackage on the r/java sub-reddit.

Java SE Specification Impacts

Iris Clark has posted that two of these JEPs have Java SE scope. She writes that JEP 361 (switch expressions standard) and JEP 367 (removing Pack200) are "of scope 'SE' [and] have been Proposed to Target for Java SE 14."

Conclusion

If all six of these JEPs proposed for JDK 14 get targeted for JDK 14, there will be nine JEPs associated with JDK 14. Two that I'm particularly looking forward to are the standardizing the switch expressions feature with JEP 361 and better NPE messages with JEP 358.

Saturday, October 26, 2019

Better NPE Messages in JDK 14

My March 2019 blog post "Better Default NullPointerException Messages Coming to Java?" was written when the draft JEP for better NullPointerException messages had not yet been targeted to a particular JDK release. Since then, that draft JEP became JEP 358 ("Helpful NullPointerExceptions"), which has been targeted for JDK 14. Even better, the initial implementation (JDK-8218628) is already in the JDK 14 branch and is available to play with in JDK 14 Early Access Builds Build 20 (2019/10/23).

In this post, I will run the example code introduced in my previous post against the JDK 14 Early Access Build 20 to demonstrate the additional details now provided. To see that example code that was written to intentionally introduce a variety of situations resulting in NullPointerExceptions, see that earlier post or view the source code on GitHub.

With the JDK 14 Early Access Build 20 downloaded and pointed to by my path, I see the following when I run java -version:

openjdk version "14-ea" 2020-03-17
OpenJDK Runtime Environment (build 14-ea+20-879)
OpenJDK 64-Bit Server VM (build 14-ea+20-879, mixed mode, sharing)

With the JDK 14 Early Access Build 20 configured appropriately, I rebuilt the source code mentioned previously and then re-ran the code with the java launcher without any new options. The output from this (shown below) is not materially different from the output with previous JDK versions.

=========================================
| #1: Element [0] on null boolean array |
=========================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateFirstExampleIndexAccessOnNullBooleanArray(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

=================================
| #2: .length on null boolean[] |
=================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateSecondExampleLengthOnNullBooleanArray(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

=======================================
| #3: Assigning float to null float[] |
=======================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateThirdExampleAssigningValueToElementOfNullFloatArray(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

======================================
| #4: Accessing field on null object |
======================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateFourthExampleAccessInstanceFieldOfNullObject(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

===================
| #5: throw null; |
===================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateFifthExampleThrowingConstantNull(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

================================================
| #6: Method invocation on null instance field |
================================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateSixthExampleMethodInvocationOnNullInstanceField(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

=============================================
| #7: synchronized() on null instance field |
=============================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateSeventhExampleSynchronizedNullInstanceField(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

==========================================================================
| >>> Null Lost in Long Series of Method Invocations in Single Statement |
==========================================================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateNullLostInSeriesOfMethodInvocationsInSingleStatement(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

=======================================================
| >>> Null Lost in Dereferenced Constructor Arguments |
=======================================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateNullLostInConstructorAcceptingMultiplePotentiallyNullArgumentsDereferenced(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

==================================================
| >>> Null Lost in Dereferenced Method Arguments |
==================================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateNullLostInMethodAcceptingMultiplePotentiallyNullArgumentsDereferenced(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

As the above output demonstrates, even with the new JDK 14 Early Access Build 20, I don't see any new detailed information regarding NullPointerExceptions when I run my application as normal. I included this output to show that a special flag is needed to enable the more detailed NullPointerExceptions and to make it more convenient to compare the output without and with the extra details. The next output listing shows the additional details provided when the java launcher is passed the flag -XX:+ShowCodeDetailsInExceptionMessages:

=========================================
| #1: Element [0] on null boolean array |
=========================================

java.lang.NullPointerException: Cannot load from byte/boolean array because "<local1>" is null
 at dustin.examples.npe.NpeDemo.demonstrateFirstExampleIndexAccessOnNullBooleanArray(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

=================================
| #2: .length on null boolean[] |
=================================

java.lang.NullPointerException: Cannot read the array length because "<local1>" is null
 at dustin.examples.npe.NpeDemo.demonstrateSecondExampleLengthOnNullBooleanArray(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

=======================================
| #3: Assigning float to null float[] |
=======================================

java.lang.NullPointerException: Cannot store to float array because "<local1>" is null
 at dustin.examples.npe.NpeDemo.demonstrateThirdExampleAssigningValueToElementOfNullFloatArray(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

======================================
| #4: Accessing field on null object |
======================================

java.lang.NullPointerException: Cannot read field "nullInstanceField" because "<local1>" is null
 at dustin.examples.npe.NpeDemo.demonstrateFourthExampleAccessInstanceFieldOfNullObject(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

===================
| #5: throw null; |
===================

java.lang.NullPointerException: Cannot throw exception because "null" is null
 at dustin.examples.npe.NpeDemo.demonstrateFifthExampleThrowingConstantNull(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

================================================
| #6: Method invocation on null instance field |
================================================

java.lang.NullPointerException: Cannot invoke "String.isEmpty()" because "this.nullInstanceField" is null
 at dustin.examples.npe.NpeDemo.demonstrateSixthExampleMethodInvocationOnNullInstanceField(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

=============================================
| #7: synchronized() on null instance field |
=============================================

java.lang.NullPointerException: Cannot enter synchronized block because "this.nullInstanceField" is null
 at dustin.examples.npe.NpeDemo.demonstrateSeventhExampleSynchronizedNullInstanceField(Unknown Source)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

==========================================================================
| >>> Null Lost in Long Series of Method Invocations in Single Statement |
==========================================================================

java.lang.NullPointerException: Cannot invoke "dustin.examples.npe.DysfunctionalLocation$Province.getCity()" because the return value of "dustin.examples.npe.DysfunctionalLocation$Nation.getProvince()" is null
 at dustin.examples.npe.NpeDemo.demonstrateNullLostInSeriesOfMethodInvocationsInSingleStatement(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

=======================================================
| >>> Null Lost in Dereferenced Constructor Arguments |
=======================================================

java.lang.NullPointerException: Cannot invoke "java.lang.Long.longValue()" because "<local6>" is null
 at dustin.examples.npe.NpeDemo.demonstrateNullLostInConstructorAcceptingMultiplePotentiallyNullArgumentsDereferenced(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

==================================================
| >>> Null Lost in Dereferenced Method Arguments |
==================================================

java.lang.NullPointerException: Cannot invoke "java.lang.Long.longValue()" because "<local6>" is null
 at dustin.examples.npe.NpeDemo.demonstrateNullLostInMethodAcceptingMultiplePotentiallyNullArgumentsDereferenced(Unknown Source)
 at dustin.examples.npe.NpeDemo.main(Unknown Source)

JEP 358 explains the use of this flag for seeing additional NullPointerException details: "The feature can be toggled with the new boolean command-line option -XX:{+|-}ShowCodeDetailsInExceptionMessages. The option will first have default 'false' so that the message is not printed. It is intended to enable code details in exception messages by default in a later release." As we see, this feature is initially turned off by default, but there is a plan to enable the more detailed NullPointerException messages in the future.

A recent Tweet asked the question, "How it will work if bytecode doesn’t contain variable names?" The question continued by providing a specific example: "Suppose we have code like Object a = ....; a.getName(); //NPE What kinds of message NPE would have?" Although an example of this is included in my battery of tests shown earlier, I thought I'd provide a more focused example here in response to that question. The next code listing (which is also available on GitHub) shows code adapted from the example used in the Tweet.

package dustin.examples.npe;

/**
 * Simple demonstration to answer Tweet-ed question
 * "How it will work if bytecode doesn't contain variable names?"
 * (https://twitter.com/2doublewhiskey/status/1180365953240055809).
 */
public class TwoDoubleWhiskeyTweetExample
{
   public static void main(final String[] arguments)
   {
      final Person person = null;
      person.getName(); //NPE
   }
   
   public static class Person
   {
      private String name;

      public Person(final String newName)
      {
         name = newName;
      }

      public String getName()
      {
         return name;
      }
   }
}

The next screen snapshot shows the result of running this simple application with the JDK 14 Early Access Build 20 without and then with the java launcher flag -XX:+ShowCodeDetailsInExceptionMessages.

As the screen snapshot indicates, using the -XX:+ShowCodeDetailsInExceptionMessages flag with the JDK 14 Early Access Build 20 provides this additional detail related to this simple NullPointerException example: "Cannot invoke "dustin.examples.npe.TwoDoubleWhiskeyTweetExample$Person.getName()" because "<local1>" is null"

An example that is simpler and even closer to the original example provided in the Tweet-ed question is available on GitHub.

JEP 358 ("Helpful NullPointerExceptions") may not be as flashy as some other JEPs that come to new JDK releases, but it may be one that in the end provides more value to Java developers on a daily basis than some of its flashier peers. There are numerous examples where this will be helpful and many of those example situations are spelled out in the JEP itself and in my code examples referenced in this post.

Monday, September 2, 2019

Sealed Types: JLS Changes (Draft)

Following the recent announcement of the candidate JEP on sealed types (preview), Gavin Bierman's message "Draft JLS spec for sealed types" on the OpenJDK amber-spec-experts mailing list announces "a draft language spec for sealed types" and provides a link to that draft. That message also states, "This spec doesn’t yet contain details on binary compatibility (Chapter 13) - to appear in the next draft."

In this post, I highlight some of the proposed changes to the Java Language Specification (JLS) for sealed types. It is important to keep in mind the tentativeness of these proposed changes: this is a draft of proposed JLS changes for a preview feature associated with a candidate JDK Enhancement Proposal (JEP).

With the caveats just outlined in mind, here are some of the interesting currently proposed changes to the JLS for sealed types with new text, deleted text, and typos highlighted differently.

  • Chapter 8: Classes
    • New sentences: "The degree to which a class can be extended can be explictly controlled. A class may be declared sealed, in which case there is a fixed set of classes that directly extend the sealed class."
    • Two new class modifiers spelled out in Section 8.1.1 ("Class Modifiers"): "abstract static sealed non-sealed final strictfp"
    • New sentence in Section 8.1.1: "It is a compile-time error if a class declaration has more than one of the class modifiers sealed, non-sealed and final."
    • Title of Section 8.1.1.2 changes to "sealed, non-sealed, and final Classes"
    • Three new sentences in renamed Section 8.1.1.2:
      • "A class can be declared sealed when it is useful to restrict its subclasses to a fixed set of classes."
      • "In certain circumstances, a class can be declared non-sealed to allow unrestricted subclasses."
      • "It is a compile-time error if a class that does not extend a sealed class or implement a sealed interface is declared non-sealed."
    • The entire section 8.1.6 ("Permitted subclasses") is new [former Section 8.1.6 is proposed to be Section 8.1.7] and here are just a few of its sentences:
      • "A sealed class can restrict its subclasses to a fixed set of classes. The permitted subclasses of a sealed class C are declared in a permits clause. A sealed class C may have an explicitly declared permits clause, which provides a non-empty list of the permitted subclasses of C."
      • "It is a compile-time error if a class is declared both abstract and sealed, and has no permitted subclasses, because the implementation of such a class could never be completed."
      • "It is a compile-time error if a class that is not sealed has a permits clause."
  • Chapter 9: Interfaces
    • New sentence: "Unlike a class, an interface cannot be declared final. However, an interface may be declared sealed, in which case it specifies a fixed set of classes and interfaces that directly implement or extend the sealed interface."
    • Two new interface modifiers spelled out in Section 9.1.1 ("Interface Modifiers"): "abstract static sealed non-sealed strictfp"
    • Section 9.1.1.3 is proposed as a new section called "sealed and non-sealed Interfaces". It has several new sentences, a subset of which are shown here.
      • "An interface can be declared sealed when it is useful to restrict its subtypes to a fixed set of classes and interfaces."
      • "It is a compile-time error if an interface that does not extend a sealed interface is declared non-sealed."
    • There is a new Section 9.1.4 (the old Section 9.1.4 is proposed to be changed to Section 9.1.5) called "Permitted Subtypes" and here is a subset of sentences in this section.
      • "It is a compile-time error if a sealed interface has no permitted subtypes."
      • "It is a compile-time error if an interface declaration has an explicit permits clause but is not sealed."
      • "If a sealed interface I does not have an explicit permits clause, then it has an implicitly declared permits clause that lists as permitted subtypes all the classes and interfaces in the same compilation unit as I that declare I as their direct superinterface."

There are additional details of sealed types discussed in this draft JLS document that are not highlighted here. In particular, the proposed draft JLS changes specify handling of sealed classes, sealed interfaces, and permitted types within Java modules. In short, it's a compile-time error to have any class designated as permitted for a sealed class or sealed interface that is not in the same module as the sealed class or sealed interface.

The JEP for the sealed types preview feature and the proposed JLS changes for the sealed types preview are both currently in "candidate"/"draft" form, but it's encouraging to see progress being made in the area of sealed types. It will be interesting to see the forthcoming language specification changes related to binary compatibility.

Thursday, August 29, 2019

Candidate JEPs: Records and Sealed Types

Mark Reinhold announced two new closely related candidate JDK Enhancement Proposals (JEPs) on the OpenJDK amber-dev mailing list this week with the posts "New candidate JEP: 359: Records (Preview)" and "New candidate JEP: 360: Sealed Types (Preview)." Both of these candidate JEPs are "preview features" (defined by JEP 12).

JEP 359: Records (Preview)

The JEP 359 "Summary" states, "Enhance the Java programming language with records. Records provide a compact syntax for declaring classes which are transparent holders for shallowly immutable data."

The "Motivations and Goals" section of JEP 359 explains how records would benefit Java developers. That section begins by stating that "it is a common complaint that 'Java is too verbose' or has too much 'ceremony" and explaining that "some of the worst offenders are classes that are nothing more than plain 'data carriers' that serve as simple aggregates." This section also states that records are intended to be more than mere "boilerplate reduction" and that they "should be easy, clear, and concise to declare shallowly-immutable, well-behaved nominal data aggregates." In short, the stated driving goal of JEP 359 is to "model data as data."

The recently proposed java.lang.Record draft specification provides significant insight into the characteristics of records. The opening paragraph of the "Description" section of JEP 359 also describes records: "Records are a new kind of type declaration in the Java language. Like an enum, a record is a restricted form of class. It declares its representation, and commits to an API that matches that representation. Records give up a freedom that classes usually enjoy: the ability to decouple API from representation. In return, records gain a significant degree of concision." There is significantly more text in the "Description" section of JEP 359.

JEP 360: Sealed Types (Preview)

The "Summary" section of JEP 360 states, "Enhance the Java programming language with sealed types. Sealed types are classes or interfaces that impose restrictions on which other classes or interfaces may extend or implement them."

The "Goals" section of JEP 360 is also concise, "Enable classes and interfaces to limit permitted subtypes to an enumerated set of types in the same maintenance domain as the type itself."

It is the "Description" section of JEP 360 that provides concreteness to JEP 360. That section begins, "A sealed type is one for which subtyping is restricted according to guidance specified with the type's declaration." The second paragraph of the "Description" section states that "sealing serves two distinct purposes" and describes those purposes:

  1. "Restricts which classes may be a subclass of a sealed class."
  2. "Potentially enables exhaustiveness analysis at the use site, such as when switching over type patterns for an instance of a sealed type."

There are other interesting characteristics of sealed types described in the "Description" section. Some of these that stood out to me are:

  • Use (with example) of the sealed modifier and permits clause.
  • "Abstract subtypes of sealed types are implicitly sealed, unless declared with the non-sealed modifier."
  • "Concrete subtypes of sealed types are implicitly final, unless declared with the non-sealed modifier."
  • "Sealing, like finality, is enforced by both the language compiler and by the JVM. The sealed-ness of a type, and its list of permitted subtypes, are reified in the class file and enforced at runtime."

Other interesting details related to sealed types that are covered in this JEP include restrictions (compiler errors that can occur), the class form for sealed types, and addition of reflection methods to support sealed types.

Conclusion

JEP 359 (Records Preview) and JEP 360 (Sealed Types Preview) reference each other in their documentation. Of the relationship between these two candidate JEPs, JEP 360 states, "Sealed types and records, taken together, form a construct often referred to as algebraic data types." Records and sealed types are key pieces in the move toward Java support for pattern matching.

Friday, August 16, 2019

java.lang.Record: Draft Specification

Work on proposed Java Records continues to proceed. Brian Goetz started three new threads on the OpenJDK amber-spec-experts mailing list yesterday and two of them are focused on Java Records. One of these two Record-oriented threads discusses whether Java records should support varargs. The other thread provides the initial draft specification for the proposed class java.lang.Record and that is the subject of this post.

The first sentence of the proposed class-level Javadoc for java.lang.Record currently says of this class, "This is the common base class of all Java language record classes." This initial specification also shows java.lang.Record being designated as a public abstract class.

Three "common" public abstract methods are explicitly declared in this initial specification of java.lang.Record: equals(Object), hashCode(), and toString(). All three methods are annotated with @Override and documented with {@inheritDoc} with Record specialization details. The specializations of the Javadoc for each of the three methods include Record-specific implementation notes using the @implNote tag. The class-level Javadoc tells us that these three "common" methods can be implicitly created: "The implicit declaration of the equals(Object), hashCode(), and toString() methods are derived from all of the component fields."

The proposed class-level Javadoc currently states, "A record class is a shallowly immutable, transparent carrier for a fixed set of values, called the record components." It also describes a "component field" as "a private static field corresponding to each component, whose name and type are the same as that of the component." The Javadoc states that these component fields are mandatory and adds that "a public accessor method corresponding to each component, whose name and return type are the same as that of the component" is also required. Further, the Javadoc adds that "implicit implementations for these members are provided" if none is expressed explicitly.

The proposed Javadoc also explains when one might choose to explicitly specify the Record constructor or accessor methods: "The primary reasons to provide an explicit declaration for the canonical constructor or accessor methods are to validate constructor arguments, perform defensive copies on mutable components, or normalize groups of components."

The draft specification for java.lang.Record adds concreteness to discussions regarding the implementation and use of Java Records. The proposed specification has already generated discussion on the amber-spec-experts mailing list. Topics discussed in relation to this specification include whether Records should prohibit cloning, whether to mention boxing of primitives in the equals method Javadoc, and whether Record.toString() and Enum.toString() should have warnings added to their Javadoc regarding changed output when a field is renamed.

The presentation of and discussion of a specification for java.lang.Record has heightened my anticipation for this feature from Project Amber.

Additional Resources

Thursday, August 15, 2019

Draft JEP to Remove Deprecated CMS Garbage Collector

In the 10 April 2017 post "Java Garbage Collectors: When Will G1GC Force CMS Out?," I discussed JEP 291 ("Deprecate the Concurrent Mark Sweep (CMS) Garbage Collector") and provided a summary of some of the feedback for and against deprecating (and ultimately removing) the Concurrent Mark Sweep (CMS) Garbage Collector (GC). Earlier this month, a draft JEP called "Remove the Concurrent Mark Sweep Garbage Collector" was created.

The "Summary" section of this draft JEP states, "Remove the Concurrent Mark Sweep (CMS) garbage collector from the set of usable garbage collection algorithms." Although the "draft" status of this JEP means it's not targeted to any specific release, the "Non-Goals" section reassures us that the draft JEP is not intended to remove CMS from any releases prior to the release in which CMS is removed. The post "RFC: JEP: Remove the Concurrent Mark Sweep Garbage Collector" states that the current plan is to target CMS removal for JDK 14.

The draft JEP states in the "Risks and Assumptions" section that this draft to remove CMS GC "might be withdrawn" if another "interested credible contributor in the community will step up for maintenance." However, in the 2+ years since JEP 291 deprecated CMS, no such "interested credible contributor" has offered to maintain CMS. Thomas Schatzl's post related to this JEP describes this better:

There has also always been the option to organize maintenance of CMS in the community, but nobody even stepped up starting to fix the long-standing existing known minor issues CMS (to get contributors to know CMS code and to give us confidence that these persons can take over maintenance of such a large component).

The draft JEP for removing CMS specifically mentions three recommend alternative garbage collectors: the now-default garbage-first (G1), Oracle-provided ZGC, and Red Hat-provided Shenandoah.

There were multiple objections to deprecating CMS when JEP 291 was under review and there are objections now to the idea of removing CMS altogether. Kirk Pepperdine has written that he and others have observed that "CMS overheads are no where near the level of those seen with G1" and they are "now recommending that customers consider Parallel GC as it offers a far better experience than G1."

It looks likely that CMS will be removed as a garbage collection option in a forthcoming JDK release (perhaps even as early as JDK 14). As Kirk Pepperdine expressed in another post on the subject, "At the end of the day, if we want CMS we're going to have to step up and do something about it." So far, it doesn't look like anyone's wanted CMS badly enough to do something about it (which might include working with their preferred JDK provider to have it supported in that provider's JDK).

Saturday, August 3, 2019

Project Valhalla Eclairs

Éclairs offer several appealing characteristics and have been described as a "trifecta" that "encompass[es] the trinity of pastry, a light and crispy shell, a silky creamy filling and intense chocolate glaze." In the OpenJDK valhalla-spec-experts mailing list, Brian Goetz's post "Collapsing the requirements" proposes the concept of Valhalla "Eclairs" and discusses how use of these "eclairs" in Valhalla can provide their own trifecta of removing the need for LV in the VM ("LPoint/QPoint distinction" or boxed projection versus unboxed projection or "LV is the null-adjunction of QV"), removing the need for V? in the language, and removing the need for null default values.

Goetz's "Summary" in the "Collapsing the requirements" summarizes the value of the proposed Valhalla eclairs and succinctly states what the proposed "eclairs" concept is (I have added the emphasis), "In one swoop, we can banish LV from the VM, V? from the language, and null-default values, by making a simple requirement: every value type is paired with an interface or abstract class 'box'. For most values, this can be automatically generated by the compiler and denoted via a well-known name (e.g., V.Box); for some values, such as those that are migrated from reference types, we can explicitly declare the box type and pick explicit names for both types."

Goetz provides more details and examples in the "Collapsing the requirements" post of "requiring that every value type have a companion interface (or abstract class) supertype." He also provides a more formal definition:

Define an envelope-class pair ("eclair") as a pair (V, I) such that:
  • V is an inline class
  • I is a sealed type
  • I permits V (and only V)
  • V <: I

Several other Goetz statements in the "Collapsing the requirements" post highlight desirable features of these proposed Valhalla eclairs and here is a sample:

  • "If every value type be a member of an eclair, we can use V when we want the flattenable, non-nullable, specializable type; and we use I when we want the non-flattenable, nullable, erased 'box'. We don’t need to denote `V?`; we can just use I, which is an ordinary, nominal type."
  • "Now, the type formerly known as `V?` is an ordinary, nominal interface (or abstract class) type. The user can say what they mean, and no magic is needed by either the language or the VM."
  • "Using the eclair wrapper also kicks the problem of erased generics down the road; if we use `Foo<I>` for erased generics, and temporarily ban `Foo<V>`, when we get to specialized generics, it will be obvious what `Foo<V>` means (their common super type will be `Foo<? extends I>`). This is a less confusing world, as then 'List of erased V' and 'List of specialized V' don’t coexist; there’s only 'List of V' and 'List of V’s Box'."

One last aspect of the "Collapsing the requirements" post that I want to highlight here is the proposal to "define a new public value class `Opt<T>` which is the value half of the eclair, and the existing Optional is the interface/abstract class half." Goetz explains that the new value class Opt is needed as a compromise (rather than adapting existing reference class Optional to be the inside of the eclair) because "existing variables of type Optional are not flattened" when today's Optional is migrated to an eclair.

Although it is a bit of a compromise to have to know, use, and differentiate another type Opt from Optional, one of the advantages of having Opt be the value portion of the eclair and having Optional be the interface/supertype portion of the eclair (and the support of boxing conversion between them) is that there is a relatively straightforward migration path between today's use of Optional and tomorrow's use of Opt. As Goetz puts it, "existing fields / arrays can migrate gradually to Opt, as they want the benefit of flattening; existing APIs can continue to truck in Optional." Goetz summarizes the migration benefit and cost of the Opt approach: "Users can migrate their fields gradually. The cost: the good name gets burned. But there is a compatible migration path from ref to value."

It's a bit disappointing to need to use "Opt" instead of "Optional", but I think most Java developers could quickly become used to applying Opt when it's available and its advantages are identified. Today's powerful Java IDEs could also identify places where existing Optional usage can be changed to Opt and warn the developer when he or she tried to introduce an Optional reference class when an Opt value class could be used.

Eclairs are tasty, tempting treats and the proposed Valhalla "eclairs" have some tempting characteristics of their own.

Monday, July 29, 2019

JEP 358: Helpful NullPointerExceptions

In the post "Better Default NullPointerException Messages Coming to Java?", I summarized background details related to a draft JEP (at that time) regarding making some types of NullPointerException's (NPE's) messages more useful. It was nice to see last week that this JEP is now a candidate JEP (JEP 358: "Helpful NullPointerExceptions"). In this post, I highlight aspects of JEP 358 that I find particularly interesting.

What is JEP 358?

JEP 358's current "Summary" succinctly describes this JEP: "Improve the usability of NullPointerExceptions generated by the JVM by describing precisely which variable was null."

The "Motivations" section of JEP 358 includes this paragraph: "The entire Java ecosystem would benefit if the JVM could give the information needed to pinpoint the source of an NPE and then identify its root cause, without using extra tooling or shuffling code around. SAP's commercial JVM has done this since 2006, to great acclaim from developers and support engineers."

Example of Community Contribution to OpenJDK

One aspect of JEP 358 that interests me is its background. The "Testing" section of JEP 358 currently states that "a predecessor implementation has been in SAP's commercial JVM since 2006 and has proven to be stable." The intention of this statement is to demonstrate that some testing has already taken place for the feature, but it also points out that SAP and its personnel are the major contributors of this proposed JDK enhancement. The authors of the JEP, Goetz Lindenmaier and Ralf Schmelter (credited as original author), are or have been affiliated with SAP. My understanding is that the SAP OpenJDK release (SapMachine) does not include this feature yet, but the commercial SAP does. This proposed JDK enhancement is a nice example of how having multiple vendors delivering implementations of OpenJDK might benefit OpenJDK for everyone in the future.

The Downside of More Details

JEP 358 addresses a potential security risk introduced by more detailed NPEs. It states that "the null-detail message may contain variable names from the source code ... if debug information is included in the class file (javac -g)" and "these have not previously been available to other programs via Java's reflection APIs."

Enabling Detailed Null Messages

The "Alternatives" section of JEP 358 outlines three reasons that one might not want the newer more detailed NPE messages (performance, security, and compatibility). It states, "The null-detail message is switched off per default and can be enabled by command-line option -XX:+SuppressCodeDetailsInExceptionMessages." That section adds that "there is no way to specify that only some NPE-raising bytecodes are of interest" (in other words, it's an all-or-nothing option). The section concludes with this statement, "We intend to enable the null-detail message by default in a future release."

More Detailed Messages Are Not for All NullPointerExceptions

JEP 358 points out types of NullPointerExceptions to which the more detailed messages will apply and contrasts that with the types to which the more detailed messages will not apply. It states (I added emphasis), "Only NPEs that are created and thrown directly by the JVM will include the null-detail message. NPEs that are explicitly created and/or explicitly thrown by programs running on the JVM are not subject to the bytecode analysis and null-detail message creation described below. In addition, the null-detail message is not reported for NPEs caused by code in hidden methods, which are special-purpose low-level methods generated and called by the JVM to, e.g., optimize string concatenation. A hidden method has no filename or line number that could help to pinpoint the source of an NPE, so printing a null-detail message would be futile." As the text states, there is no reason to attempt to provide details for "hidden methods." As for NPEs our own code explicitly throws, we have the ability to add useful context to those exceptions' messages already when we construct them.

Conclusion

The responses to this JEP have been enthusiastic and I think there are many Java developers who look forward to the availability of more detailed NullPointerException messages in future OpenJDK distributions. JDK-8218628 ("Add detailed message to NullPointerException describing what is null.") is associated with JEP 358 (JDK-8220715) and is currently associated with JDK 14 as the "Fix Version." Here's hoping for JEP 358 to be officially targeted to JDK 14!

Tuesday, July 9, 2019

Signs of JDK 14 Beginning to Appear

JDK 13 is currently in Rampdown Phase 1 (RDP 1), is scheduled to enter Rampdown Phase 2 (RDP 2) in a little over one week (on 18 July 2019), and is tentatively scheduled for General Availability on 17 September 2019. What this means, of course, is that it's time to start thinking about JDK 14! This post references and summarizes some of the online resources related to JDK 14 that are starting to appear.

Project JDK 14

The main OpenJDK JDK 14 page is the best place to start when wishing to see an overview of the release and its progress. Besides a reference to its associated specification (JSR 389: "Java SE 14 Platform"), the only other information available on this page as of this writing is a simple "Status" paragraph that references development repositories and the JDK Enhancement Proposal (JEP) process.

JDK 14 Early Access Builds

JDK 14 Early Access Builds are already available! As of this writing, the latest is Build 4 (2019/7/3). Not surprisingly, the commits in the JDK 14 early access builds at this time are fixes made to earlier JDK versions that will also be available in JDK 14.

Proposed JEP 352

The Mark Reinhold e-mail message "JEP proposed to target JDK 14: 352: Non-Volatile Mapped Byte Buffers" proposes that JEP 352 ("Non-Volatile Mapped Byte Buffers") be targeted to JDK 14.

JDK 14 Speculations

It's really far too early to recklessly speculate on what might land in JDK 14, but I won't let that stop me. It seems that the two JDK 13 "preview" JEPs [JEP 354 ("Switch Expressions") and JEP 355 ("Text Blocks")] could potentially become "permanent" in JDK 14. JEP 343 ("Packaging Tool") is another that could potentially be targeted for JDK 14.

Saturday, July 6, 2019

Project Valhalla: A First Look at LW2 Inline Types

I summarized some recent Project Valhalla LW2 "inline types" progress that was made public recently in my blog post "Valhalla LW2 Progress - Inline Types." In this post, I illustrate some of the concepts summarized in that post with code examples executed against recently released Valhalla Early Access Build jdk-14-valhalla+1-8 (2019/7/4). All code examples featured in this post are available on GitHub.

The OpenJDK Wiki page "LW2" provides an illustrative example of inline types via source code for a class called "InlineType." My example makes some minor adaptions and additions to this class and is available on GitHub as a class called InlineTypeExample. Some items that stand out immediately when reviewing this source code are the presence of the keyword inline and the presence of the ? in the Comparable's generic parameter.

The source code for my adapted InlineTypeExample class has an attempt to have the inline type class extend another class commented out because that leads to a compiler error: error: Inline type may not extend another inline type or class

Similarly, that source code also has the method that attempts to set the integer field of the inline type class commented out because that will also not compile: error: cannot assign a value to final variable

With the current Valhalla LW2 build, I'm allowed to make my inline type class Serializable and it still compiles successfully.

Another GitHub-hosted illustrative class is Lw2Demonstration that compares and contrasts characteristics of the inline type class (and instances of it) to the JDK-provided java.lang.Integer class (and its instances) and to a simple custom-built Integer wrapper (and its instances). This demonstration class calls reflective methods (some which are new to the JDK 14-based Valhalla build) on the "class" types of all three things (inline type, Integer, and custom Integer wrapper) and calls some "common" methods [toString(), equals(Object), hashCode()] on instances of all three types.

Two methods are commented out in the class Lw2Demonstration because they each attempt to perform functionality on the inline type that is not supported for inline types. One of these methods attempts to synchronize on a variable of the inline type. When attempting to compile this synchronization of an inline type, the following compiler error message is seen: error: unexpected type ... required: reference ... found: InlineTypeExample

Another attempts to assign an inline type to null. When attempting to compile this, the following error message is encountered: error: incompatible types: <null> cannot be converted to InlineTypeExample

The following method from Lw2Demonstration writes out several metadata characteristics of the class types.

/**
 * Provides metadata extracted from the provided instance of
 * {@link Class} as a single {@link String}.
 *
 * @param classToInvokeInlineMethodsOn Class for which metadata
 *    is to be extracted and returned in {@link String} format;
 *    should NOT be {@code null}.
 * @return Single string representation of metadata extracted
 *    from the provided {@link Class} instance.
 * @throws NullPointerException Thrown if {@code null} is
 *    provided for my sole parameter.
 */
public static String extractClassMetadata(final Class classToInvokeInlineMethodsOn)
{
   Objects.requireNonNull("Provided Class must be non-null to extract its metadata.");

   final String className = classToInvokeInlineMethodsOn.getSimpleName();
   final String outputPrefix = "\n" + className + ".class.";
   return outputPrefix + "getName(): " + classToInvokeInlineMethodsOn.getName()
      + outputPrefix + "getSimpleName(): " + classToInvokeInlineMethodsOn.getSimpleName()
      + outputPrefix + "getCanonicalName(): " + classToInvokeInlineMethodsOn.getCanonicalName()
      + outputPrefix + "toGenericString(): " + classToInvokeInlineMethodsOn.toGenericString()
      + outputPrefix + "getTypeName(): " + classToInvokeInlineMethodsOn.getTypeName()
      + outputPrefix + "getComponentType(): " + classToInvokeInlineMethodsOn.getComponentType()
      + outputPrefix + "isInlineClass(): " + classToInvokeInlineMethodsOn.isInlineClass()
      + outputPrefix + "isIndirectType(): " + classToInvokeInlineMethodsOn.isIndirectType()
      + outputPrefix + "isNullableType(): " + classToInvokeInlineMethodsOn.isNullableType()
      + outputPrefix + "isPrimitive(): " + classToInvokeInlineMethodsOn.isPrimitive()
      + outputPrefix + " final?: " + isFinal(classToInvokeInlineMethodsOn);
}

Some of the methods invoked on the Class instance in the previous method are new to the JDK 14-based Valhalla LW2 early access build. These include isInlineClass(), isIndirectType(), and isNullableType().

The main demonstration class Lw2Demonstration creates instances of the inline type class InlineTypeExample, of JDK-provided java.lang.Integer, and of a custom wrapper for an Integer. The demonstration then runs instances of these three classes and the class definitions through the same methods and writes out the results for each so that they can be compared and contrasted. Here is the output from running this example against the Valhalla Early Access Build mentioned at the beginning of this post.

InlineTypeExample.class.getName(): dustin.examples.valhalla.lw2.InlineTypeExample
InlineTypeExample.class.getSimpleName(): InlineTypeExample
InlineTypeExample.class.getCanonicalName(): dustin.examples.valhalla.lw2.InlineTypeExample
InlineTypeExample.class.toGenericString(): public final inline class dustin.examples.valhalla.lw2.InlineTypeExample
InlineTypeExample.class.getTypeName(): dustin.examples.valhalla.lw2.InlineTypeExample
InlineTypeExample.class.getComponentType(): null
InlineTypeExample.class.isInlineClass(): true
InlineTypeExample.class.isIndirectType(): false
InlineTypeExample.class.isNullableType(): false
InlineTypeExample.class.isPrimitive(): false
InlineTypeExample.class. final?: true
InlineTypeExample: toString(): [dustin.examples.valhalla.lw2.InlineTypeExample someIntegerValue=1]
InlineTypeExample: hashCode(): 1303372796
Inline Type Example ==: true

Integer.class.getName(): java.lang.Integer
Integer.class.getSimpleName(): Integer
Integer.class.getCanonicalName(): java.lang.Integer
Integer.class.toGenericString(): public final class java.lang.Integer
Integer.class.getTypeName(): java.lang.Integer
Integer.class.getComponentType(): null
Integer.class.isInlineClass(): false
Integer.class.isIndirectType(): true
Integer.class.isNullableType(): true
Integer.class.isPrimitive(): false
Integer.class. final?: true
Integer: toString(): 1
Integer: hashCode(): 1
Integer Type Example ==: false

IntegerWrapper.class.getName(): dustin.examples.valhalla.lw2.IntegerWrapper
IntegerWrapper.class.getSimpleName(): IntegerWrapper
IntegerWrapper.class.getCanonicalName(): dustin.examples.valhalla.lw2.IntegerWrapper
IntegerWrapper.class.toGenericString(): public class dustin.examples.valhalla.lw2.IntegerWrapper
IntegerWrapper.class.getTypeName(): dustin.examples.valhalla.lw2.IntegerWrapper
IntegerWrapper.class.getComponentType(): null
IntegerWrapper.class.isInlineClass(): false
IntegerWrapper.class.isIndirectType(): true
IntegerWrapper.class.isNullableType(): true
IntegerWrapper.class.isPrimitive(): false
IntegerWrapper.class. final?: false
IntegerWrapper: toString(): dustin.examples.valhalla.lw2.IntegerWrapper@5442a311
IntegerWrapper: hashCode(): 1413653265
Integer Wrapper Example ==: false

The output shown above demonstrates some of the advertised characteristics of the inline type. The most interesting are the focus of the following table.

CharacteristicInline Type Wrapping Integerjava.lang.IntegerCustom Integer Wrapper
Inline?truefalsefalse
Indirect?falsetruetrue
Nullable?falsetruetrue
Final?truetruefalse
== Valid for Equality?truefalsefalse
toString()Implicitly CustomizedExplicitly CustomizedUses Object's
hashCode()Implicitly CustomizedExplicitly CustomizedUses Object's

To compile and execute these examples, I needed to provide the Java compiler and launcher with some special arguments. Specifically, I compiled with --enable-preview, -Xlint:preview, and -source 14. For executing the demonstration, I passed the flag --enable-preview to the Java launcher.

The updated Valhalla Early Access Build [Build jdk-14-valhalla+1-8 (2019/7/4)] provides a convenient pre-built binary for Java developers interested in trying out Valhalla LW2 prototype inline types. This post has demonstrated some of these current LW2 inline types concepts using this build. Rémi Forax has provided many more examples on GitHub (forax/valuetype-lworld).

Friday, July 5, 2019

Valhalla LW2 Progress - Inline Types

There has been significant public-facing progress in the past few weeks (June/July 2019) for Project Valhalla's LW2 phase/prototype originally proposed in October 2018. In this post, I briefly summarize some of the messages, documents, and Valhalla Early Access Builds that have been released recently. LW2 is described in the Valhalla Wiki page as an early prototype that "[moves] inline types into the existing type system."

Draft LW2 (JVM) Spec

Dan Smith announced "a first look at a preview-feature-quality spec for values/inline classes" on the valhalla-spec-experts mailing list in a message titled "Draft LW2 spec." That message sought feedback on the initial specification and has been updated based on that feedback. This is a work-in-progress, but the URL http://cr.openjdk.java.net/~dlsmith/inline-classes is currently setup to redirect to the current version of the draft specification.

Everything related to Valhalla is a work-in-progress and might change at some point, but it is interesting to read in this draft spec the direction Valhalla is currently headed from the JVM specification perspective. Here is a sample of some of the document's more interesting points (that are subject to change):

  • "This document describes changes to the Java Virtual Machine Specification, as modified by Improved Treatment of JVM Types, to support inline classes."
  • "There are five kinds of reference types: identity class types, inline class types, array types, interface types, and the Object type. Their values are references to objects."
  • "Most reference types are nullable, meaning the special null reference (representing the absence of an object) is a value of the type. Some reference types are null-free, meaning that null is not a value of the type."
  • "An identity class type names a non-inline, non-interface class defined in a class file (excluding the special class Object). Identity class types are nullable."
  • "An inline class type names an inline class defined in a class file. For each inline class, there is a nullable and a null-free inline class type; these types share the same set of values, with the exception of null."
  • "An array type consists of a component type with a single dimension (whose length is not given by the type). The component type of an array type may be boolean, any numeric type, or any reference type. Array types are nullable."
  • "An interface type names an interface defined in a class file. Interface types are nullable."
  • "The Object type names the Object class. The Object type is nullable. ... Instances of the Object class have identity but no fields, and behave like identity class instances."
  • "The default value of a nullable reference type is null. The default value of a null-free inline class type is a reference to a class instance whose fields all have the default value for their type."

OpenJDK Valhalla Wiki: L-World

The OpenJDK Valhalla Wiki page "L-World" asks and answers the questions "What Are Inline Types?" and "What is the L-World Project?" In answering those questions, the page currently describes Inline Types as "small, immutable, identity-less types" that were "formerly known as 'Value Types'" and are now known as "inline-types." The current page describes the "L-World Project" as "a series of early prototypes for bring[ing] Inline Types to the language and JDK" that "builds on work of the previous Minimal Values Types prototype (MVT)."

OpenJDK Valhalla Wiki: LW2

The OpenJDK Valhalla Wiki page "LW2" describes the LW2 prototype as "an iteration of a previous prototype, adding further language support and JDK API support for 'inline types' (formerly referred to as 'value types')." The page indicates that the LW2 prototype is based on the JDK 14 branch and the page includes several current characteristics of "inline types," some of which I list here:

  • "Inline Types may not declare an explicit super class (except Object)."
  • "All instance fields of a inline class are implicitly final."
  • "'Indirect' projections of inline types via the '?' operator."
  • "javac automatically generates hashCode, equals, longHashCode and toString computed solely from the instance's state and not from its identity"
  • "javac allows comparison of inline type using ==, !="
  • "Inline Types can not be assigned null, null can not be cast to or compared with inline types With the exception of an 'indirect' reference using the '?' operator."
  • New methods to potentially be added to java.lang.Class: isInlineClass(), asPrimaryType(), asIndirectType() / isIndirectType(), asNullableType() / isNullableType().
  • Inline types cannot be synchronized on, cannot be serialized, cannot reference themselves as direct fields, and cannot be assigned null.

Valhalla Early Access Builds

As of this writing, the Valhalla Early Access Builds page (now renamed "Project Valhalla 'L-World Inline Types' Early-Access Builds") features "Build jdk-14-valhalla+1-8 (2019/7/4)." The page opens with the paragraph, "This is an early access build, from Project Valhalla, aimed at testing a prototype implementation of L-World Inline Types." The page also mentions, "This build is based on an incomplete version of JDK 14." As is typical for early access builds, there are limitations and disclaimers regarding use of an early access build. It sounds like there are plans to periodically update these builds "with bug fixes and performance updates."

OpenJDK Valhalla Wiki: LW2 Command Line Options/p>

For developers wanting to test out the LW2 prototype, command-line options are listed in the OpenJDK Valhalla Wiki page "LW2 Command Line Options." As stated on the page itself, it "describes the command line for the experimental Java compiler, the Java launcher and HotSpot JVM for the support of L-World LW2 Inline Types in the Valhalla project."

Meeting Minutes from 19 June 2019 Valhalla Expert Group Meeting

Karen Kinnear has provided meeting minutes from the 19 June 2019 Valhalla Expert Group Meeting that provide insight into design topics on the minds of the expert group ("nullable", "flattenable", "inlinability", "specialized generics"). The notes indicate that the LW2 specification draft discussion is not intended to finalize all design decisions.

Conclusion

There's still quite a bit of work required for "inline types" (formerly "values types") to be fully introduced to Java. However, the current focus on "inline types" and the artifacts and builds associated with this concept are evidence that progress is being made in a concrete direction.

Saturday, June 15, 2019

History and Motivations Behind Java's Maligned Serialization

Issues related to Java's serialization mechanism are well-advertised. The entire last chapter of Effective Java 1st Edition (Chapter 10) and of Effective Java 2nd Edition (Chapter 11) are dedicated to the subject of serialization in Java. The final chapter of Effective Java 3rd Edition (Chapter 12) is still devoted to serialization, but includes a new item (Item 85) that goes even further emphasize two assertions related to Java serialization:

  • "The best way to avoid serialization exploits is to never deserialize anything."
  • "There is no reason to use Java serialization in any new system you write."

In the recently released document "Towards Better Serialization," Brian Goetz "explores a possible direction for improving serialization in the Java Platform." Although the main intention of this document is to propose potential new direction for Java serialization, it is an "exploratory document only and does not constitute a plan for any specific feature." This means that it is an interesting read for the direction Java serialization might take, but there is significant value in reading this document for a summary of Java serialization as it currently exists and how we got to this place. That is the main theme of the rest of my post in which I'll reference and summarize sections of "Towards Better Serialization" that I feel best articulate the current issues with Java's serialization mechanism and why we have these issues.

Goetz opens his document's "Motivation" section with an attention-grabbing paragraph on the "paradox" of Java serialization:

Java's serialization facility is a bit of a paradox. On the one hand, it was probably critical to Java's success --- Java would probably not have risen to dominance without it, as serialization enabled the transparent remoting that in turn enabled the success of Java EE. On the other hand, Java's serialization makes nearly every mistake imaginable, and poses an ongoing tax (in the form of maintenance costs, security risks, and slower evolution) for library maintainers, language developers, and users.

The other paragraph in the "Motivation" section of the Goetz document distinguishes between the general concept of serialization and the specific design of Java's current serialization mechanism:

To be clear, there's nothing wrong with the concept of serialization; the ability to convert an object into a form that can be easily transported across JVMs and reconstituted on the other side is a perfectly reasonable idea. The problem is with the design of serialization in Java, and how it fits (or more precisely, does not fit) into the object model.

Goetz states that "Java's serialization [mistakes] are manifold" and he outlines the "partial list of sins" committed by Java's serialization design. I highly recommend reading the original document for the concise and illustrative descriptions of these "sins" that I only summarize here.

  • "Pretends to be a library feature, but isn't."
    • "Serialization pretends to be a library feature. ... In reality, though, serialization extracts object state and recreates objects via privileged, extralinguistic mechanisms, bypassing constructors and ignoring class and field accessibility."
  • "Pretends to be a statically typed feature, but isn't."
    • "Serializability is a function of an object's dynamic type, not its static type."
    • "implements Serializable doesn't actually mean that instances are serializable, just that they are not overtly serialization-hostile."
  • "The compiler won't help you" identify "all sorts of mistakes one can make when writing serializable classes"
  • "Magic methods and fields" are "not specified by any base class or interface) that affect the behavior of serialization"
  • "Woefully imperative."
  • "Tightly coupled to encoding."
  • "Unfortunate stream format" that is "neither compact, nor efficient, nor human-readable."

Goetz also outlines the ramifications of these Java serialization design decisions (see the original document for more background on each of these "serious problems"):

  • "Cripples library maintainers."
    • "Library designers must think very carefully before publishing a serializable class --- as doing so potentially commits you to maintaining compatibility with all the instances that have ever been serialized."
  • "Makes a mockery of encapsulation."
    • "Serialization constitutes an invisible but public constructor, and an invisible but public set of accessors for your internal state."
  • "Readers cannot verify correctness merely by reading the code."
    • "But because serialization constitutes a hidden public constructor, you have to also reason about the state that objects might be in based on previous versions of the code."
    • "By bypassing constructors, serialization completely subverts the integrity of the object model."
  • "Too hard to reason about security."
    • "The variety and subtlety of security exploits that target serialization is impressive; no ordinary developer can keep them all in their head at once."
  • "Impedes language evolution."
    • "Complexity in programming languages comes from unexpected interactions between features, and serialization interacts with nearly everything."
    • "Serialization is an ongoing tax on evolving the language."

Perhaps my favorite section of Goetz's "Toward Better Serialization" document is the section "The underlying mistake" because the items that Goetz outlines in this section are common reasons for mistakes in other Java code I've written, read, and worked with. In other words, while Goetz is specifically discussion how these design decisions led to problems for Java's serialization mechanism, I have (unsurprisingly) found these general design decisions to cause problems in other areas as well.

Goetz opens the section "The underlying mistake" with this statement: "Many of the design errors listed above stem from a common source --- the choice to implement serialization by 'magic' rather than giving deconstruction and reconstruction a first-class place in the object model itself." I have found "magic" code written by other developers and even myself at a later date to often be confusing and difficult to reason. I've definitely realized that clean, explicit code is often preferable.

Goetz adds, "Worse, the magic does its best to remain invisible to the reader." Invisible "magic" designs often seem clever when we first implement them, but then cause developers who must read, maintain, and change the code a lot of pain when they suddenly need some visibility to the underlying magic.

Goetz cites Edsger W.Dijkstra and writes, "Serialization, as it is currently implemented, does the exact opposite of minimizing the gap between the text of the program and its computational effect; we could be forgiven for mistakenly assuming that our objects are always initialized by the constructors written in our classes, but we shouldn't have to be".

Goetz concludes "The underlying mistake" section withe a paragraph that begins, "In addition to trying to be invisible, serialization also tries to do too much. Although Goetz is writing particularly about Java's serialization currently "serializing programs [rather than] merely serializing data," I have seen this issue countless times in a more general sense. It is tempting for we developers to design and implement code that performs every little feature we think might be useful to someone at some point even if the vast majority of (or even all currently known) users and use cases only require a simpler subset of the functionality.

Given that the objective of "Towards Better Serialization" is to "explore a possible direction for improving serialization in the Java Platform," it's not surprising that the document goes into significant detail about design and even implementation details that might influence Java's future serialization mechanism. In addition, the Project Amber mailing lists (amber-dev and amber-spec-experts) also have significant discussion on possible future direction of Java serialization. However, the purpose of my post here is not to look at the future of Java's serialization, but to instead focus on how this document has nicely summarized Java's current serialization mechanism and its history.

Although the previously mentioned Project Amber mailing lists messages focus on the potential future of Java's serialization mechanism, there are some interesting comments in these posts about Java's current serialization that add to what Goetz summarized in "Toward Better Serialization." Here are some of the most interesting:

  • Goetz's post that announced "Toward Better Serialization" states that the proposal "addresses the risks of serialization at their root" and "brings object serialization into the light, where it needs to be in order to be safer."
  • Brian Goetz post reiterates through implication that big part of problems with Java's serialization today is constructing objects without invoking a constructor: "our main security goal [is to allow] deserialization [to] proceed through constructors."
  • Stuart Marks's post states, "The line of reasoning about convenience in the proposal is not that convenience itself is evil, but that in pursuit of convenience, the original design adopted extralinguistic mechanisms to achieve it. This weakens some of the fundamentals of the Java platform, and it has led directly to several bugs and security holes, several of which I've fixed personally."
    • Marks outlines some specific examples of subtle bugs in the JDK due to serialization-related design decisions.
    • Marks outlines the explicit and specific things a constructor must do ("bunch of special characteristics") that are circumvented when current deserialization is used.
    • He concludes, "THIS is the point of the proposal. Bringing serialization into the realm of well-defined language constructs, instead of using extralinguistic 'magic' mechanisms, is a huge step forward in improving quality and security of Java programs."
  • Kevin Bourrillion's post states, "Java's implementation of serialization has been a gaping wound for a long time" and adds that "every framework to support other wire formats has always had to start from scratch."

I highly recommend reading "Towards Better Serialization" to anyone interested in Java serialization regardless of whether their primary interest is Java's current serialization mechanism or what it might one day become. It's an interesting document from both perspectives.

Monday, June 10, 2019

JDK 13: VM.events Added to jcmd

CSR (Compatibility and Specification Review) JDK-8224601 ["Provide VM.events diagnostic command"] is implemented in JDK 13 as of JDK 13 Early-Access Build #24 (dated 2019/6/6) and was added via Enhancement JDK-8224600 ["Provide VM.events command"]. The CSR's "Summary" describes this enhancement: "Add a VM.events command to jcmd to display event logs." The CSR's "Solution" states, "Add a command to jcmd to print out event logs. The proposed name is 'VM.events'."

The "Problem" section of CSR JDK-8224601 explains the value achieved from adding VM.events to the already multi-functioning jcmd: "Event logs are a valuable problem analysis tool. Right now the only way to see them is via hs-err file in case the VM died, or as part of the VM.info output."

To demonstrate jcmd's new VM.events in action, I downloaded JDK 13 Early Access Build #24 and used it to compile a simple, contrived Java application that I could then run jcmd tool delivered with that same JDK 13 Early Access Build #24 against.

The first screen snapshot shown here demonstrates using jcmd to detect the PID of the simple Java application and using jcmd <pid> help to see what jcmd options are available for that particular running Java process. The presence of VM.events is highlighted.

The next screen snapshot demonstrates applying jcmd <pid> help VM.events to see the usage (including available options) for the newly added VM.events command.

The final screen snapshot demonstrates application of jcmd's new VM.events command by showing the top (most) portion of the output from running that command without any options.

The options for the VM.events command are to narrow down results to a specified log to be printed or to limit the number of events shown. By not specifying any options, I was implicitly requesting the default of all logs and all events.

In the last displayed screen snapshot, we could see that types of JVM events rendered in the output include "compilation events", "deoptimization events", garbage collection events, classes unloaded, classes redefined, and classes loaded.

I have been a big fan of jcmd for a number of years and believe it is still generally an underappreciated command-line tool for many Java developers. The addition of the VM.events command in JDK 13 makes the tool even more useful for diagnosing a wider variety of issues.