Tuesday, February 18, 2020

Updates on Valhalla and Amber Records (Mid-February 2020)

Two posts on OpenJDK mailing lists today summarize the status of Project Valhalla and Project Amber's Records.


Brian Goetz begins his post "Valhalla -- finding the primitives" with this sentence, "I think its worth reflecting on how far we've come in Valhalla, both for the specific designs in the VM and language, and the clarity of the basic concepts." This post contains a brief history of the major design models that have been employed with Valhalla to this point and talks about the advantages and disadvantages of each of these models.

About "Q World," Goetz writes (I've added the emphasis), "The idea was that we would declare a class as either a value class or a 'regular' class, and we would derive various properties based on that." Goetz outlines these properties:

Q-World: Differentiating Properties of Regular Versus Value Classes
PropertyRegular ClassesValue Classes
Reference types?YesNo

Goetz explains that "there were many aspects which were either confusing or unsatisfying" (he provides several examples of these) about the "Q World" approach and that the approach lacked a "clean story for migration."

Goetz describes "conflating" certain "distinctions" when working with "L World":

  • "nullable vs non-nullable"
  • "pass-by-reference vs pass-by-value / flattened"
  • "reference type vs value type"
  • "identity-ful vs identity-free"

Goetz discusses how the current state of Valhalla builds on lessons learned from the "Q World" and "L World" stages. He writes:

The primitive that Valhalla introduces into class declaration is whether the instances of the class have identity or not. Traditional classes are now revealed to be "identity classes"; the new kind (identity-free) are called "inline classes". (This might not be the final word on the subject.)

Another lesson learned in Valhalla efforts is summarized:

A big AHA of the recent iterations is that it makes sense to talk about both values of inline classes and references to those values. Reference type has (almost) nothing to do with inline vs identity -- it has to do with whether the value set of the type contains values, or references.

Based on these summarized observations, Goetz writes about the "ref/val notation for the types":

"Inline" is a way of saying "identity free" when declaring classes, but it doesn't say anything (yet) about the semantics of how we represent variables on the heap or pass them on the stack. For this, we need an additional property of the type, and ref vs val seems to ideally describe what we mean."

Amber Records

In the post "[records] Summary so far," Gavin Bierman writes about "some remaining design decisions concerning records." This post outlines the following questions/design decisions and then provides partial or complete answers to the design decisions (see the post for the additional details related to each design decision).

  1. Q1. Should the distinguished supertype of records java.lang.Record be renamed?
  2. Q2. Accessibility of mandated members.
  3. Q3. Nesting.
  4. Q4. Abstract records.
  5. Q5. Deconstruction patterns.
  6. Q6. @deprecated
  7. Q7. .equals and .toString are slow.
  8. Q8. Translation strategy.
  9. Q9. Hashing.
  10. Q10. Special annotation for explicit declaration of accessors.
  11. Q11. Changing the spec of .toString method
  12. Q12. Transactional methods
  13. Q13. Factory methods instead of constructors.

Although Records are available as a preview feature in JDK 14, it's obvious much work is still being done on them. This preview status provides an opportunity for Records to be used and tested to gain feedback that can change how they are implemented. The thirteen design decisions listed above (some of which are decidedly no change and some which are possible changes) demonstrate that Records are still going to be fine-tuned.

Saturday, February 1, 2020

JDK 14/JEP 305 instanceof Pattern Matching "Smart Casts"

I generally view the presence of of the instanceof operator in Java code as a "red flag," meaning that it's not necessarily wrong to use instanceof in certain situations, but its use sometimes indicates a design issue that could be resolved in a cleaner way as described in some resources referenced at the end of this post (including resources about similar type checking functionality in languages other than Java).

Although I've seen instanceof used several times when it does not need to be, I've run into even more situations where it was not easy to avoid instanceof. This is particularly true when working with legacy code bases and certain libraries and frameworks in which I have no ability to refactor relationships between classes to support interfaces, method overriding, and other tactics that can be used to remove the need for instanceof.

A very common technique employed with instanceof is to immediately cast to the type checked in the conditional using instanceof. JEP 305 ["Pattern Matching for instanceof (Preview)"] provides an example of this common pattern and I've slightly adapted that example here:

if (object instanceof String)
    final String string = (String) object;
    // Do something with the 'string' variable typed as String

Benji Weber has posted on using reflection and on using lambda expressions to achieve Kotlin-like "instanceof smart casts." Fortunately, JDK 14 and JEP 305 bring built-in language support (albeit preview status) for this approach.

JDK 14 introduces a preview feature that allows the instanceof conditional and associated cast to be implemented completely within the conditional. The effect on the above code example is shown next:

if (object instanceof String string)
    // Do something with the 'string' variable typed as String

This preview feature is available in the JDK 14 Early Access Builds and I'm using JDK 14 Early Access Build 34 for my examples in this post.

The JEP 305 preview feature in JDK 14 is a small nicety whose advantage is more obvious in lengthy if-then-else conditional statements. The next two code listings provide a comparison of the "old way" of calling instanceof and explicitly casting to the "new preview way" of using instanceof pattern matching.

Traditional instanceof Coupled with Explicit Cast
static void makeAnimalNoises(final Object animal)
   if (animal instanceof Dog)
      final Dog dog = (Dog) animal;
   else if (animal instanceof Cat)
      final Cat cat = (Cat) animal;
   else if (animal instanceof Duck)
      final Duck duck = (Duck) animal;
   else if (animal instanceof Horse)
      final Horse horse = (Horse) animal;
   else if (animal instanceof Cow)
      final Cow cow = (Cow) animal;
   else if (animal instanceof Lion)
      final Lion lion = (Lion) animal;
      out.println("ERROR: Unexpected animal: " + animal);

JDK 14/JEP 305 Preview Feature

static void makeAnimalNoises(final Object animal)
   if (animal instanceof Dog dog)
   else if (animal instanceof Cat cat)
   else if (animal instanceof Duck duck)
   else if (animal instanceof Horse horse)
   else if (animal instanceof Cow cow)
   else if (animal instanceof Lion lion)
      out.println("ERROR: Unexpected animal: " + animal);

The full code is on GitHub and the difference between the old approach and new preview approach is available.

Because instanceof pattern matching is a preview feature, the code using this feature must be compiled with the javac flags --enable-preview and -source 14. It must be executed with java flag --enable-preview.


For more details on how this feature is implemented, see the post "RFR: JDK-8237528: Inefficient compilation of Pattern Matching for instanceof." Pattern matching support for instanceof is another Amber-provided step toward reduced boilerplate code in Java.


Resources on Issues Using instanceof

Friday, January 31, 2020

Source Code for Effective Java Third Edition Updated to Use Newer Features

Those who have read the Third Edition of Effective Java are likely aware of the source code associated with that book available on GitHub. The jbloch/effective-java-3e-source-code project has 1700+ stars and has been forked nearly 800 times as of this writing. The version of Java featured in the Third Edition of Effective Java is largely JDK 8 with some coverage of JDK 9 (see my earlier post for details on what is covered in this third edition).

Much has been added to the JDK since the publication of the Third Edition of Effective Java and many new releases have arrived with the faster 6-month cadence. Given this, I was particularly interested to see in an amber-spec-experts mailing list post that Rémi Forax has forked jbloch/effective-java-3e-source-code into the GitHub project forax/effective-java-3e-source-code that has "taken the source of Effective Java (3rd Ed) and change them to use var, switch expression, records and the instanceof with the type test pattern."

There are several things that I like about the idea of refreshing examples from Effective Java (Third Edition) to use newer features:

  • Developers can see how to apply effective Java practices using recently released features.
  • Developers can view the differences between the JDK 8/9 versions and the newer versions to see how new constructs replace older constructs and thus gain a better understanding of the newer constructs.
  • It is useful to see some of the changes when deciding whether a particular change to use a newer construct really helps with code readability in a given situation.

The main page for the forked forax/effective-java-3e-source-code (README.md) states, "The source code have been updated to use new constructs available since Java 9, the version used by the 3rd edition." That page then provides bullets on the types of new constructs applied to the source code with links to each new construct's associated JDK Enhancement Proposal (JEP).

As of this writing, Commit 275eef87e4661f7f1edc41f4730cecf7a1096a97 is the main commit of interest. It covers changes to 113 files. I'll call out a few specific changes here to illustrate the types of changes applied (some of which are to apply preferred constructs that were available even before JDK 9):


The ability to view changes to the original source code associated with the Third Edition of Effective Java to accommodate new language constructs is highly useful in terms of learning about the new constructs and how they relate to or replace old constructs and in deciding if the differences are desirable in different situations.

Thursday, January 30, 2020

Recent Valhalla News (Late January 2020)

There has been significant discussion, design, implementation, and testing related to aspects of OpenJDK Project Valhalla in recent months, but a couple things have struck me as particularly interesting over the last few days in terms of Valhalla progress. I briefly summarize those here.


Latest on JDK Release Associated with Valhalla

Frédéric Parain has posted "Valhalla EG 20200129" on the valhalla-spec-experts mailing list. This posting constitutes the meeting notes from the Valhalla Expert Group Meeting on 29 January 2020.

There are several interesting discussion points in these Valhalla Expert Group Meeting notes, but none may be more interesting than the answer to the question, "Is there a JDK release targeted for Valhalla?" In response to that question, the notes provide "bullets" that constitute Brian Goetz's response:

  • "too early to say"
  • "VM work is consolidating"
  • "language work still needs significant work before having a road-map"
  • "first release won't have all the features but have to have further additions in mind"

Although there's still much work required for Valhalla, I interpret these comments in a positive manner, especially when compared to responses to a similar question in months and years past.


The Future of Integer

The previously referenced Valhalla Expert Group Meeting Notes for 29 January 2020 include this note attributed to Brian Goetz: "Retrofitting of Integer: seemed impossible, but now seems more feasible."

I am not certain of the context of those notes, but reading these notes did make me think of Brian Goetz's discussion in the December 2019 edition of "State of Valhalla" regarding "Integer and friends." In particular, Goetz discusses migrating primitives by changing the "wrapper types (such as Integer and friends) to be interfaces" so that "these interfaces will become the reference projection for the primitives." Or, as Rafael Winterhalter articulates it, "inline class int implements Integer."

Related to this, Goetz wrote in the 6 December 2019 post "Long-awaited State of the Values docs" (I added emphasis):

The other major AHA buried in this document is the realization that the language and VM models need not align 100%. LWorld is the right VM model, but we can use it as a _translation target_ rather than our language model. This allows the primitives to blossom into full inline classes. In the new world, we have inline classes and identity classes, primitives are mostly just inline classes, and the relationship between inlines and identity classes is isomorphic to the current relationship between primitives and classes.

Goetz also discusses "migrating Integer and friends to be inline-friendly" in the post "Superclasses for inline classes."


Optional and Integer Will Be Interfaces Instead of Classes

The December 2019 "State of Valhalla" document explains that the Optional class of today will be changed to an interface and that the "wrapper types Integer and friends" will be migrated "to be interfaces, using the same techniques as with migrating Optional."

One downside to this conversion of Optional, Integer, and friends from classes to interfaces is that comparisons of getClass() calls in existing code against these literal types will break (because the classes implementing the interfaces will be returned in the future instead of the interface name). To deal with this, Bug JDK-8237074 ("[lworld] Compiler should emit a warning if code compares the result of getClass() against the class literal for an interface") has been written and addressed in the repo-valhalla repository. Any legacy code that compares the results of a getClass() call to a literal such as java.lang.Integer.class or java.util.Object.class may not work correctly with these changes, but at least there will be a warning.


New Edition of "State of Valhalla" Coming Soon?

It sounds possible that there will soon be an updated version of "State of Valhalla." In the 29 January 2020 Vallhalla EG Meeting notes, there are these notes attributed to Brian Goetz:

  • "working on reference projection as an interface or an abstract class"
  • "possible update next week, working on simplification"
  • "would lead to rewrite of 'State of Valhalla'"


Interfaces IdentityObject and InlineObject Introduced

In the December 2019 edition of "State of Valhalla," Goetz introduced "new top interfaces" IdentityObject and InlineObject:

To distinguish between inline and identity classes at compile time and runtime, we introduce a pair of restricted interfaces: IdentityObject and InlineObject. Inline classes implicitly implement InlineObject; identity classes implicitly implement IdentityObject.

JDK-8237069 ("[lworld] Introduce and wire-in the new top interfaces") captures the need to write these interfaces and Srikanth Adayapalam has added these new interfaces to repo-valhalla. Adayapalam describes these changes in the post "[LWorld] Definition and wiring in of the new top interface types.," which includes a link to the changeset. This changeset includes the new interfaces IdentityObject and InlineObject. From this new source, we can see that both interfaces are in the java.lang package and both interfaces have no methods defined ("marker" or "tagging" interfaces similar to java.io.Serializable).

The Javadoc for IdentityObject simply states, "A restricted interface implemented by all identity objects." Similarly, the Javadoc for InlineObject simply states, "A restricted interface implemented by all inline objects."



Project Valhalla continues to progress and developments in recent months create a feeling that Valhalla design and implementation is solidifying.

Friday, December 20, 2019

General, Safe, and Deterministic Foreign Memory Access in JDK 14

In the post "JDK 14 Rampdown: Build 27," I summarized the numerous JDK 14-targeted features newly available with JDK 14 Early Access Build #27. There is already another JDK 14 Early Access Build available and this one [Build 28 (2019/12/18)] includes one particularly interesting feature: Enhancement JDK-8234049 ["Implementation of Memory Access API (Incubator)"]. This is the implementation of JEP 370 ["Foreign-Memory Access API (Incubator)"], which is summarized as, "Introduce an API to allow Java programs to safely and efficiently access foreign memory outside of the Java heap."

JEP 370 is intended to achieve a "foreign memory API" that provides "generality", "safety", and "determinism" (with each of these means described in further detail in the JEP). The JEP also states that this foreign memory API is intended as an alternative to currently used approaches (java.nio.ByteBuffer and sun.misc.Unsafe). Downsides and limitations of both of these currently available alternates are discussed in the JEP.

The JEP points out that libraries "access foreign memory" to "avoid the cost and unpredictability associated with garbage collection," to "share memory across multiple processes," and to "serialize and deserialize memory content by mapping files into memory." It adds that "the Java API does not provide a satisfactory solution for accessing foreign memory."

JEP 370's "Description" section introduces the implementation of the foreign memory access API. The section begins, "The foreign-memory access API introduces three main abstractions: MemorySegment, MemoryAddress and MemoryLayout." The section then describes these three main abstractions: "A MemorySegment is used to model a contiguous memory region with given spatial and temporal bounds. A MemoryAddress can be thought of as an offset within a segment. Finally, a MemoryLayout is a programmatic description of a memory segment's contents."

After several detailed examples of using the foreign memory access, the JEP's "Description" section states, "The foreign-memory access API will initially be provided as an incubating module, named jdk.incubator.foreign, in a package of the same name."

Additional background details related to JEP 370 are available in OpenJDK mailing list messages. One key thread starts with the message "RFR JDK-8234049: Implementation of Memory Access API (Incubator)." The CSR for this JEP [JDK-8234050: Implementation of Memory Access API (Incubator)] presents alternate useful descriptions. Javadoc-based documentation for the foreign memory access API is also temporarily available at http://cr.openjdk.java.net/~mcimadamore/panama/memaccess_javadoc/jdk/incubator/foreign/package-summary.html.

The JEP 370 implementation is included in the JDK 14 Early Access Build #28 and available to try.

Saturday, December 14, 2019

JDK 14 Rampdown: Build 27

Mark Reinhold's recent post "JDK 14 is now in Rampdown Phase One" announced that "we are now in Rampdown Phase One" and that "the overall feature set is frozen." JDK 14 Early Access Build #27 (2019/12/12) is a hefty build. In this post, I summarize some of the changes associated with JDK 14 Early Access Build 27 that are particularly interesting to me.

One might say that JDK 14 Early Access Build 27 is the "records build." This build is full of updates related to the implementation of the records preview feature (JEP 359). Here are some of the changes associated with this "records build" filled with JEP 359 implementation changes (note that there are already bugs written and fixed for records):

  • JEP 359 JDK-8222777: Records (Preview)
  • Sub-Task JDK-8227113: Specification for java.lang.Record
  • Sub-Task JDK-8225053: Preview APIs support for records
  • Sub-Task JDK-8233526: JVM support for records
  • Sub-Task JDK-8225054: Compiler implementation for records
  • Sub-Task JDK-8225057: Java Language Specification changes for Records
  • Sub-Task JDK-8225058: JVM Specification changes for Records
  • Sub-Task JDK-8225055: Javadoc for records
  • Sub-Task JDK-8225052: javax.lang.model support for records
  • Sub-Task JDK-8226314: com.sun.source support for records
  • Enhancement JDK-8235359: Simplify method Class.getRecordComponents()
  • Bug JDK-8235550: Clarify record reflective support specification
  • Bug JDK-8235369: Class.toGenericString need to be updated for records
  • Bug JDK-8235446: confusing error message for records with no parens
  • Bug JDK-8234101: Compilation error thrown when repeating annotation is used on record component
  • Bug JDK-8235474: JShell does not handle records properly

Although JEP 359/Records (Preview) will likely dominate interest of many Java developers in the JDK 14 Early Access Build 27, there are other interesting changes in this build that are summarized here (with links to more details and quotes from the change descriptions):

In addition to the changes that I called out above, other changes associated with JDK 14 Early Access Build 27 include changes to areas that have changes in many JDK 14 builds (Shenandoah garbage collector, Z garbage collector [ZGC], Java Flight Recorder, and the Java packaging tool).

The JDK 14 Early Access Build 27 Release Notes state that "SSLv2Hello and SSLv3 have been removed from the default enabled TLS protocols." The release notes add that "If a client or server still needs to use the SSLv3 protocol they can do so by enabling it via the jdk.tls.client.protocols or jdk.tls.server.protocols system properties or with the SSLSocket.setEnabledProtocols(), SSLServerSocket.setEnabledProtocols() and SSLEngine.setEnabledProtocols() APIs."

I am excited about Records in Java 14 and look forward to playing with them via the OpenJDK 14 Early Access Build 27.

Monday, November 18, 2019

JDK 14: Records, Text Blocks, and More

Today's Mark Reinhold post "JEPs proposed to target JDK 14: 305, 343, 359, 366, & 368" proposes targeting five more JEPs to JDK 14. Some highly anticipated features are among this group, although all but one are proposed to be in "preview" or "incubating":

  • JEP 305: Pattern Matching for instanceof (Preview)
  • JEP 343: Packaging Tool (Incubator)
  • JEP 359: Records (Preview)
  • JEP 366: Deprecate the ParallelScavenge + SerialOld GC Combination
  • JEP 368: Text Blocks (Second Preview)

The Java SE 13 Java Language Updates document describes preview features (JEP 12): "A preview feature is a new feature whose design, specification, and implementation are complete, but which is not permanent, which means that the feature may exist in a different form or not at all in future JDK releases."

JEP 11 ("Incubator Modules") describes "incubator modules" as "a means of putting non-final APIs and non-final tools in the hands of developers, while the APIs/tools progress towards either finalization or removal in a future release."

Four of the five JEPs proposed today for targeting JDK 14 are "preview" or "incubator" and so will be subject to potentially "not exist .... at all in future JDK releases" or potential "removal in a future release." However, the opportunity to take these features for a spin is welcome and there seems likely that these features will eventually become permanent even if they have some modifications. We could have these JEPs officially targeted for JDK 14 by Thanksgiving!

Additional Resources