Finalizing instanceof Pattern Matching and Records in JDK 16

Gavin Bierman has recently posted two approachable posts regarding instanceof pattern matching and records on the amber-spec-experts mailing list and both posts are thorough but succinct. I recommend that anyone interested in the latest state of these two Project Amber initiatives read these posts: "Finalizing in JDK 16 - Pattern matching for instanceof" and "Finalizing in JDK 16 - Records". Even for those not interested in the details, it is probably interesting to see the likelihood of instanceof pattern matching and records being finalized in JDK 16.

Pattern Matching instanceof Operator

In "Finalizing in JDK 16 - Pattern matching for instanceof," Bierman writes: "In JDK 16 we are planning to finalize two JEPs: Pattern matching for instanceof and Records." About instanceof pattern matching, Bierman states, "Adding conditional pattern matching to an expression form is the main technical novelty of our design of this feature."

The "Finalizing in JDK 16 - Pattern matching for instanceof" post talks about advantages of the instanceof pattern matching approach. Advantages discussed in the post include refactoring of a "common programming pattern" (checking for instanceof and then explicitly casting the thing checked with instanceof to the type it was checked against) to smaller code that is more readable. Another discussed advantage is the ability "to compactly express things with expressions that are unnecessarily complicated using statements."

Bierman also discusses scope issues, local pattern variable "poisoning", and how these will be dealt with using "flow scoping." About this, Bierman writes:

Java already uses flow analysis - both in checking the access of local variables and blank final fields, and detecting unreachable statements. We lean on this concept to introduce the new notion of flow scoping. A pattern variable is only in scope where the compiler can deduce that the pattern has matched and the variable will be bound. This analysis is flow sensitive and works in a similar way to the existing analyses. ... The motto is "a pattern variable is in scope where it has definitely matched". This is intuitive, allows for the safe reuse of pattern variables, and Java developers are already used to flow sensitive analyses.

The post on instanceof pattern matching ends with a discussion of the "Swiss cheese property" ("unfortunate interaction of flow scoping and shadowing [local pattern variables shadowing a field declaration]") and expresses the hope that IDEs will help developers to deal with these unfortunate interactions.

Records

Bierman's post "Finalizing in JDK 16 - Records" provides some nice summary descriptions related to Records. The first is, "Record classes are a special kind of class that are used primarily to define a simple aggregate of values." This is immediately followed by, "Records can be thought of as nominal tuples; their declaration commits to a description of their state and given that their representation, as well as all of the interesting protocols an object might expose -- construction, property access, equality, etc -- are derived from that state description."

This post covers the "implicit declaration of a number of members" of a Record from the Record's "state description." The post also provides a brief overview of why a developer might "explicitly [provide] a canonical constructor" and how to do so. About this, Bierman writes, "The intention of a compact constructor declaration is that only validation and/or normalization code need be given in the constructor body; the remaining initialization code is automatically supplied by the compiler."

"Finalizing in JDK 16 - Records" also discusses canonical constructor accessibility, @Override, and general annotations related to Records. The post concludes by mentioning that Records can be declared locally and explains how this is an advantage for an expected common use case of Records: "Records will often be used as containers for intermediate data within method bodies. Being able to declare these record classes locally is essential to stop proliferation of classes."

Conclusion

The two Project Amber initiatives of instanceof pattern matching and records will be welcome additions to the Java programming language. Although these features have been available in versions of the JDK before JDK 16 as preview features, it will be their finalization (hopefully in JDK 16) that will enable many more developers to apply them in production code.

Potential Inlined JDK Classes

Earlier this week, Dan Smith posted "JEP draft: Identity Warnings for Inline Class Candidates" on the valhalla-spec-experts mailing list. The purpose of this post was to introduce a draft JEP ("Identity Warnings for Inline Class Candidates") associated with JDK-8249100.

The main intent of this JEP is to "provide warnings at compile time and runtime for behaviors that will change when certain classes of the Java SE APIs become inline classes." Although I think it's a good idea to have these warnings to help Java developers recognize coming change and opportunities for change, I find the background details provided to in this draft JDK Enhancement Proposal to be as interesting as the end objective. This draft JEP consolidates discussions and decisions discussed among many different threads and messages on the mailing lists in a single easily referenced location.

Inline Classes in Java

The "Motivation" section of the draft JEP states that inline classes "declare their instances to be identity-free and capable of 'inline' or 'flattened' representations, where instances can be copied freely between memory locations and encoded using solely the values of the instances' fields." That same section also outlines two operations (monitoring/locking with synchronized and instantiation with new) that will not be supported by the identity-less inline classes. Because certain operations will not be allowed on inline classes and because some current JDK classes that may be used with these operations may be transitioned to inline classes at some point, it is desirable to have this draft JEP's proposed warnings. These warnings alert developers to their current uses of these inline class candidate JDK classes in conjunction with these operations that will not be supported on inline classes in the future (eventually an error will result).

For more background on inline classes, see the March 2020 version of the State of Valhalla document, in which Brian Goetz makes these observations:

• "Inline classes are heterogeneous aggregates that explicitly disavow object identity."
• Inline classes "must be immutable and cannot be layout-polymorphic."
• "Inline classes have some restrictions compared to ordinary (identity) classes; they are final, their fields are final, and their ability to participate in inheritance is limited."
• "Inline classes can use most mechanisms available to classes: methods, constructors, fields, encapsulation, type variables, annotations, etc"
• "The slogan" for inline classes "is Codes like a class, works like an int."
• "Inline types could equally be thought of as 'faster classes' or 'user-definable primitives'."

Identified JDK Inline Class Candidates

The "Description" section of the draft JEP lists several bullets of classes and groups of classes that are "likely to become inline classes." These candidate JDK inline classes are listed, as of this writing, as follows:

• Primitive Wrapper Classes: java.lang.Byte, java.lang.Short, java.lang.Integer, java.lang.Long, java.lang.Float, java.lang.Double, java.lang.Boolean, java.lang.Character
• Optional Classes: java.util.Optional, java.util.OptionalInt, java.util.OptionalLong, java.util.OptionalDouble
• Java Time Classes: java.time.Instant, java.time.LocalDate, java.time.LocalTime, java.time.LocalDateTime, java.time.ZonedDateTime, java.time.ZoneId, java.time.OffsetTime, java.time.OffsetDateTime, java.time.ZoneOffset, Duration, java.time.Period, java.time.Year, java.time.YearMonth, java.time.MonthDay, and select classes in java.time.chrono package.
• Collections Returned from Collection Factories: java.util.List.of, java.util.List.copyOf, java.util.Set.of, java.util.Set.copyOf, java.util.Map.of, java.util.Map.copyOf, java.util.Map.ofEntries, and java.util.Map.entry
• java.lang.Runtime.Version
• java.lang.ProcessHandles Implementations
• java.lang.constant.ConstantDesc Implementations (except java.lang.String)
• Select Classes in the jdk.incubator.foreign Module

JDK Classes That Will Not Be Inlined

There are some JDK-provided classes that, at first thought, might seem like candidates to be inline classes, but turn out to have characteristics that make them unlikely to ever be transitioned. These include java.lang.String and java.math.BigInteger and java.math.BigDecimal.

Conclusion

The recently proposed draft JEP discussed in this post provides a centralized location for updated background details related to the currently planned inline classes feature being discussed and worked as part of Project Valhalla. Keeping in mind that this is a draft JEP about an in-work feature, this draft JEP provides insight into how inline classes will change how we write and use Java classes.

jcmd Increasingly the Single JDK Command-line Tool to Rule Them All

What would you miss from the JDK command line tools jinfo, jmap, and jstack if those tools were no longer available?

I've been a fan of the command-line tool jcmd for a long time. I like that it combines features and functions of many other JDK command-line tools in a single tool. I also like that I like its interactive nature that supports my asking the tool what it's capable of for a given JVM process, meaning I don't have to remember exact options when applying its different functions.

Since its inception, jcmd has been taking on functionality from various other command-line JDK tools. In a recent post on the OpenJDK jdk-dev mailing list, Stephen Fitch announced a survey intended to "gather more information and help us evaluate and understand how others are using [three 'j* tools' jinfo, jmap, jstack] in the JDK." The surveyors are "specifically interested in your use-cases and how these tools are effective for you in resolving JVM issues."

Fitch explains the need for this survey:

We are considering deprecation and (eventual) removal of the jinfo, jmap, jstack - (aka “j* tools”) and building out a future foundation for some aspect of serviceability on jcmd, however we don’t have a lot of data about how how these tools are used in practice, especially outside of Oracle.

I'm encouraged that it sounds likely that jcmd will continue to add functionality under its single approach that was formerly available with different tools.

The survey will be open through 15 July 2020 and is an opportunity for Java developers to share their experiences and use cases with the JDK command line tools jinfo, jmap, and jstack so that it's more likely those use cases will be supported in jcmd (if not already supported) in the future. Fitch's post presents the URL for the survey: https://www.questionpro.com/t/AQk5jZhiww

Select jcmd Posts

Here are some of my previous posts regarding jcmd (I have used jcmd in numerous other posts to demonstrate various output as well):

• jcmd: One JDK Command-Line Tool to Rule Them All
• jcmd, Circa JDK 11
• JavaOne 2012: Diagnosing Your Application on the JVM
• Programmatic jcmd Access
• jhsdb Functions covered by jcmd
• [JDK 11] Class Loader Hierarchy Details Coming to jcmd
• JDK 13: VM.events Added to jcmd
• Determining the Active HotSpot Garbage Collector

Better NullPointerException Messages Automatic in JDK 15

I discussed long-awaited and highly appreciated improvements to NullPointerException (NPE) messages in the posts "Better Default NullPointerException Messages Coming to Java?" and "Better NPE Messages in JDK 14". When this JEP 358-driven feature was added to JDK 14, a developer who wanted to benefit from these more insightful NPE messages needed to explicitly state that desire by passing the argument -XX:+ShowCodeDetailsInExceptionMessages to the Java launcher (java).

The JDK 15 Early Access Build #29 was released this week and makes the use of better NPE messages automatic. The release notes associated with this early access build state: "The default of the flag ShowCodeDetailsInExceptionMessages was changed to 'true'. The helpful NullPointerException messages of JEP 358 are now printed by default. The messages contain snippets of the code where the NullPointerException was raised."

The next screen snapshot demonstrates that the helpful NullPointerException details are provided automatically with JDK 15 Early Access Build #29.

The release notes also point out that one potential risk of having the "helpful" NullPointerException messages written by default is the accidental exposure of sensitive details. The release notes warn: "App deployers should double check the output of their web applications and similar usage scenarios. The NullPointerException message could be included in application error messages or be displayed by other means in the app. This could give remote attackers valuable hints about a potential vulnerable state of the software components being used."

The next screen snapshot demonstrates that automatic presentation of helpful NullPointerException details can be disabled via the use of java launcher option -XX:-ShowCodeDetailsInExceptionMessages (and the old -XX:+ShowCodeDetailsInExceptionMessages is still supported even though this is now the default):

One on the interesting consequences of the JDK-8233014 change to make helpful NullPointerException messages enabled by default is that there will undoubtedly be some Java developers pleasantly surprised when they upgrade to JDK 15 and start seeing suddenly far more useful messages when encountering the ubiquitous NullPointerException.

Tycoon: Ransomware Targeting Java's JIMAGE on Multiple Platforms

The Blackberry Research and Intelligence Team and KPMG's UK Cyber Response Services Team have reported "Threat Spotlight: Tycoon Ransomware Targets Education and Software Sectors." This report outlines the "multi-platform Java ransomware targeting Windows and Linux that has been observed in-the-wild since at least December 2019" and which they've called "Tycoon."

The report provides a high level description of how the Tycoon ransomware is executed: "Tycoon ransomware comes in form of a ZIP archive containing a Trojanized Java Runtime Environment (JRE) build. The malware is compiled into a Java image file (JIMAGE) located at lib\modules within the build directory." The report describes the "sparsely documented" JIMAGE "file format file format that stores custom JRE images which is designed to be used by the Java Virtual Machine (JVM) at runtime." Additional high-level overviews of the JIMAGE file format can be found in JIMAGE - Java Image File Format, How Modules Are Packaged in Java 9, What Is a Custom Runtime Image in Java 9?, So what is a .jimage?, Alan Bateman's description, and slide 49 of JDK 9 Java Platform Module System. The JIMAGE format was introduced with JDK 9 modularity (Project Jigsaw).

Alan Bateman (owner of JEP 220 ["Modular Run-Time Images"])has explained why it's difficult to find documentation on the JIMAGE format: "The format is deliberately not documented and best to assume it will change and evolve over time as it inhales new optimizations." The jdk.jlink module documentation provides very brief mention of the jimage command-line tool and mentions that it is used "for inspecting the JDK implementation-specific container file for classes and resources." It also points out that there is no API for accessing jimage, which is different than for the module's two other tools (jlink and jmod). Similarly, there is no jimage tool reference on the "Java Development Kit Version 14 Tool Specifications" page even though jlink, jmod, and many other tools are featured there.

The previously mentioned report states that "the ransomware is triggered by executing a shell script that runs the Main function of the malicious Java module using the java -m command." The report also talks about Tycoon using Image File Execution Options Injection with Windows registry and then appearing to target Windows and Linux: "The malicious JRE build contains both Windows and Linux versions of this script." This reports adds that Tycoon's "Trojanized Java Runtime Environment (JRE) build" exists as a ZIP file placed in lib/module.

The "Conclusions" section of "Tycoon Ransomware Targets Education and Software Sectors" makes some interesting conclusions based on research observations. In particular, I find it interesting to read why they believe Tycoon in a targeted attack. The authors also point out, "This is the first sample we've encountered that specifically abuses the Java JIMAGE format to create a custom malicious JRE build."

I recommend reading the report "Threat Spotlight: Tycoon Ransomware Targets Education and Software Sectors" directly, but for those interested in summaries and others' observations related to this report, the following may be of interest.

• A new Java-based ransomware targets Windows and Linux (TechCrunch, TechCrunch)
• This new ransomware is targeting Windows and Linux PCs with a 'unique' attack (ZDNet)
• Newly discovered Tycoon ransomware leverages obscure Java image format (Silicon Angle)
• Multi-Platform 'Tycoon' Ransomware Uses Rare Java Image Format for Evasion (SecurityWeek)
• There's a new Java ransomware family on the block (CyberScoop)

The discovery of Tycoon is likely to bring significantly more attention to JIMAGE than ever before.

Octopus Scanner: Java Build Tools and Malware

Alvaro Muñoz recently posted "The Octopus Scanner Malware: Attacking the open source supply chain" on the GitHub Security Lab site. I found this post to be interesting for a number of reasons, including its detailed coverage of how the Octopus Scanner malware works and how it was discovered, how the GitHub Security Incident Report Team (SIRT) went about addressing it, how it affected a popular Java IDE, and how GitHub works to detect and address risks to open source software deployed on its site.

Muñoz calls Octopus Scanner "an OSS supply chain malware" and writes that 26 GitHub-hosted open source projects "were backdoored by this malware." In the post, Muñoz describes in detail how Octopus Scanner works. The post in its entirety is worth reading, but here are some highlights describing Octopus Scanner:

• "Octopus Scanner malware is only interested in the pre-jar and post-jar tasks" of "a NetBeans project build."
• "The malware disguises itself as an ocs.txt file, but we can easily determine is is actually a Java Archive (JAR) file."
• "The malware also infected any JAR files that were available in the project, such as dependencies - not necessarily just build artifacts."
• "The octopus.dat payload is the binary that actually performs the NetBeans build infections."
• "cache.dat is responsible for backdooring the built classes so that when these classes get executed, they will infect the underlying system."
• Octopus Scanner is designed to target "UNIX-like systems", MacOS, and Windows.

I found the highly detailed approach to diagnosing Octopus Scanner's behavior in the GitHub post to be fascinating and insightful reading. It was especially insightful to see the tools and methods used to understand better how Octopus Scanner behaves. For example, they used ClassFileTransformer and "a Bytecode manipulation library such as Javassist or ByteBuddy to inject our analysis code" into the "class responsible for decrypting the blob ... right before it actually gets loaded into the JVM."

Besides the interesting details of how Octopus Scanner works and how it was discovered and researched, other interesting insights in this GitHub post are related to the risks open source builds face. Muñoz writes, "Infecting build artifacts is a means to infect more hosts since the infected project will most likely get built by other systems and the build artifacts will probably be loaded and executed on other systems as well." Muñoz adds, "In an OSS context, it gives the malware an effective means of transmission since the affected projects will presumably get cloned, forked, and used on potentially many different systems. The actual artifacts of these builds may spread even further in a way that is disconnected from the original build process and harder to track down after the fact."

Muñoz opens the post and concludes the post with some discussions about this and other attempts at sabotaging open source products and their builds. One chilling thought is included in the Conclusion: "Since the primary-infected users are developers, the access that is gained is of high interest to attackers since developers generally have access to additional projects, production environments, database passwords, and other critical assets. There is a huge potential for escalation of access, which is a core attacker objective in most cases."

Exact Absolute Integral Numbers in JDK 15

JDK 15 Early Access Build b18 introduced new methods to the Math and StrictMath classes that will throw ArithmeticException on provided values outside the range supported by the methods without overflow. These methods bring to the concept of "absolute value" in Java what the methods such as Math.addExact, Math.subtractExact, and Math.multiplyExact have brought to basic arithmetic functions.

Prior to JDK 15, Integer.MIN_VALUE and Long.MIN_VALUE caused the respective methods Math.abs and StrictMath.abs to return the same negative number as represented by the MIN_VALUE largest negative possible value. This behavior is described in the Javadoc documentation for the affected methods and is demonstrated by the code shown below (and available on GitHub):

Demonstrating Pre-JDK 15 Absolute Value Methods

/**
 * Demonstrates "absExact" methods added to {@link Math}
 * and {@link StrictMath} with JDK 15 Early Access Build b18
 * (JDK-8241374: https://bugs.openjdk.java.net/browse/JDK-8241374).
 */
public class AbsoluteExactness
{
   public void demonstrateMathAbsInteger(final int integer)
   {
      out.println("Math.abs(" + integer + "): " + Math.abs(integer));
   }

   public void demonstrateMathAbsLong(final long longNumber)
   {
      out.println("Math.abs(" + longNumber + "L): " + Math.abs(longNumber));
   }

   public void demonstrateStrictMathAbsInteger(final int integer)
   {
      out.println("StrictMath.abs(" + integer + "): " + StrictMath.abs(integer));
   }

   public void demonstrateStrictMathAbsLong(final long longNumber)
   {
      out.println("StrictMath.abs(" + longNumber + "L): " + StrictMath.abs(longNumber));
   }

   public static void main(final String[] arguments)
   {
      final AbsoluteExactness instance = new AbsoluteExactness();

      // Demonstrate pre-JDK 15 Math/StrictMath "abs" functions on minimum values.
      instance.demonstrateMathAbsInteger(Integer.MIN_VALUE+1);
      instance.demonstrateMathAbsInteger(Integer.MIN_VALUE);
      instance.demonstrateMathAbsLong(Long.MIN_VALUE+1);
      instance.demonstrateMathAbsLong(Long.MIN_VALUE);
      instance.demonstrateStrictMathAbsInteger(Integer.MIN_VALUE+1);
      instance.demonstrateStrictMathAbsInteger(Integer.MIN_VALUE);
      instance.demonstrateStrictMathAbsLong(Long.MIN_VALUE+1);
      instance.demonstrateStrictMathAbsLong(Long.MIN_VALUE);
   }
}

When the above code is executed, the following output is written:

Math.abs(-2147483647): 2147483647
Math.abs(-2147483648): -2147483648
Math.abs(-9223372036854775807L): 9223372036854775807
Math.abs(-9223372036854775808L): -9223372036854775808
StrictMath.abs(-2147483647): 2147483647
StrictMath.abs(-2147483648): -2147483648
StrictMath.abs(-9223372036854775807L): 9223372036854775807
StrictMath.abs(-9223372036854775808L): -9223372036854775808

This output demonstrates that the maximum negative allowable value in the int and long ranges leads to that same value being returned from the appropriate abs method on Math and StrictMath.

JDK 15 Early Access Build b18 introduces absExact methods that throw ArithmeticException in this case instead of returning the negative value. This is demonstrated with the following code (also available on GitHub):

Demonstrating JDK 15-Introduced absExact Methods

public class AbsoluteExactness
{
   public void demonstrateMathAbsExactInteger(final int integer)
   {
      try
      {
         out.println("Math.absExact(" + integer + "): " + Math.absExact(integer));
      }
      catch (ArithmeticException exception)
      {
         err.println("Math.absExact(" + integer + "): " + exception);
      }
   }

   public void demonstrateMathAbsExactLong(final long longNumber)
   {
      try
      {
         out.println("Math.absExact(" + longNumber + "L): " + Math.absExact(longNumber));
      }
      catch (ArithmeticException exception)
      {
         err.println("Math.absExact(" + longNumber + "L): " + exception);
      }
   }

   public void demonstrateStrictMathAbsExactInteger(final int integer)
   {
      try
      {
         out.println("StrictMath.absExact(" + integer + "): " + StrictMath.absExact(integer));
      }
      catch (ArithmeticException exception)
      {
         err.println("StrictMath.absExact(" + integer + "):" + exception);
      }
   }

   public void demonstrateStrictMathAbsExactLong(final long longNumber)
   {
      try
      {
         out.println("StrictMath.absExact(" + longNumber + "L): " + StrictMath.absExact(longNumber));
      }
      catch (ArithmeticException exception)
      {
         err.println("StrictMath.absExact(" + longNumber + "L): " + exception);
      }
   }

   public static void main(final String[] arguments)
   {
      final AbsoluteExactness instance = new AbsoluteExactness();

      // Demonstrate JDK 15-introduced Math/StrictMath "absExact" functions
      // on minimum values.
      instance.demonstrateMathAbsExactInteger(Integer.MIN_VALUE+1);
      instance.demonstrateMathAbsExactInteger(Integer.MIN_VALUE);
      instance.demonstrateMathAbsExactLong(Long.MIN_VALUE+1);
      instance.demonstrateMathAbsExactLong(Long.MIN_VALUE);
      instance.demonstrateStrictMathAbsExactInteger(Integer.MIN_VALUE+1);
      instance.demonstrateStrictMathAbsExactInteger(Integer.MIN_VALUE);
      instance.demonstrateStrictMathAbsExactLong(Long.MIN_VALUE+1);
      instance.demonstrateStrictMathAbsExactLong(Long.MIN_VALUE);
   }

The output from this code is shown next and demonstrates the clear exception message that is thrown when the MIN_VALUE is passed to the absExact methods.

Math.absExact(-2147483647): 2147483647
Math.absExact(-2147483648): java.lang.ArithmeticException: Overflow to represent absolute value of Integer.MIN_VALUE
Math.absExact(-9223372036854775807L): 9223372036854775807
Math.absExact(-9223372036854775808L): java.lang.ArithmeticException: Overflow to represent absolute value of Long.MIN_VALUE
StrictMath.absExact(-2147483647): 2147483647
StrictMath.absExact(-2147483648):java.lang.ArithmeticException: Overflow to represent absolute value of Integer.MIN_VALUE
StrictMath.absExact(-9223372036854775807L): 9223372036854775807
StrictMath.absExact(-9223372036854775808L): java.lang.ArithmeticException: Overflow to represent absolute value of Long.MIN_VALUE

I find it generally better to have an exception thrown for a surprising edge case than to have "something" returned that requires me to read the Javadoc to find out what that case is and what is returned in that case. The exception makes it obvious that the edge case was encountered rather than the discovery of a negative number being returned from an absolute value function call only being realized sometime later and "downstream" in the code. If nothing else, the mere presence of the Math.absExact and StrictMath.absExact methods should imply to Java developers that there is some "non-exact" possibilities to consider when using Java's math libraries to compute an absolute value and that realization might lead to reading the Javadoc to find out what those non-exact cases are.