Better NullPointerException Messages Automatic in JDK 15

I discussed long-awaited and highly appreciated improvements to NullPointerException (NPE) messages in the posts "Better Default NullPointerException Messages Coming to Java?" and "Better NPE Messages in JDK 14". When this JEP 358-driven feature was added to JDK 14, a developer who wanted to benefit from these more insightful NPE messages needed to explicitly state that desire by passing the argument -XX:+ShowCodeDetailsInExceptionMessages to the Java launcher (java).

The JDK 15 Early Access Build #29 was released this week and makes the use of better NPE messages automatic. The release notes associated with this early access build state: "The default of the flag ShowCodeDetailsInExceptionMessages was changed to 'true'. The helpful NullPointerException messages of JEP 358 are now printed by default. The messages contain snippets of the code where the NullPointerException was raised."

The next screen snapshot demonstrates that the helpful NullPointerException details are provided automatically with JDK 15 Early Access Build #29.

The release notes also point out that one potential risk of having the "helpful" NullPointerException messages written by default is the accidental exposure of sensitive details. The release notes warn: "App deployers should double check the output of their web applications and similar usage scenarios. The NullPointerException message could be included in application error messages or be displayed by other means in the app. This could give remote attackers valuable hints about a potential vulnerable state of the software components being used."

The next screen snapshot demonstrates that automatic presentation of helpful NullPointerException details can be disabled via the use of java launcher option -XX:-ShowCodeDetailsInExceptionMessages (and the old -XX:+ShowCodeDetailsInExceptionMessages is still supported even though this is now the default):

One on the interesting consequences of the JDK-8233014 change to make helpful NullPointerException messages enabled by default is that there will undoubtedly be some Java developers pleasantly surprised when they upgrade to JDK 15 and start seeing suddenly far more useful messages when encountering the ubiquitous NullPointerException.

Tycoon: Ransomware Targeting Java's JIMAGE on Multiple Platforms

The Blackberry Research and Intelligence Team and KPMG's UK Cyber Response Services Team have reported "Threat Spotlight: Tycoon Ransomware Targets Education and Software Sectors." This report outlines the "multi-platform Java ransomware targeting Windows and Linux that has been observed in-the-wild since at least December 2019" and which they've called "Tycoon."

The report provides a high level description of how the Tycoon ransomware is executed: "Tycoon ransomware comes in form of a ZIP archive containing a Trojanized Java Runtime Environment (JRE) build. The malware is compiled into a Java image file (JIMAGE) located at lib\modules within the build directory." The report describes the "sparsely documented" JIMAGE "file format file format that stores custom JRE images which is designed to be used by the Java Virtual Machine (JVM) at runtime." Additional high-level overviews of the JIMAGE file format can be found in JIMAGE - Java Image File Format, How Modules Are Packaged in Java 9, What Is a Custom Runtime Image in Java 9?, So what is a .jimage?, Alan Bateman's description, and slide 49 of JDK 9 Java Platform Module System. The JIMAGE format was introduced with JDK 9 modularity (Project Jigsaw).

Alan Bateman (owner of JEP 220 ["Modular Run-Time Images"])has explained why it's difficult to find documentation on the JIMAGE format: "The format is deliberately not documented and best to assume it will change and evolve over time as it inhales new optimizations." The jdk.jlink module documentation provides very brief mention of the jimage command-line tool and mentions that it is used "for inspecting the JDK implementation-specific container file for classes and resources." It also points out that there is no API for accessing jimage, which is different than for the module's two other tools (jlink and jmod). Similarly, there is no jimage tool reference on the "Java Development Kit Version 14 Tool Specifications" page even though jlink, jmod, and many other tools are featured there.

The previously mentioned report states that "the ransomware is triggered by executing a shell script that runs the Main function of the malicious Java module using the java -m command." The report also talks about Tycoon using Image File Execution Options Injection with Windows registry and then appearing to target Windows and Linux: "The malicious JRE build contains both Windows and Linux versions of this script." This reports adds that Tycoon's "Trojanized Java Runtime Environment (JRE) build" exists as a ZIP file placed in lib/module.

The "Conclusions" section of "Tycoon Ransomware Targets Education and Software Sectors" makes some interesting conclusions based on research observations. In particular, I find it interesting to read why they believe Tycoon in a targeted attack. The authors also point out, "This is the first sample we've encountered that specifically abuses the Java JIMAGE format to create a custom malicious JRE build."

I recommend reading the report "Threat Spotlight: Tycoon Ransomware Targets Education and Software Sectors" directly, but for those interested in summaries and others' observations related to this report, the following may be of interest.

• A new Java-based ransomware targets Windows and Linux (TechCrunch, TechCrunch)
• This new ransomware is targeting Windows and Linux PCs with a 'unique' attack (ZDNet)
• Newly discovered Tycoon ransomware leverages obscure Java image format (Silicon Angle)
• Multi-Platform 'Tycoon' Ransomware Uses Rare Java Image Format for Evasion (SecurityWeek)
• There's a new Java ransomware family on the block (CyberScoop)

The discovery of Tycoon is likely to bring significantly more attention to JIMAGE than ever before.

Octopus Scanner: Java Build Tools and Malware

Alvaro Muñoz recently posted "The Octopus Scanner Malware: Attacking the open source supply chain" on the GitHub Security Lab site. I found this post to be interesting for a number of reasons, including its detailed coverage of how the Octopus Scanner malware works and how it was discovered, how the GitHub Security Incident Report Team (SIRT) went about addressing it, how it affected a popular Java IDE, and how GitHub works to detect and address risks to open source software deployed on its site.

Muñoz calls Octopus Scanner "an OSS supply chain malware" and writes that 26 GitHub-hosted open source projects "were backdoored by this malware." In the post, Muñoz describes in detail how Octopus Scanner works. The post in its entirety is worth reading, but here are some highlights describing Octopus Scanner:

• "Octopus Scanner malware is only interested in the pre-jar and post-jar tasks" of "a NetBeans project build."
• "The malware disguises itself as an ocs.txt file, but we can easily determine is is actually a Java Archive (JAR) file."
• "The malware also infected any JAR files that were available in the project, such as dependencies - not necessarily just build artifacts."
• "The octopus.dat payload is the binary that actually performs the NetBeans build infections."
• "cache.dat is responsible for backdooring the built classes so that when these classes get executed, they will infect the underlying system."
• Octopus Scanner is designed to target "UNIX-like systems", MacOS, and Windows.

I found the highly detailed approach to diagnosing Octopus Scanner's behavior in the GitHub post to be fascinating and insightful reading. It was especially insightful to see the tools and methods used to understand better how Octopus Scanner behaves. For example, they used ClassFileTransformer and "a Bytecode manipulation library such as Javassist or ByteBuddy to inject our analysis code" into the "class responsible for decrypting the blob ... right before it actually gets loaded into the JVM."

Besides the interesting details of how Octopus Scanner works and how it was discovered and researched, other interesting insights in this GitHub post are related to the risks open source builds face. Muñoz writes, "Infecting build artifacts is a means to infect more hosts since the infected project will most likely get built by other systems and the build artifacts will probably be loaded and executed on other systems as well." Muñoz adds, "In an OSS context, it gives the malware an effective means of transmission since the affected projects will presumably get cloned, forked, and used on potentially many different systems. The actual artifacts of these builds may spread even further in a way that is disconnected from the original build process and harder to track down after the fact."

Muñoz opens the post and concludes the post with some discussions about this and other attempts at sabotaging open source products and their builds. One chilling thought is included in the Conclusion: "Since the primary-infected users are developers, the access that is gained is of high interest to attackers since developers generally have access to additional projects, production environments, database passwords, and other critical assets. There is a huge potential for escalation of access, which is a core attacker objective in most cases."

Exact Absolute Integral Numbers in JDK 15

JDK 15 Early Access Build b18 introduced new methods to the Math and StrictMath classes that will throw ArithmeticException on provided values outside the range supported by the methods without overflow. These methods bring to the concept of "absolute value" in Java what the methods such as Math.addExact, Math.subtractExact, and Math.multiplyExact have brought to basic arithmetic functions.

Prior to JDK 15, Integer.MIN_VALUE and Long.MIN_VALUE caused the respective methods Math.abs and StrictMath.abs to return the same negative number as represented by the MIN_VALUE largest negative possible value. This behavior is described in the Javadoc documentation for the affected methods and is demonstrated by the code shown below (and available on GitHub):

Demonstrating Pre-JDK 15 Absolute Value Methods

/**
 * Demonstrates "absExact" methods added to {@link Math}
 * and {@link StrictMath} with JDK 15 Early Access Build b18
 * (JDK-8241374: https://bugs.openjdk.java.net/browse/JDK-8241374).
 */
public class AbsoluteExactness
{
   public void demonstrateMathAbsInteger(final int integer)
   {
      out.println("Math.abs(" + integer + "): " + Math.abs(integer));
   }

   public void demonstrateMathAbsLong(final long longNumber)
   {
      out.println("Math.abs(" + longNumber + "L): " + Math.abs(longNumber));
   }

   public void demonstrateStrictMathAbsInteger(final int integer)
   {
      out.println("StrictMath.abs(" + integer + "): " + StrictMath.abs(integer));
   }

   public void demonstrateStrictMathAbsLong(final long longNumber)
   {
      out.println("StrictMath.abs(" + longNumber + "L): " + StrictMath.abs(longNumber));
   }

   public static void main(final String[] arguments)
   {
      final AbsoluteExactness instance = new AbsoluteExactness();

      // Demonstrate pre-JDK 15 Math/StrictMath "abs" functions on minimum values.
      instance.demonstrateMathAbsInteger(Integer.MIN_VALUE+1);
      instance.demonstrateMathAbsInteger(Integer.MIN_VALUE);
      instance.demonstrateMathAbsLong(Long.MIN_VALUE+1);
      instance.demonstrateMathAbsLong(Long.MIN_VALUE);
      instance.demonstrateStrictMathAbsInteger(Integer.MIN_VALUE+1);
      instance.demonstrateStrictMathAbsInteger(Integer.MIN_VALUE);
      instance.demonstrateStrictMathAbsLong(Long.MIN_VALUE+1);
      instance.demonstrateStrictMathAbsLong(Long.MIN_VALUE);
   }
}

When the above code is executed, the following output is written:

Math.abs(-2147483647): 2147483647
Math.abs(-2147483648): -2147483648
Math.abs(-9223372036854775807L): 9223372036854775807
Math.abs(-9223372036854775808L): -9223372036854775808
StrictMath.abs(-2147483647): 2147483647
StrictMath.abs(-2147483648): -2147483648
StrictMath.abs(-9223372036854775807L): 9223372036854775807
StrictMath.abs(-9223372036854775808L): -9223372036854775808

This output demonstrates that the maximum negative allowable value in the int and long ranges leads to that same value being returned from the appropriate abs method on Math and StrictMath.

JDK 15 Early Access Build b18 introduces absExact methods that throw ArithmeticException in this case instead of returning the negative value. This is demonstrated with the following code (also available on GitHub):

Demonstrating JDK 15-Introduced absExact Methods

public class AbsoluteExactness
{
   public void demonstrateMathAbsExactInteger(final int integer)
   {
      try
      {
         out.println("Math.absExact(" + integer + "): " + Math.absExact(integer));
      }
      catch (ArithmeticException exception)
      {
         err.println("Math.absExact(" + integer + "): " + exception);
      }
   }

   public void demonstrateMathAbsExactLong(final long longNumber)
   {
      try
      {
         out.println("Math.absExact(" + longNumber + "L): " + Math.absExact(longNumber));
      }
      catch (ArithmeticException exception)
      {
         err.println("Math.absExact(" + longNumber + "L): " + exception);
      }
   }

   public void demonstrateStrictMathAbsExactInteger(final int integer)
   {
      try
      {
         out.println("StrictMath.absExact(" + integer + "): " + StrictMath.absExact(integer));
      }
      catch (ArithmeticException exception)
      {
         err.println("StrictMath.absExact(" + integer + "):" + exception);
      }
   }

   public void demonstrateStrictMathAbsExactLong(final long longNumber)
   {
      try
      {
         out.println("StrictMath.absExact(" + longNumber + "L): " + StrictMath.absExact(longNumber));
      }
      catch (ArithmeticException exception)
      {
         err.println("StrictMath.absExact(" + longNumber + "L): " + exception);
      }
   }

   public static void main(final String[] arguments)
   {
      final AbsoluteExactness instance = new AbsoluteExactness();

      // Demonstrate JDK 15-introduced Math/StrictMath "absExact" functions
      // on minimum values.
      instance.demonstrateMathAbsExactInteger(Integer.MIN_VALUE+1);
      instance.demonstrateMathAbsExactInteger(Integer.MIN_VALUE);
      instance.demonstrateMathAbsExactLong(Long.MIN_VALUE+1);
      instance.demonstrateMathAbsExactLong(Long.MIN_VALUE);
      instance.demonstrateStrictMathAbsExactInteger(Integer.MIN_VALUE+1);
      instance.demonstrateStrictMathAbsExactInteger(Integer.MIN_VALUE);
      instance.demonstrateStrictMathAbsExactLong(Long.MIN_VALUE+1);
      instance.demonstrateStrictMathAbsExactLong(Long.MIN_VALUE);
   }

The output from this code is shown next and demonstrates the clear exception message that is thrown when the MIN_VALUE is passed to the absExact methods.

Math.absExact(-2147483647): 2147483647
Math.absExact(-2147483648): java.lang.ArithmeticException: Overflow to represent absolute value of Integer.MIN_VALUE
Math.absExact(-9223372036854775807L): 9223372036854775807
Math.absExact(-9223372036854775808L): java.lang.ArithmeticException: Overflow to represent absolute value of Long.MIN_VALUE
StrictMath.absExact(-2147483647): 2147483647
StrictMath.absExact(-2147483648):java.lang.ArithmeticException: Overflow to represent absolute value of Integer.MIN_VALUE
StrictMath.absExact(-9223372036854775807L): 9223372036854775807
StrictMath.absExact(-9223372036854775808L): java.lang.ArithmeticException: Overflow to represent absolute value of Long.MIN_VALUE

I find it generally better to have an exception thrown for a surprising edge case than to have "something" returned that requires me to read the Javadoc to find out what that case is and what is returned in that case. The exception makes it obvious that the edge case was encountered rather than the discovery of a negative number being returned from an absolute value function call only being realized sometime later and "downstream" in the code. If nothing else, the mere presence of the Math.absExact and StrictMath.absExact methods should imply to Java developers that there is some "non-exact" possibilities to consider when using Java's math libraries to compute an absolute value and that realization might lead to reading the Javadoc to find out what those non-exact cases are.

Updates on Valhalla and Amber Records (Mid-February 2020)

Two posts on OpenJDK mailing lists today summarize the status of Project Valhalla and Project Amber's Records.

Valhalla

Brian Goetz begins his post "Valhalla -- finding the primitives" with this sentence, "I think its worth reflecting on how far we've come in Valhalla, both for the specific designs in the VM and language, and the clarity of the basic concepts." This post contains a brief history of the major design models that have been employed with Valhalla to this point and talks about the advantages and disadvantages of each of these models.

About "Q World," Goetz writes (I've added the emphasis), "The idea was that we would declare a class as either a value class or a 'regular' class, and we would derive various properties based on that." Goetz outlines these properties:

Q-World: Differentiating Properties of Regular Versus Value Classes
Property	Regular Classes	Value Classes
Identity?	Yes	No
Nullable?	Yes	No
Reference types?	Yes	No

Goetz explains that "there were many aspects which were either confusing or unsatisfying" (he provides several examples of these) about the "Q World" approach and that the approach lacked a "clean story for migration."

Goetz describes "conflating" certain "distinctions" when working with "L World":

• "nullable vs non-nullable"
• "pass-by-reference vs pass-by-value / flattened"
• "reference type vs value type"
• "identity-ful vs identity-free"

Goetz discusses how the current state of Valhalla builds on lessons learned from the "Q World" and "L World" stages. He writes:

The primitive that Valhalla introduces into class declaration is whether the instances of the class have identity or not. Traditional classes are now revealed to be "identity classes"; the new kind (identity-free) are called "inline classes". (This might not be the final word on the subject.)

Another lesson learned in Valhalla efforts is summarized:

A big AHA of the recent iterations is that it makes sense to talk about both values of inline classes and references to those values. Reference type has (almost) nothing to do with inline vs identity -- it has to do with whether the value set of the type contains values, or references.

Based on these summarized observations, Goetz writes about the "ref/val notation for the types":

"Inline" is a way of saying "identity free" when declaring classes, but it doesn't say anything (yet) about the semantics of how we represent variables on the heap or pass them on the stack. For this, we need an additional property of the type, and ref vs val seems to ideally describe what we mean."

Amber Records

In the post "[records] Summary so far," Gavin Bierman writes about "some remaining design decisions concerning records." This post outlines the following questions/design decisions and then provides partial or complete answers to the design decisions (see the post for the additional details related to each design decision).

1. Q1. Should the distinguished supertype of records java.lang.Record be renamed?
2. Q2. Accessibility of mandated members.
3. Q3. Nesting.
4. Q4. Abstract records.
5. Q5. Deconstruction patterns.
6. Q6. @deprecated
7. Q7. .equals and .toString are slow.
8. Q8. Translation strategy.
9. Q9. Hashing.
10. Q10. Special annotation for explicit declaration of accessors.
11. Q11. Changing the spec of .toString method
12. Q12. Transactional methods
13. Q13. Factory methods instead of constructors.

Although Records are available as a preview feature in JDK 14, it's obvious much work is still being done on them. This preview status provides an opportunity for Records to be used and tested to gain feedback that can change how they are implemented. The thirteen design decisions listed above (some of which are decidedly no change and some which are possible changes) demonstrate that Records are still going to be fine-tuned.

JDK 14/JEP 305 instanceof Pattern Matching "Smart Casts"

I generally view the presence of of the instanceof operator in Java code as a "red flag," meaning that it's not necessarily wrong to use instanceof in certain situations, but its use sometimes indicates a design issue that could be resolved in a cleaner way as described in some resources referenced at the end of this post (including resources about similar type checking functionality in languages other than Java).

Although I've seen instanceof used several times when it does not need to be, I've run into even more situations where it was not easy to avoid instanceof. This is particularly true when working with legacy code bases and certain libraries and frameworks in which I have no ability to refactor relationships between classes to support interfaces, method overriding, and other tactics that can be used to remove the need for instanceof.

A very common technique employed with instanceof is to immediately cast to the type checked in the conditional using instanceof. JEP 305 ["Pattern Matching for instanceof (Preview)"] provides an example of this common pattern and I've slightly adapted that example here:

if (object instanceof String)
{
    final String string = (String) object;
    // Do something with the 'string' variable typed as String
}

Benji Weber has posted on using reflection and on using lambda expressions to achieve Kotlin-like "instanceof smart casts." Fortunately, JDK 14 and JEP 305 bring built-in language support (albeit preview status) for this approach.

JDK 14 introduces a preview feature that allows the instanceof conditional and associated cast to be implemented completely within the conditional. The effect on the above code example is shown next:

if (object instanceof String string)
{
    // Do something with the 'string' variable typed as String
}

This preview feature is available in the JDK 14 Early Access Builds and I'm using JDK 14 Early Access Build 34 for my examples in this post.

The JEP 305 preview feature in JDK 14 is a small nicety whose advantage is more obvious in lengthy if-then-else conditional statements. The next two code listings provide a comparison of the "old way" of calling instanceof and explicitly casting to the "new preview way" of using instanceof pattern matching.

Traditional instanceof Coupled with Explicit Cast

static void makeAnimalNoises(final Object animal)
{
   if (animal instanceof Dog)
   {
      final Dog dog = (Dog) animal;
      out.println(dog.bark());
   }
   else if (animal instanceof Cat)
   {
      final Cat cat = (Cat) animal;
      out.println(cat.meow());
   }
   else if (animal instanceof Duck)
   {
      final Duck duck = (Duck) animal;
      out.println(duck.quack());
   }
   else if (animal instanceof Horse)
   {
      final Horse horse = (Horse) animal;
      out.println(horse.neigh());
   }
   else if (animal instanceof Cow)
   {
      final Cow cow = (Cow) animal;
      out.println(cow.moo());
   }
   else if (animal instanceof Lion)
   {
      final Lion lion = (Lion) animal;
      out.println(lion.roar());
   }
   else
   {
      out.println("ERROR: Unexpected animal: " + animal);
   }
}

JDK 14/JEP 305 Preview Feature

static void makeAnimalNoises(final Object animal)
{
   if (animal instanceof Dog dog)
   {
      out.println(dog.bark());
   }
   else if (animal instanceof Cat cat)
   {
      out.println(cat.meow());
   }
   else if (animal instanceof Duck duck)
   {
      out.println(duck.quack());
   }
   else if (animal instanceof Horse horse)
   {
      out.println(horse.neigh());
   }
   else if (animal instanceof Cow cow)
   {
      out.println(cow.moo());
   }
   else if (animal instanceof Lion lion)
   {
      out.println(lion.roar());
   }
   else
   {
      out.println("ERROR: Unexpected animal: " + animal);
   }
}

The full code is on GitHub and the difference between the old approach and new preview approach is available.

Because instanceof pattern matching is a preview feature, the code using this feature must be compiled with the javac flags --enable-preview and -source 14. It must be executed with java flag --enable-preview.

Conclusion

For more details on how this feature is implemented, see the post "RFR: JDK-8237528: Inefficient compilation of Pattern Matching for instanceof." Pattern matching support for instanceof is another Amber-provided step toward reduced boilerplate code in Java.

 

Resources on Issues Using instanceof

• Java 'instanceOf': Why And How To Avoid It In Code
• Do you really need instanceof?
• Is using instanceof in Java consider bad practice?
• Using Instanceof Is Mostly a Code Smell
• Instanceof in Conditionals: A Code Smell
• Beware of instanceof Operator
• How Evil is 'instanceof'?
• Type check is a code smell

Source Code for Effective Java Third Edition Updated to Use Newer Features

Those who have read the Third Edition of Effective Java are likely aware of the source code associated with that book available on GitHub. The jbloch/effective-java-3e-source-code project has 1700+ stars and has been forked nearly 800 times as of this writing. The version of Java featured in the Third Edition of Effective Java is largely JDK 8 with some coverage of JDK 9 (see my earlier post for details on what is covered in this third edition).

Much has been added to the JDK since the publication of the Third Edition of Effective Java and many new releases have arrived with the faster 6-month cadence. Given this, I was particularly interested to see in an amber-spec-experts mailing list post that Rémi Forax has forked jbloch/effective-java-3e-source-code into the GitHub project forax/effective-java-3e-source-code that has "taken the source of Effective Java (3rd Ed) and change them to use var, switch expression, records and the instanceof with the type test pattern."

There are several things that I like about the idea of refreshing examples from Effective Java (Third Edition) to use newer features:

• Developers can see how to apply effective Java practices using recently released features.
• Developers can view the differences between the JDK 8/9 versions and the newer versions to see how new constructs replace older constructs and thus gain a better understanding of the newer constructs.
• It is useful to see some of the changes when deciding whether a particular change to use a newer construct really helps with code readability in a given situation.

The main page for the forked forax/effective-java-3e-source-code (README.md) states, "The source code have been updated to use new constructs available since Java 9, the version used by the 3rd edition." That page then provides bullets on the types of new constructs applied to the source code with links to each new construct's associated JDK Enhancement Proposal (JEP).

As of this writing, Commit 275eef87e4661f7f1edc41f4730cecf7a1096a97 is the main commit of interest. It covers changes to 113 files. I'll call out a few specific changes here to illustrate the types of changes applied (some of which are to apply preferred constructs that were available even before JDK 9):

• NutritionFacts.java (Chapter 2, Item 2) changed from a class to a record.
• PickTwo.java (Chapter 5, Item 32) change demonstrates switch expressions.
• CaseInsenstiveString.java (Chapter 3, Item 10) change demonstrates application of instanceof with type test pattern.
• WordList.java (Chapter 3, Item 14) changed to apply var for variable declarations taking advantage of local type inference. There are many of these changes, but this is one I liked. I'm not convinced that some of the changes in other places (such as changing int to var) are helpful. The change for InstrumentedSet (Chapter 4 Item 18) is even more compelling.
• Change to RecursiveTypeBound.java (Chapter 5, Item 31) demonstrates using var in for loop.
• Copy.copy method (Chapter 2, Item 9) changed to accept instances of Path instead of instances of String and changed to use Files.newInputStream and Files.newOutputStream in place of new FileInputStream and new FileOutputStream respectively.

Conclusion

The ability to view changes to the original source code associated with the Third Edition of Effective Java to accommodate new language constructs is highly useful in terms of learning about the new constructs and how they relate to or replace old constructs and in deciding if the differences are desirable in different situations.