Better Default NullPointerException Messages Coming to Java?

I have recently been interested in a February 2019 into March 2019 discussion on the OpenJDK core-libs-dev mailing list regarding addressing the lack of a detailed message associated with a NullPointerException that was thrown after being instantiated with its no-arguments constructor. This is an issue I've run into frequently when using Java and which has even caused me to change code in a few cases to better deal with the issue.

In many cases, NullPointerException (NPE) can be one of the easier exceptions to resolve (or at least diagnose what was null) if only one possible source of a NullPointerException exists in the statement and if line numbers are available in the stack trace (not compiled with -g:none).

Although it is especially challenging for those new to Java, a NullPointerException with no message can be a disappointing experience even for experienced Java developers in certain cases. The most obvious case when having no message associated with the NullPointerException is when there are multiple candidates in a given statement that might be throwing the NullPointerException. One example of this case is calling methods upon each previous method's return object in a manner such as this: getA().getB().getC()... where each of the methods potentially returns null. Another example is when multiple arguments of primitive data types to a method (or constructor) might lead to a NullPointerException if the caller passes a null to to that method that is dereferenced to be a primitive.

Enhancement JDK-8218628 ("Add detailed message to NullPointerException describing what is null.") addresses some of these cases. The description of this enhancement states, "When getting a NPE it is often hard to determine which reference in an expression had been null. This change adds a message telling this." This enhancement also provides several examples of Java statements that typically result in NullPointerException with potentially frustrating lack of detail. I have captured similar cases to these examples in the GitHub-hosted class NpeDemo (see this version to match to line numbers in output below). When those demonstration examples are executed (they all intentionally throw NPEs), the output appears as shown in the following when compiled with default settings (full stack information still available):

=========================================
| #1: Element [0] on null boolean array |
=========================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateFirstExampleIndexAccessOnNullBooleanArray(NpeDemo.java:37)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(NpeDemo.java:179)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:310)

=================================
| #2: .length on null boolean[] |
=================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateSecondExampleLengthOnNullBooleanArray(NpeDemo.java:59)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(NpeDemo.java:180)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:310)

=======================================
| #3: Assigning float to null float[] |
=======================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateThirdExampleAssigningValueToElementOfNullFloatArray(NpeDemo.java:80)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(NpeDemo.java:181)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:310)

======================================
| #4: Accessing field on null object |
======================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateFourthExampleAccessInstanceFieldOfNullObject(NpeDemo.java:101)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(NpeDemo.java:182)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:310)

===================
| #5: throw null; |
===================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateFifthExampleThrowingConstantNull(NpeDemo.java:121)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(NpeDemo.java:183)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:310)

================================================
| #6: Method invocation on null instance field |
================================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateSixthExampleMethodInvocationOnNullInstanceField(NpeDemo.java:141)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(NpeDemo.java:184)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:310)

=============================================
| #7: synchronized() on null instance field |
=============================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateSeventhExampleSynchronizedNullInstanceField(NpeDemo.java:161)
 at dustin.examples.npe.NpeDemo.demonstrateJdk8218628Examples(NpeDemo.java:185)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:310)

==========================================================================
| <<< Null Lost in Long Series of Method Invocations in Single Statement |
==========================================================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateNullLostInSeriesOfMethodInvocationsInSingleStatement(NpeDemo.java:198)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:311)

=======================================================
| <<< Null Lost in Dereferenced Constructor Arguments |
=======================================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateNullLostInConstructorAcceptingMultiplePotentiallyNullArgumentsDereferenced(NpeDemo.java:226)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:312)

==================================================
| <<< Null Lost in Dereferenced Method Arguments |
==================================================

java.lang.NullPointerException
 at dustin.examples.npe.NpeDemo.demonstrateNullLostInMethodAcceptingMultiplePotentiallyNullArgumentsDereferenced(NpeDemo.java:254)
 at dustin.examples.npe.NpeDemo.main(NpeDemo.java:313)

There is no message provided with any of the NullPointerExceptions shown in the examples above. However, in these cases, the culprit is relatively easy to identify because the methods they occur in are small and there are line numbers that point directly to where the NPE was thrown. These would be more difficult to identify if there were no line numbers (source compiled with -g:none) and the methods were long (multiple lines in which NPEs could be thrown) or there were overloaded versions of the method with the same name.

Had the code been compiled with -g:none, there would be no class name or line number shown in the stack traces [would just list (Unknown Source) instead of (file name:line number)] and it could be trickier to detect where the NPE was thrown, especially if thrown from a lengthy method with many candidates for NPEs or from a method that was overloaded multiple times in the same class such that method name alone is not as helpful.

Some of the examples demonstrated above feature NPEs that are difficult to identify even when one knows the line number because there are so many potential throwers of NPE on that line. Changes such as those proposed by JDK-8218628 would be most welcome in these cases.

Although a solution was implemented for JDK-8218628, it has since been decided that there are enough considerations involved to justify a JDK Enhancement Proposal (JEP) to work out more design and implementation details. This JEP is JDK-8220715 ("Add detailed message to NullPointerException describing what is null") and its "Summary" states, "NullPointerExceptions are freqently encountered developing or maintaining a Java application. NullPointerExceptions often don't contain a message. This complicates finding the cause of the exception. This JEP proposes to enhance the exception text to tell what was null and which action failed."

JEP JDK-8220715 also provides detailed description of the proposed basic algorithm for computing a message for NPE when one is not explicitly provided. The text points out that when a NullPointerException is thrown in its example, "the original Java code is not available," but the information is still "stored in the 'backtrace' field of an exception object" which is "a field private to the jvm implementation."

JEP JDK-8220715 highlights that "computing the NullPointerException message proposed here is a considerable overhead," but addresses that by proposing "delay computing the message until it is actually accessed." In other words, the "default" NPE message would only be calculated if an explicit one had not been provided when the NullPointerException was instantiated.

The "Alternatives" section of JEP JDK-8220715 states that "the current proposal is to implement this in the Java runtime in C++ accessing directly the available datastructures in the metaspace." The section considers some alternatives to this approach (such as implementing it via a JDK library such as StackWalker) and explains why the proposed approach may be preferable to the alternatives.

For more background details related to the proposed enhancements to NullPointerException messages, see the OpenJDK core-libs-dev mailing list. Here are some posts from that discussion that may be of interest with excerpt shown from each post:

• Goetz Lindenmaier: "... since Java 5, our internal VM reports verbose null pointer exception messages. I would like to contribute this feature to OpenJDK. ... The messages are generated by parsing the bytecodes. For not to have any overhead when the NPE is allocated, the message is only generated when it is accessed by getMessage() or serialization. For this I added a field to NPE to indicate that the message still needs to be computed lazily."
• Christoph Langer: "... thanks for bringing this into OpenJDK finally. I know of people that'll be quite happy about this feature."
• Peter Levart: "Make sure to initialize the NPE_MESSAGE_PENDING to a new String("something") or else you may be sharing this constant reference with somebody else via string interning..."
• Andrew Dinn: "Also, if you want your message to reflect the bytecode that is actually in use when the exception occurs then you really need to do it by pulling the bytecodes out of the method metadata. The bytecode returned by JvmtiClassFileReconstitutor will not include any bytecode changes that were installed by a ClassFileTransformer. However, this is a potential can of worms because old and new versions of a method and associated bytecode can exist at the same time. You need to be sure which version of the method and, hence, bytecode the exception was generated from. If you are trying to do this from Java by calling into the JVM then I think you are going to have problems."
• Goetz Lindenmaier: "The original implementation is C++ and walks the metaspace given the method* and BCI where the exception occurred. So it uses only data already sitting in memory. See JVM_GetExtendedNPEMessage() in jvm.cpp. The idea was to implement this in Java using StackWalker and ASM. If I had the right bytecodes, and the right starting point, ASM would be helpful to implement the analysis I think."
• Mandy Chung: "We all think that improving NPE message is a useful enhancement for the platform and helps developers to tell what causes NPE. ... This would get the discussion on the proposal feature and then the discussion of the best way to to implement it in the VM, library, or combination."
• Maurizio Cimadamore: "... this enhancement will be a great addition to our platform ... I also think that the design space for such an enhancement is non trivial, and would best be explored (and captured!) in a medium that is something other than a patch."
• Goetz Lindenmaier: "... better wording of the messages ... Especially look at the first few messages, they point out the usefulness of this change. They precisely say what was null in a chain of dereferences."
• Maurizio Cimadamore: "... please find the attached ASM-based patch. It is just a PoC, as such it does not provide as fine-grained messages as the one discussed in the RFE/JEP, but can be enhanced to cover custom debugging attribute ..."

There are numerous other posts in the threads and the posts above are samples of the discussion.

Having better "default" information associated with NPEs will be a welcome addition. JDK-8218628 is currently associated with JDK 13, but now that JDK-8220715 exists, it may be a bit less certain whether this will be associated with JDK 13. A draft JEP has been written for this, but as a draft JEP, it is not yet targeted to a particular JDK release.