Friday, August 21, 2020

JDK16 javac xlint Warning about Default Constructors

I mentioned in my blog post "Explicit No-Arguments Constructor Versus Default Constructor" that "it is possible that one day javac will have an available lint warning to point out classes with default constructors." In that post, I referenced JDK-8071961 ("Add javac lint warning when a default constructor is created"), which has now been implemented as of JDK 16 Early Access Build #12. This post introduces that newly available javac -xlint warning.

To see this new javac -Xlint warning in action, one must download at least JDK 16 Early Access Build #12 (19 August 2020) or later.

To demonstrate the new javac -Xlint warning, we need a class with no explicit constructor so that javac will generate a "default constructor."

<rant>(By the way, a minor pet peeve of mine is when someone comments an explicit constructor with no arguments with Javadoc text that states "Default constructor." It's not really a default constructor once it's explicitly specified!)</rant>

An example of a class with no explicit constructor is available on GitHub and is shown here:

DefaultConstructor.java
package dustin.examples.jdk16;

import static java.lang.System.out;

/**
 * This class intentionally does NOT specify an explicit constructor
 * so that a "default constructor" will be generated and trigger the
 * new JDK 16 warning.
 */
public class DefaultConstructor
{
   private String name;

   public String getName()
   {
      return name;
   }

   public void setName(final String newName)
   {
      name = newName;
   }

   public static void main(final String[] arguments)
   {
      final DefaultConstructor instance = new DefaultConstructor();
      instance.setName(arguments.length > 0 ? arguments[0] : "");
      out.println("Hello " + instance.getName() + "!");
   }
}

If we compile the new class with no explicitly specified constructor with javac provided by the OpenJDK JDK 16 Early Access Build #12 or later, we won't see the new warning demonstrated unless we export the package that class is in and enable -Xlint warnings. An example of exporting the package is available on GitHub and is shown here:

module-info.java
module dustin.examples
{
   exports dustin.examples.jdk16;
}

Compiling with -Xlint

When I run javac -X with the JDK 16 Early Access Build #12 compiler, I see these Xlint-related options that are now available (emphasis added):

  -Xlint                       Enable recommended warnings
  -Xlint:(,)*
        Warnings to enable or disable, separated by comma.
        Precede a key by - to disable the specified warning.
        Supported keys are:
          all                  Enable all warnings
          auxiliaryclass       Warn about an auxiliary class that is hidden in a source file, and is used from other files.
          cast                 Warn about use of unnecessary casts.
          classfile            Warn about issues related to classfile contents.
          deprecation          Warn about use of deprecated items.
          dep-ann              Warn about items marked as deprecated in JavaDoc but not using the @Deprecated annotation.
          divzero              Warn about division by constant integer 0.
          empty                Warn about empty statement after if.
          exports              Warn about issues regarding module exports.
          fallthrough          Warn about falling through from one case of a switch statement to the next.
          finally              Warn about finally clauses that do not terminate normally.
          missing-explicit-ctor Warn about missing explicit constructors in public classes in exported packages.
          module               Warn about module system related issues.
          opens                Warn about issues regarding module opens.
          options              Warn about issues relating to use of command line options.
          overloads            Warn about issues regarding method overloads.
          overrides            Warn about issues regarding method overrides.
          path                 Warn about invalid path elements on the command line.
          processing           Warn about issues regarding annotation processing.
          rawtypes             Warn about use of raw types.
          removal              Warn about use of API that has been marked for removal.
          requires-automatic   Warn about use of automatic modules in the requires clauses.
          requires-transitive-automatic Warn about automatic modules in requires transitive.
          serial               Warn about Serializable classes that do not provide a serial version ID. 
                             Also warn about access to non-public members from a serializable element.
          static               Warn about accessing a static member using an instance.
          text-blocks          Warn about inconsistent white space characters in text block indentation.
          try                  Warn about issues relating to use of try blocks (i.e. try-with-resources).
          unchecked            Warn about unchecked operations.
          varargs              Warn about potentially unsafe vararg methods
          preview              Warn about use of preview language features
          none                 Disable all warnings

As shown in these usage details, one can use -Xlint, -Xlint:all, or -Xlint:missing-explicit-ctor to see this new warning about default constructors being exposed by classes in publicly exported packages.

Compiling the new class with -Xlint, -Xlint:all, or -Xlint:missing-explicit-ctor demonstrates the new warning about default constructors being used in a formal API:

-Xlint:all

-Xlint and -Xlint:missing-explicit-ctor

As shown in the screenshots, the warning message states (emphasis added by me): "warning: [missing-explicit-ctor] class DefaultConstructor in exported package dustin.examples.jdk16 declares no explicit constructors, thereby exposing a default constructor to clients of module dustin.examples"

The warning message that javac provides when -Xlint is appropritely specified describes the issue and specifically calls out the exported package with the offending class and the name of module that exports that package.

Summary of Steps to See Default Constructor Warning

  1. Download and "install" OpenJDK 16 Early Access Build #12 (or later) from https://jdk.java.net/16/
  2. Write Java class with no explicitly specified constructor so that javac will generate a "default constructor" (example).
  3. Export package with class with no explicit constructor via a module-info.java file (example).
  4. Compile class with no explicit constructor with -Xlint:all provided to the javac compiler.

Not All Classes with No Explicit Constructors Will Be Flagged

Not all Java classes that lack an explicit constructor will lead to this new warning being emitted even when a relevant -Xlint option is specified. As stated earlier, even the DefaultConstructor class used in this post's example does not lead to the warning message being generated until it's package is exported in the module-info.java file. Joe Darcy explains on the OpenJDK compiler-dev mailing list:

In terms of detailed criteria to issue the new warnings, there was the usual tension in warnings between reducing false positives and false negatives. For example, warning for *any* default constructor, even in a throw-away class, would be more annoying than helpful. With some guidance from the JDK code base, criteria in the current patch are a default constructor merits a warning if:

  • The class is in a named package and the packaged has an unqualified export from its module AND
  • The class is public and, if it is a nested class, all of its lexically enclosing types are public too.

An unqualified export, where the package is available to use to any module and not just named ones, was taken to indicate classes in the package can comprise a "formal API". It would be simple to change this to an unqualified export, but I wanted to avoid unwanted instances of a new warning. If a public nested class is a non-public enclosing class, the nested class is not directly part of the exported API. These combinations of kinds of exports and nesting are tested in the tests in the DefaultCtor directory.

Why Warn on Use of Default Constructors in a "Formal API" Class?

The previously mentioned Joe Darcy post explains why this warning has been added:

Some background on the design of the warning and broader usage context, while default constructors can be convenient for informal code, they can be a bit troublesome for more formal APIs, such as the public classes of packages in the JDK. One issue is that default constructors do not have javadoc. Another is that a class that semantically should not be instantiated, say it is a solely a holder for static constants and methods, can get instantiated and subclassed. (Subclasssing such a class is an anti-pattern to allow use short names for the static members, which is no longer necessary since static imports as of Java SE 5.0.)

Conclusion

This seemingly small change to add this new warning about "default constructors" in "formal API" classes has required more effort than might initially be assumed. A large number of issues were written to not only introduce the xlint warning, but to clean up numerous classes throughout the JDK that triggered this warning when compiled. Furthermore, naming and logging can often be tricky and the particular warning mssage went through review and iterative changes as well.

Friday, August 7, 2020

Project Amber Design Documents on GitHub

As discussed in a Brian Goetz post on the amber-spec-experts mailing list, an effort is currently underway for "migrating design docs to GitHub." The GitHub repository is openjdk/amber-docs and it already hosts some frequently-referenced Project Amber design documents.

The project's main README states that this repository "is for design notes, presentations, guides, FAQs, and other collateral surrounding OpenJDK Project Amber." It also explains the relationship between the Amber design documents in this repository with those hosted on the OpenJDK Project Amber site: "Most documents here are written in (pandoc) Markdown, and changes made to Markdown sources are automatically formatted and pushed to the OpenJDK website.

The Amber design documents hosted on openjdk/amber-docs are currently categorized as either design-notes or guides. Here are some of the Amber design documents already hosted in the new openjdk/amber-docs GitHub repository (mostly in GitHub-friendly MarkDown format):

The moving of OpenJDK source and artifacts to GitHub via Project Skara is paying benefits already in terms of accessibility for Java developers. The moving of Amber design documents to GitHub will likewise make these documents more readily available. As a side note, the OpenJDK design discussions and decisions are also often readily available from the relatively new Inside.java blog that features "news and views from the Java team at Oracle."

Although these Project Amber design documents have long been publicly available, this migration to GitHub should make them even easier to access and will likely increase awareness of and access to these documents. Undoubtedly, advantages inherent with a modern version control system will be gained including easier ability to make, review, track, audit, and contribute changes.

Saturday, August 1, 2020

Finalizing instanceof Pattern Matching and Records in JDK 16

Gavin Bierman has recently posted two approachable posts regarding instanceof pattern matching and records on the amber-spec-experts mailing list and both posts are thorough but succinct. I recommend that anyone interested in the latest state of these two Project Amber initiatives read these posts: "Finalizing in JDK 16 - Pattern matching for instanceof" and "Finalizing in JDK 16 - Records". Even for those not interested in the details, it is probably interesting to see the likelihood of instanceof pattern matching and records being finalized in JDK 16.

Pattern Matching instanceof Operator

In "Finalizing in JDK 16 - Pattern matching for instanceof," Bierman writes: "In JDK 16 we are planning to finalize two JEPs: Pattern matching for instanceof and Records." About instanceof pattern matching, Bierman states, "Adding conditional pattern matching to an expression form is the main technical novelty of our design of this feature."

The "Finalizing in JDK 16 - Pattern matching for instanceof" post talks about advantages of the instanceof pattern matching approach. Advantages discussed in the post include refactoring of a "common programming pattern" (checking for instanceof and then explicitly casting the thing checked with instanceof to the type it was checked against) to smaller code that is more readable. Another discussed advantage is the ability "to compactly express things with expressions that are unnecessarily complicated using statements."

Bierman also discusses scope issues, local pattern variable "poisoning", and how these will be dealt with using "flow scoping." About this, Bierman writes:

Java already uses flow analysis - both in checking the access of local variables and blank final fields, and detecting unreachable statements. We lean on this concept to introduce the new notion of flow scoping. A pattern variable is only in scope where the compiler can deduce that the pattern has matched and the variable will be bound. This analysis is flow sensitive and works in a similar way to the existing analyses. ... The motto is "a pattern variable is in scope where it has definitely matched". This is intuitive, allows for the safe reuse of pattern variables, and Java developers are already used to flow sensitive analyses.

The post on instanceof pattern matching ends with a discussion of the "Swiss cheese property" ("unfortunate interaction of flow scoping and shadowing [local pattern variables shadowing a field declaration]") and expresses the hope that IDEs will help developers to deal with these unfortunate interactions.

Records

Bierman's post "Finalizing in JDK 16 - Records" provides some nice summary descriptions related to Records. The first is, "Record classes are a special kind of class that are used primarily to define a simple aggregate of values." This is immediately followed by, "Records can be thought of as nominal tuples; their declaration commits to a description of their state and given that their representation, as well as all of the interesting protocols an object might expose -- construction, property access, equality, etc -- are derived from that state description."

This post covers the "implicit declaration of a number of members" of a Record from the Record's "state description." The post also provides a brief overview of why a developer might "explicitly [provide] a canonical constructor" and how to do so. About this, Bierman writes, "The intention of a compact constructor declaration is that only validation and/or normalization code need be given in the constructor body; the remaining initialization code is automatically supplied by the compiler."

"Finalizing in JDK 16 - Records" also discusses canonical constructor accessibility, @Override, and general annotations related to Records. The post concludes by mentioning that Records can be declared locally and explains how this is an advantage for an expected common use case of Records: "Records will often be used as containers for intermediate data within method bodies. Being able to declare these record classes locally is essential to stop proliferation of classes."

Conclusion

The two Project Amber initiatives of instanceof pattern matching and records will be welcome additions to the Java programming language. Although these features have been available in versions of the JDK before JDK 16 as preview features, it will be their finalization (hopefully in JDK 16) that will enable many more developers to apply them in production code.

Friday, July 10, 2020

Potential Inlined JDK Classes

Earlier this week, Dan Smith posted "JEP draft: Identity Warnings for Inline Class Candidates" on the valhalla-spec-experts mailing list. The purpose of this post was to introduce a draft JEP ("Identity Warnings for Inline Class Candidates") associated with JDK-8249100.

The main intent of this JEP is to "provide warnings at compile time and runtime for behaviors that will change when certain classes of the Java SE APIs become inline classes." Although I think it's a good idea to have these warnings to help Java developers recognize coming change and opportunities for change, I find the background details provided to in this draft JDK Enhancement Proposal to be as interesting as the end objective. This draft JEP consolidates discussions and decisions discussed among many different threads and messages on the mailing lists in a single easily referenced location.

Inline Classes in Java

The "Motivation" section of the draft JEP states that inline classes "declare their instances to be identity-free and capable of 'inline' or 'flattened' representations, where instances can be copied freely between memory locations and encoded using solely the values of the instances' fields." That same section also outlines two operations (monitoring/locking with synchronized and instantiation with new) that will not be supported by the identity-less inline classes. Because certain operations will not be allowed on inline classes and because some current JDK classes that may be used with these operations may be transitioned to inline classes at some point, it is desirable to have this draft JEP's proposed warnings. These warnings alert developers to their current uses of these inline class candidate JDK classes in conjunction with these operations that will not be supported on inline classes in the future (eventually an error will result).

For more background on inline classes, see the March 2020 version of the State of Valhalla document, in which Brian Goetz makes these observations:

  • "Inline classes are heterogeneous aggregates that explicitly disavow object identity."
  • Inline classes "must be immutable and cannot be layout-polymorphic."
  • "Inline classes have some restrictions compared to ordinary (identity) classes; they are final, their fields are final, and their ability to participate in inheritance is limited."
  • "Inline classes can use most mechanisms available to classes: methods, constructors, fields, encapsulation, type variables, annotations, etc"
  • "The slogan" for inline classes "is Codes like a class, works like an int."
  • "Inline types could equally be thought of as 'faster classes' or 'user-definable primitives'."

Identified JDK Inline Class Candidates

The "Description" section of the draft JEP lists several bullets of classes and groups of classes that are "likely to become inline classes." These candidate JDK inline classes are listed, as of this writing, as follows:

  • Primitive Wrapper Classes: java.lang.Byte, java.lang.Short, java.lang.Integer, java.lang.Long, java.lang.Float, java.lang.Double, java.lang.Boolean, java.lang.Character
  • Optional Classes: java.util.Optional, java.util.OptionalInt, java.util.OptionalLong, java.util.OptionalDouble
  • Java Time Classes: java.time.Instant, java.time.LocalDate, java.time.LocalTime, java.time.LocalDateTime, java.time.ZonedDateTime, java.time.ZoneId, java.time.OffsetTime, java.time.OffsetDateTime, java.time.ZoneOffset, Duration, java.time.Period, java.time.Year, java.time.YearMonth, java.time.MonthDay, and select classes in java.time.chrono package.
  • Collections Returned from Collection Factories: java.util.List.of, java.util.List.copyOf, java.util.Set.of, java.util.Set.copyOf, java.util.Map.of, java.util.Map.copyOf, java.util.Map.ofEntries, and java.util.Map.entry
  • java.lang.Runtime.Version
  • java.lang.ProcessHandles Implementations
  • java.lang.constant.ConstantDesc Implementations (except java.lang.String)
  • Select Classes in the jdk.incubator.foreign Module

JDK Classes That Will Not Be Inlined

There are some JDK-provided classes that, at first thought, might seem like candidates to be inline classes, but turn out to have characteristics that make them unlikely to ever be transitioned. These include java.lang.String and java.math.BigInteger and java.math.BigDecimal.

Conclusion

The recently proposed draft JEP discussed in this post provides a centralized location for updated background details related to the currently planned inline classes feature being discussed and worked as part of Project Valhalla. Keeping in mind that this is a draft JEP about an in-work feature, this draft JEP provides insight into how inline classes will change how we write and use Java classes.

Thursday, July 2, 2020

jcmd Increasingly the Single JDK Command-line Tool to Rule Them All

What would you miss from the JDK command line tools jinfo, jmap, and jstack if those tools were no longer available?

I've been a fan of the command-line tool jcmd for a long time. I like that it combines features and functions of many other JDK command-line tools in a single tool. I also like that I like its interactive nature that supports my asking the tool what it's capable of for a given JVM process, meaning I don't have to remember exact options when applying its different functions.

Since its inception, jcmd has been taking on functionality from various other command-line JDK tools. In a recent post on the OpenJDK jdk-dev mailing list, Stephen Fitch announced a survey intended to "gather more information and help us evaluate and understand how others are using [three 'j* tools' jinfo, jmap, jstack] in the JDK." The surveyors are "specifically interested in your use-cases and how these tools are effective for you in resolving JVM issues."

Fitch explains the need for this survey:

We are considering deprecation and (eventual) removal of the jinfo, jmap, jstack - (aka “j* tools”) and building out a future foundation for some aspect of serviceability on jcmd, however we don’t have a lot of data about how how these tools are used in practice, especially outside of Oracle.

I'm encouraged that it sounds likely that jcmd will continue to add functionality under its single approach that was formerly available with different tools.

The survey will be open through 15 July 2020 and is an opportunity for Java developers to share their experiences and use cases with the JDK command line tools jinfo, jmap, and jstack so that it's more likely those use cases will be supported in jcmd (if not already supported) in the future. Fitch's post presents the URL for the survey: https://www.questionpro.com/t/AQk5jZhiww

Select jcmd Posts

Here are some of my previous posts regarding jcmd (I have used jcmd in numerous other posts to demonstrate various output as well):

Friday, June 26, 2020

Better NullPointerException Messages Automatic in JDK 15

I discussed long-awaited and highly appreciated improvements to NullPointerException (NPE) messages in the posts "Better Default NullPointerException Messages Coming to Java?" and "Better NPE Messages in JDK 14". When this JEP 358-driven feature was added to JDK 14, a developer who wanted to benefit from these more insightful NPE messages needed to explicitly state that desire by passing the argument -XX:+ShowCodeDetailsInExceptionMessages to the Java launcher (java).

The JDK 15 Early Access Build #29 was released this week and makes the use of better NPE messages automatic. The release notes associated with this early access build state: "The default of the flag ShowCodeDetailsInExceptionMessages was changed to 'true'. The helpful NullPointerException messages of JEP 358 are now printed by default. The messages contain snippets of the code where the NullPointerException was raised."

The next screen snapshot demonstrates that the helpful NullPointerException details are provided automatically with JDK 15 Early Access Build #29.

The release notes also point out that one potential risk of having the "helpful" NullPointerException messages written by default is the accidental exposure of sensitive details. The release notes warn: "App deployers should double check the output of their web applications and similar usage scenarios. The NullPointerException message could be included in application error messages or be displayed by other means in the app. This could give remote attackers valuable hints about a potential vulnerable state of the software components being used."

The next screen snapshot demonstrates that automatic presentation of helpful NullPointerException details can be disabled via the use of java launcher option -XX:-ShowCodeDetailsInExceptionMessages (and the old -XX:+ShowCodeDetailsInExceptionMessages is still supported even though this is now the default):

One on the interesting consequences of the JDK-8233014 change to make helpful NullPointerException messages enabled by default is that there will undoubtedly be some Java developers pleasantly surprised when they upgrade to JDK 15 and start seeing suddenly far more useful messages when encountering the ubiquitous NullPointerException.

Thursday, June 4, 2020

Tycoon: Ransomware Targeting Java's JIMAGE on Multiple Platforms

The Blackberry Research and Intelligence Team and KPMG's UK Cyber Response Services Team have reported "Threat Spotlight: Tycoon Ransomware Targets Education and Software Sectors." This report outlines the "multi-platform Java ransomware targeting Windows and Linux that has been observed in-the-wild since at least December 2019" and which they've called "Tycoon."

The report provides a high level description of how the Tycoon ransomware is executed: "Tycoon ransomware comes in form of a ZIP archive containing a Trojanized Java Runtime Environment (JRE) build. The malware is compiled into a Java image file (JIMAGE) located at lib\modules within the build directory." The report describes the "sparsely documented" JIMAGE "file format file format that stores custom JRE images which is designed to be used by the Java Virtual Machine (JVM) at runtime." Additional high-level overviews of the JIMAGE file format can be found in JIMAGE - Java Image File Format, How Modules Are Packaged in Java 9, What Is a Custom Runtime Image in Java 9?, So what is a .jimage?, Alan Bateman's description, and slide 49 of JDK 9 Java Platform Module System. The JIMAGE format was introduced with JDK 9 modularity (Project Jigsaw).

Alan Bateman (owner of JEP 220 ["Modular Run-Time Images"])has explained why it's difficult to find documentation on the JIMAGE format: "The format is deliberately not documented and best to assume it will change and evolve over time as it inhales new optimizations." The jdk.jlink module documentation provides very brief mention of the jimage command-line tool and mentions that it is used "for inspecting the JDK implementation-specific container file for classes and resources." It also points out that there is no API for accessing jimage, which is different than for the module's two other tools (jlink and jmod). Similarly, there is no jimage tool reference on the "Java Development Kit Version 14 Tool Specifications" page even though jlink, jmod, and many other tools are featured there.

The previously mentioned report states that "the ransomware is triggered by executing a shell script that runs the Main function of the malicious Java module using the java -m command." The report also talks about Tycoon using Image File Execution Options Injection with Windows registry and then appearing to target Windows and Linux: "The malicious JRE build contains both Windows and Linux versions of this script." This reports adds that Tycoon's "Trojanized Java Runtime Environment (JRE) build" exists as a ZIP file placed in lib/module.

The "Conclusions" section of "Tycoon Ransomware Targets Education and Software Sectors" makes some interesting conclusions based on research observations. In particular, I find it interesting to read why they believe Tycoon in a targeted attack. The authors also point out, "This is the first sample we've encountered that specifically abuses the Java JIMAGE format to create a custom malicious JRE build."

I recommend reading the report "Threat Spotlight: Tycoon Ransomware Targets Education and Software Sectors" directly, but for those interested in summaries and others' observations related to this report, the following may be of interest.

The discovery of Tycoon is likely to bring significantly more attention to JIMAGE than ever before.