JDK 9: Highlights from The State of the Module System

Mark Reinhold's The State of the Module System (SOMS) was published earlier this month and provides an information-packed readable "informal overview of enhancements to the Java SE Platform prototyped in Project Jigsaw and proposed as the starting point for JSR 376." In this post, I summarize and highlight some of concepts and terms I found interesting while reading the document.

• The State of the Module System states that a subset of the features discussed in the document will be used regularly by Java developers. These features and concepts are "module declarations, modular JAR files, module graphs, module paths, and unnamed modules."
• A module is a "fundamental new kind of Java program component" that is "a named, self-describing collection of code and data."
• "A module declares which other modules it requires in order to be compiled and run."
  • requires keyword seems analogous to OSGI's Require-Bundle or Import-Package
• "A module declares which ... packages it exports" to other modules.
  • exports keyword seems analogous to OSGI's Export-Package.
• A module declaration is "a new construct of the Java programming language" that provides "a module’s self-description."
  • Convention is to place "source code for a module declaration" in a "file named module-info.java at the root of the module’s source-file hierarchy."
  • This module-info.java file specification of requires and exports is analogous to how OSGi uses the JAR MANIFEST.MF file to specify Import-Package and Export-Package.
• "Module names, like package names, must not conflict."
• "A module’s declaration does not include a version string, nor constraints upon the version strings of the modules upon which it depends."
• "A modular JAR file is like an ordinary JAR file in all possible ways, except that it also includes a module-info.class file in its root directory."
• "Modular JAR files allow the maintainer of a library to ship a single artifact that will work both as a module, on Java 9 and later, and as a regular JAR file on the class path, on all releases."
• "The base module defines and exports all of the platform’s core packages," "is named java.base," is "the only module known specifically to the module system," "is always present," is depended upon by all other modules, and depends on no other modules.
• All "platform modules" begin with the "java." prefix and "are likely to include "java.sql for database connectivity, java.xml for XML processing, and java.logging for logging."
• The prefix "jdk." is applied to the names of "modules that are not defined in the Java SE 9 Platform Specification," but are "specific to the JDK."
• Implied Readability: The keyword public can be added after the requires keyword to state that a given module's readable module can be read by dependent modules that read it. In other words, if module B references a package provided by module C as requires public, then that package is readable by module A that can read module B.
• "The loose coupling of program components via service interfaces and service providers" is facilitated in the Java module system by use of the keywords provides ... with ... to indicate when a module provides an implementation of a service and by the use of the keyword uses to indicate when a module uses a provided service.
• Because a given class is associated with a single module, Class::getModule() will allow access to a class's associated module.
• "Every class loader has a unique unnamed module" from which types are loaded that are not associated with packages exposed by a module. A given class loader's unnamed module can be retrieved with new method ClassLoader::getUnnamedModule.
  • An unnamed module can read all other modules and can be read by all other modules.
  • Allows existing classpath-based applications to run in Java SE 9 (backwards compatibility).
• "JMOD" is the "provisional" name for a "new artifact format" that "goes beyond JAR files" for holding "native code, configuration files, and other kinds of data that do not fit naturally ... into JAR files." This is currently implemented as part of the JDK and potentially could be standardized in Java SE at a later point.

The items summarized above don't include the "Advanced Topics" covered in "The State of the Module System" such as qualified exports, increasing readability, and layers. The original document is also worth reading for its more in-depth explanations, brief code listings, and illustrative graphics.

Project Jigsaw and OSGi

Project Jigsaw, like OSGi, aims for greater modularity in Java-based applications. I look forward to seeing if the built-in modularity support can provide some of the same advantages that OSGi provides while at the same time eliminating or reducing some of the disadvantages associated with OSGi. In the article Mule Drop OSGi For Being Too Complex, Jessica Thornsby has summarized some developers' thoughts regarding the perceived disadvantage of OSGi that have led Spring and Mule, among others, to stop using OSGi. The Thornsby article quotes Dmitry Sklyut, Kirk Knoerschild, and Ian Skerrett, who suggest that better tooling, better documentation (including by the community), better exposure at conferences, and more familiarity through use would help OSGi adoption and help overcome the perceived steep learning curve and complexity.

I will be curious to see if having modularity built-in to the Java platform will almost automatically bring some of the things that OSGi advocates have argued would increase OSGi's adoption. I suspect that Project Jigsaw, by being built into the platform will have better tooling support, better exposure to general Java developers, and will be more widely and generally covered in the Java developer community (blogs, conferences, books, etc.). With these advantages, I also wonder if Java 9 and Jigsaw will cause current users of OSGi to move away from OSGi or if those users will find creative ways to use the two together or will do what they can (such as use of unnamed modules) to use OSGi instead of Jigsaw. Because OSGi works on versions of Java prior to Java 9 and Jigsaw will only work on Java 9 and later, there will probably be no hurry to move OSGi-based applications to Jigsaw until Java 9 adoption heats up. An interesting discussion on current and forthcoming Java modularity approaches is available in Modularity in Java 9: Stacking up with Project Jigsaw, Penrose, and OSGi.

Cited / Related Resources

• The State of the Module System
• The Gist of Mark Reinhold's 'State of Java 9's Modules' (includes video)
• The Java Module system: a first look
• Jigsaw Quick Start
• Modularity in Java 9: Stacking up with Project Jigsaw, Penrose, and OSGi
• NEW: Oracle Publishes Report on the State of Java’s Module System

JAR Manifest Class-Path is Not for Java Application Launcher Only

I've known almost since I started learning about Java that the Class-Path header field in a Manifest file specifies the relative runtime classpath for executable JARs (JARs with application starting point specified by another manifest header called Main-Class). A colleague recently ran into an issue that surprised me because it proved that a JAR file's Manifest's Class-Path entry also influences the compile-time classpath when the containing JAR is included on the classpath while running javac. This post demonstrates this new-to-me nuance.

The section "Adding Classes to the JAR File's Classpath" of the Deployment Trail of The Java Tutorials states, "You specify classes to include in the Class-Path header field in the manifest file of an applet or application." This same section also states, "By using the Class-Path header in the manifest, you can avoid having to specify a long -classpath flag when invoking Java to run the your application." These two sentences essentially summarize how I've always thought of the Class-Path header in a manifest file: as the classpath for the containing JAR being executed via the Java application launcher (java executable).

It turns out that the Class-Path entry in a JAR's manifest affects the Java compiler (javac) just as it impacts the Java application launcher (java). To demonstrate this, I'm going to use a simple interface (PersonIF), a simple class (Person) that implements that interface, and a simple class Main that uses the class that implements the interface. The code listings are shown next for these.

PersonIF.java

public interface PersonIF
{
   void sayHello();
}

Person.java

import static java.lang.System.out;

public class Person implements PersonIF
{
   public void sayHello()
   {
      out.println("Hello!");
   }
}

Main.java

public class Main
{
   public static void main(final String[] arguments)
   {
      final Person person = new Person();
      person.sayHello();
   }
}

As can be seen from the code listings above, class Main depends upon (uses) class Person and class Person depends upon (implements) PersonIF. I will intentionally place the PersonIF.class file in its own JAR called PersonIF.jar and will store that JAR in a (different) subdirectory. The Person.class file will exist in its own Person.jar JAR file and that JAR file includes a MANIFEST.MF file with a Class-Path header referencing PersonIF.jar in the relative subdirectory.

I will now attempt to compile the Main.class from Main.java with only the current directory on the classpath. I formerly would have expected compilation to fail when javac would be unable to find PersonIF.jar in a separate subdirectory. However, it doesn't fail!

This seemed surprising to me. Why did this compile when I had not explicitly specified PersonIF.class (or a JAR containing it) as the value of classpath provided via the -cp flag? The answer can be seen by running javac with the -verbose flag.

The output of javac -verbose provides the "search path for source files" and the "search path for class files". The "search path for class files" was the significant one in this case because I had moved the PersonIF.java and Person.java source files to a completely unrelated directory not in those specified search paths. It's interesting to see that the search path for class files (as well as the search path for source files) includes archive/PersonIF.jar even though I did not specify this JAR (or even its directory) in the value of -cp. This demonstrates that the Oracle-provided Java compiler considers the classpath content specified in the Class-Path header of the MANIFEST.MF of any JAR on specified on the classpath.

The next screen snapshot demonstrates running the newly compiled Main.class class and having the dependency PersonIF.class picked up from archive/PersonIF.jar without it being specified in the value passed to the Java application launcher's java -cp flag. I expected the runtime behavior to be this way, though admittedly I had never tried it or even thought about doing it with a JAR whose MANIFEST.MF file did not have a Main-Class header (non-executable JAR). The Person.jar manifest file in this example did not specify a Main-Class header and only specified a Class-Path header, but was still able to use this classpath content at runtime when invoked with java.

The final demonstration for this post involves removing the Class-Path header and associated value from the JAR file and trying to compile with javac and the same command-line-specified classpath. In this case, the JAR containing Person.class is called Person2.jar and the following screen snapshot demonstrates that its MANIFEST.MF file does not have a Class-Path header.

The next screen snapshot demonstrates that compilation with javac fails now because, as expected, PersonIF.class is not explicitly specified on the classpath and is no longer made available by reference from the MANIFEST.MF Class-Path header of a JAR that is on the classpath.

We see from the previous screen snapshot that the search paths for source files and for class files no longer include archive/PersonIF.jar. Without that JAR available, javac is unable to find PersonIF.class and reports the error message: "class file for PersonIF not found."

General Observations

• The Class-Path header in a MANIFEST.MF file has no dependency on the existence of a Main-Class header existing in the same JAR's MANIFEST.MF file.
  • A JAR with a Class-Path manifest header will make those classpath entries available to the Java classloader regardless of whether that JAR is executed with java -jar ... or is simply placed on the classpath of a larger Java application.
  • A JAR with a Class-Path manifest header will make those classpath entries available to the Java compiler (javac) if that JAR is included in the classpath specified for the Java compiler.
• Because the use of Class-Path in a JAR's manifest file is not limited in scope to JARs whose Main-Class is being executed, class dependencies can be potentially inadvertently satisfied (perhaps even with incorrect versions) by these rather than resolving explicitly specified classpath entries. Caution is advised when constructing JARs with manifests that specify Class-Path or when using third-party JARs with Class-Path specified in their manifest files.
• The importance of the JAR's manifest file is sometimes understated, but this topic is a reminder of the usefulness of being aware of what's in a particular JAR's manifest file.
• This topic is a reminder of the insight that can be gleaned from running javac now and then with the -verbose flag to see what it's up to.
• Whenever you place a JAR on the classpath of the javac compiler or the java application launcher, you are placing more than just the class definitions within that JAR on the classpath; you're also placing any classes and JARs referenced by that JAR's manifest's Class-Path on the classpath of the compiler or application launcher.

Conclusion

There are many places from which a Java classloader may load classes for building and running Java applications. As this post has demonstrated, the Class-Path header of a JAR's MANIFEST.MF file is another touch point for influencing which classes the classloader will load both at runtime and at compile time. The use of Class-Path does not affect only JARs that are "executable" (have a Main-Class header specified in their manifest file and run with java -jar ...), but can influence the loaded classes for compilation and for any Java application execution in which the JAR with the Class-Path header-containing manifest file lies on the classpath.

Passing Arrays to a PostgreSQL PL/pgSQL Function

It can be handy to pass a collection of strings to a PL/pgSQL stored function via a PostgreSQL array. This is generally a very easy thing to accomplish, but this post demonstrates a couple of nuances to be aware of when passing an array to a PL/pgSQL function from JDBC or psql.

The next code listing is for a contrived PL/pgSQL stored function that will be used in this post. This function accepts an array of text variables, loops over them based on array length, and reports these strings via the PL/pgSQL RAISE statement.

printStrings.sql

CREATE OR REPLACE FUNCTION printStrings(strings text[]) RETURNS void AS $printStrings$
DECLARE
   number_strings integer := array_length(strings, 1);
   string_index integer := 1;
BEGIN
   WHILE string_index <= number_strings LOOP
      RAISE NOTICE '%', strings[string_index];
      string_index = string_index + 1;
   END LOOP;
END;
$printStrings$ LANGUAGE plpgsql;

The above PL/pgSQL code in file printStrings.sql can executed in psql with \ir as shown in the next screen snapshot.

The syntax for invoking a PL/pgSQL stored function with an array as an argument is described in the section "Array Value Input" in the PostgreSQL Arrays documentation. This documentation explains that "general format of an array constant" is '{ val1 delim val2 delim ... }' where delim is a delimited of comma (,) in most cases. The same documentation shows an example: '{{1,2,3},{4,5,6},{7,8,9}}'. This example provides three arrays of integral numbers with three integral numbers in each array.

The array literal syntax just shown is straightforward to use with numeric types such as the integers in the example shown. However, for strings, there is a need to escape the quotes around the strings because there are already quotes around the entire array ('{}'). This escaping is accomplished by surrounding each string in the array with two single quotes on each side. For example, to invoke the stored function just shown on the three strings "Inspired", "Actual", and "Events", the following syntax can be used in psql: SELECT printstrings('{''Inspired'', ''Actual'', ''Events''}'); as shown in the next screen snapshot.

Arrays can be passed to PL/pgSQL functions from Java code as well. This provides an easy approach for passing Java collections to PL/pgSQL functions. The following Java code snippet demonstrates how to call the stored function shown earlier with JDBC. Because this stored function returns void (it's more like a stored procedure), the JDBC code does not need to invoke any CallableStatement's overridden registerOutParameter() methods.

JDBC Code Invoking Stored Function with Java Array

final CallableStatement callable =
   connection.prepareCall("{ call printstrings ( ? ) }");
final String[] strings = {"Inspired", "Actual", "Events"};
final Array stringsArray = connection.createArrayOf("varchar", strings);
callable.setArray(1, stringsArray);
callable.execute();
callable.close();

Java applications often work more with Java collections than with arrays, but fortunately Collection provides the toArray(T[]) for easily getting an array representation of a collection. For example, the next code listing is adapted from the previous code listing, but works against an ArrayList rather than an array.

JDBC Code Invoking Stored Function with Java Collection

final CallableStatement callable =
   connection.prepareCall("{ call printstrings ( ? ) }");
final ArrayList<String> strings = new ArrayList<>();
strings.add("Inspired");
strings.add("Actual");
strings.add("Events");
final Array stringsArray =
   connection.createArrayOf(
      "varchar",
      strings.toArray(new String[strings.size()]));
callable.setArray(1, stringsArray);
callable.execute();
callable.close();

Conclusion

The ability to pass an array as a parameter to a PostgreSQL PL/pgSQL stored function is a straightforward process. This post specifically demonstrated passing an array of strings (including proper escaping) to a PL/pgSQL stored function from psql and passing an array of Strings to a PL/pgSQL stored function from JDBC using java.sql.Array and Connection.createArrayOf(String, Object[]).

The Latest Twist in the Java IDE Wars: Subscription-based IntelliJ IDEA

I've been watching with some interest the discussion surrounding this past week's announcement that JetBrains is moving to an Adobe-like and Microsoft Office-like software subscription licensing model. The feedback has been fast and furious and JetBrains has responded with a very brief follow-up post We are listening that is entirely reproduced in this quote: "We announced a new subscription licensing model and JetBrains Toolbox yesterday. We want you to rest assured that we are listening. Your comments, questions and concerns are not falling on deaf ears. We will act on this feedback."

There are several places to see the reaction, positive and negative, to this announcement. The feedback comments on both the original announcement post and on the follow-up post are good places to start. There are also Java subreddit threads JetBrains switches to subscription model for tools (101 comments currently) and Are you sticking with IntelliJ IDEA or you are moving on to another IDE? (180 comments currently). I was going to summarize some of the pros and cons of this announcement, but Daniel Yankowsky has done such a good job of this that I'll simply reference his post How JetBrains Lost Years of Customer Loyalty in Just a Few Hours.

After reading these posts, it is clear that the change announced by JetBrains would benefit some consumers but might cost other consumers more and, in some cases, quite a bit more. It all seems to depend on what each individual user actually uses and how he or she uses it. In many ways, this makes me think of my most recent Microsoft Office purchase. I purchased Microsoft Office 2013 for my new PC outright rather than via subscription. It would take 2-3 years of subscription payments to meet or exceed the one-time payment I made for Office, but I anticipate needing few new features in Office in the life of this computer. In fact, I have older versions of Microsoft Office running on older computers and am happy with them. It seems that subscriptions to any software product benefit those who need or strongly desire new functions and features and are more expensive for those who are happy with the current functionality set of a particular product. I personally like that Microsoft allows consumers to choose and either buy the software license outright or use a subscription. Having choice in the matter seems to be what's really best for the consumer.

JetBrains provides numerous tools for a wide variety of programming languages and frameworks, but there are also "substitute products" available for most of these. For example, in the Java world, IDEA competes with freely available open source competitors NetBeans and Eclipse. There is, of course, JetBrains's own freely available Community Edition of IDEA that can be seen as a substitute for the Java developer.

The fact that JetBrains can sell its IDEA IDE when there are good alternative Java IDEs available in NetBeans and Eclipse is evidence that many Java developers like what IntelliJ IDEA has to offer enough to pay for it. I am interested to see how the subscription model will affect this balance. JetBrains's relatively quick follow-up announcement indicates that they are considering at least some tweaks to the announced policy already.

Some of the arguments being made against the announcement seem to be out of principle rather than against the actual cost. The JetBrains Toolbox costs are shown currently to be $19.90/month (USD) for existing users to use all tools, just under $12 (USD) for existing uses of IntelliJ IDEA Ultimate Edition, and just under $8/month (USD) for existing users of WebStorm. Those who are not existing users can expect to pay a bit more per month.

One mistake I think was made in the announcement is typical of consumer-facing companies: presenting a change that has obvious benefits for the vendor as if the change is being made primarily for the benefit of the consumer. Although there seems to be benefits for many of the consumers of this announcement, these changes also do not benefit a large number of consumers. There's no question these changes have advantages for JetBrains and it does seem difficult to believe that the changes were driven more by consumers' interests than out of the company's interests. It is not unusual for companies to present changes that benefit themselves as being made primarily in the consumers' interest, but that doesn't mean we like it to hear it presented like that. I think this tone can lead to people reacting negatively to the announcement out of principle.

There are also some interesting articles on software subscription pricing models. Joshua Brustein wrote in Adobe's Controversial Subscription Model Proves Surprisingly Popular that "[Adobe] is making more money selling monthly subscriptions to its Creative Cloud software—the family of programs that includes Photoshop and Illustrator—than it is by selling the software outright." In Software Makers' Subscription Drive, Sam Grobart wrote that "in 2013 consumer software companies proved they could pull off the switch from one-time software purchases to an online subscriber model that costs customers more long term." That article discusses the advantages and disadvantages of consumer software subscriptions from users' and sellers' perspectives.

I'll be watching developments related to JetBrains's announcement in the short-term and long-term to see what effects result from the change in licensing. I'm particularly interested in how this affects IntelliJ IDEA in the Java IDE/text editor space where IntelliJ IDEA Ultimate will continue to compete with IntelliJ IDEA Community Edition, NetBeans, Eclipse, JDeveloper, Sublime, and more.