The Brief but Complicated History of JDK 12's String::transform Method

It was recently proposed that the Java preview feature Raw String Literals (JEP 326) be removed from JDK 12 and it is now official that the preview feature will be removed (version 25 of Java SE 12 [JSR 386] removes it). Several methods had been added to the JDK String class to support this feature. Those methods that were added to versions of the JDK prior to JDK 12 [such as String::lines] are likely to remain available even after the raw string literals preview feature has been removed. However, it has already been decided that one method added to String in JDK 12 (String::align) should be removed from JDK 12 as part of removing raw string literals. The method String::transform was added to JDK 12 and the remainder of this post looks at String::transform, as currently implemented in JDK 12, in more detail and discusses why its already controversial short history implies that it could be a potential candidate for removal along with raw string literals.

The current String::transform implementation has been available in JDK 12 Early Access Builds since Build 22 (Build 24 [15 December 2018] is latest available build as of this writing) and was introduced via JDK-8203442 ("String::transform").

There was quite a bit of discussion related to this method being added to the JDK. The following bullets outline key discussion points.

• Jim Laskey wrote that the "originating goal" of String::transform was to "allow custom alignment methods for those developers not satisfied with String::align()"
  • String::align has been added to and then removed from JDK 12 since that post on String::transform.
• Other messages further describe the motivation for, intent of, and benefits of String::transform:
  • Rémi Forax wrote, "...it's nice to be able to be able to write code fluently from left to right..."
  • Jim Laskey wrote, "String::transform was intended to facilitate custom manipulation (alignment) of raw string literals, in the most string generalized way."
  • The "Description" of JDK-8203442 states, "The String::transform instance method allows the application of a lambda function to a string."
  • JDK-8203703 supplies examples to illustrate that "...steps can be discerned more clearly" with String::transform than with static methods in which the "reader is forced to interpret portions of the expression from the inside out."
• String::transform originally returned String, but then was changed to return Object and Jim Laskey wrote about that change, "'transform' became generic when the case was made that other types might also be relevant." He concluded, "I might be led back to just supporting String."
• The naming of String::transform has been challenging with some of the following names proposed (listed in alphabetic order):
  • "apply"
  • "applyMutation" (used by AWS SDK, but not everyone likes it)
  • "asInputTo"
  • "chain"
  • "map" (was named this temporarily but there were concerns)
  • "process"
  • "with"
• Rémi Forax has written that "more variants (transformToInt, transformToLong, transformToDouble) [are needed] to be useful."
• Brian Goetz has described why the current plan is to implement this functionality via the method String::transform rather than an operator such as |>.
• Stuart Marks has written that "this particular decision [String::transform] sets a precedent for the use of the name 'transform' for methods that do similar things on other classes" and references JDK-8140283 and JDK-8214753:
  • JDK-8140283 proposes the addition of the "chain" method for Stream and Optional to "mitigate" the "disrupt[ion of] the linear flow of the pipeline stages" when using methods that acts upon a Stream or Optional and return something that is itself "chainable").
  • JDK-8214753 proposes the addition of "Optional::transform" that would allow for "an arbitrary operation on an Optional."
• There was some confusion and consternation related to how String::transform was added to OpenJDK 12, but Stuart Marks's message summarizes the events leading to the addition of this method.
  • A particularly interesting sentence in Marks's message states (I have added the emphasis): "While this API point stands on its own, this is really part of Jim's RSL work which includes several API additions to String, and which will likely have a significant effect on how String literals are used in Java code."
• Tomasz Linkowski has pointed out that it's likely that String::transform (and any similar method added to Stream) will get used in select cases where there are easier ways to do the same thing already without the new method. The examples he provides of potential misuse of String::transform are "string.transform(String::toLowerCase)" and "stream.chain(s->s.map(mapper))".

Two online examples demonstrate how String::transform might be used in its most common use cases:

• JDK-8203703 ("String::transform") provides a "Solution" example that demonstrates how String::transform can improve code readability by allowing for operations acting on Strings to be read in order from left-to-right rather than being read "from the inside out."
• A message on the core-libs-dev mailing list provides an example of using String::transform to convert a String into an instance of a class other than String.

Stephen Colebourne asked the same question I was wondering when I read that raw string literals were to be removed from JDK 12: "Is String::transform going to be removed as well given the removal of raw strings and its controversial nature?" Although I have not seen anything authoritative and definitive regarding whether String::transform will remain in JDK 12, there are three pieces of evidence that lead me to think it will be staying.

1. I have not seen anything saying that String::transform, which is already in JDK 12 as of Early Access Build 22, is to be removed. There are issues written to remove compiler support associated with raw string literals and even to remove another String method (String::align), but I'm not aware of a similar issue written for String::transform.
2. It has been stated that while String::transform was added as part of the raw string literal work, it was also stated that String::transform "stands on its own."
3. The two examples I cited earlier on how to use this method do not rely on or require raw string literals. In other words, the method can be used regardless of the presence or absence of raw string literals.

String::transform has not been around for a long time (less than one year), but it already has significant history. The method is available currently in JDK 12 (since Early Access Build 22) and I suspect it will remain part of String's API despite the removal of raw string literals from JDK 12.

Dropping Raw String Literals from JDK 12

It has been proposed that raw string literals (preview) be dropped from JDK 12 (which enters Rampdown Phase One on December 13). Brian Goetz has written a detailed description of the motivations for dropping this preview feature (JEP 326). There is also discussion on this on the Java subreddit. In the post "JSR 386 (Java SE 12) JEP Propose to Drop: 326: Raw String Literals (Preview)," Iris Clark writes that JEP 326 "of scope 'SE' has been Proposed to Drop for Java SE 12."

In Goetz's explanation for the proposal to remove raw string literals preview functionality from JDK 12, he writes, "The Preview Feature mechanism is intended for features for which there is a high confidence that the feature is 'done', and the likelihood that significant changes would be made before making the feature permanent is low." Goetz adds, "I am no longer convinced that we've yet got to the right set of tradeoffs between complexity and expressiveness, or that we've explored enough of the design space to be confident that the current design is the best we can do. By withdrawing, we can continue to refine the design, explore more options, and aim for a preview that actually meets the requirements of the Preview Feature process (JEP 12)."

Goetz also provides a sample of the feedback items they've received regarding raw string literals preview design and implementation. He concludes the message with the statement, "Discussion on the technical details of this feature can continue to take place on the amber-* lists" (amber-dev, amber-spec-comments, amber-spec-experts, and amber-spec-observers).

It sounds like there are still plans for raw string literals to come to Java, but they'll be implemented differently than they are currently in JDK 12 Early Access Builds.

4 New JEPs Proposed for Java 12

Four JEPs have just been proposed for JDK 12 and they are briefly summarized in this post based on the JEPs' own text descriptions.

JEP 189: Shenandoah: A Low-Pause-Time Garbage Collector (Experimental)

JEP 189 ["Shenandoah: A Low-Pause-Time Garbage Collector (Experimental)"] is described as "a new garbage collection (GC) algorithm ... [that] reduces GC pause times by doing evacuation work concurrently with the running Java threads." The JEP text adds, "Shenandoah is an appropriate algorithm for applications which value responsiveness and predictable short pauses."

JEP 334: JVM Constants API

JEP 334 ["JVM Constants API"]'s "Summary" states that this JEP's goal is to "Introduce an API to model nominal descriptions of key class-file and run-time artifacts, in particular constants that are loadable from the constant pool." This JEP is also proposed for Java SE 12.

JEP 344: Abortable Mixed Collections for G1

JEP 344 ["Abortable Mixed Collections for G1"] aims to "make G1 mixed collections abortable if they might exceed the pause target."

JEP 346: Promptly Return Unused Committed Memory from G1

JEP 346 ["Promptly Return Unused Committed Memory from G1"] is another JEP related to the G1 garbage collector and it aims to "enhance the G1 garbage collector to automatically return Java heap memory to the operating system when idle."

Project Atlantis Proposed for JDK Performance Monitoring and Analysis Improvement Experimentation

The thread "Call For Discussion: New Project: Atlantis" on the OpenJDK discuss mailing list discusses JC Beyler's proposal of an OpenJDK project called "Project Atlantis" that "would provide a venue to explore and incubate monitoring and performance analysis features that could be integrated into the Hotspot JVM and the JVM Tool Interface." This proposed project would include "working, evaluating, and incubating a Thread-Sanitizer implementation for Java."

Beyler's proposal provides background for this proposed project based on work done at Google:

The Google platform team has historically worked on providing new features, including monitoring hooks due to certain monitoring features not being available in a low-overhead, turned on-by-default manner. Therefore, for Google Java users, the platform team augmented or added mechanisms to assess various metrics such as but not limited to:

• Thread Sanitizer support for Java (see JDK-8208520) (though this one specifically is not a low-overhead one, sorry ;-))
• Lock contention profiles to better understand where threads are spending time in locks
• Heap dumping mechanisms to improve heap dump times for large heaps
• Efficient thread tagging to help filter profiling
• etc.

All of the above have added changes to the JDK in various forms and the Atlantis project proposes to investigate, in the open, how these changes could be made general enough to be pushed into the mainline, or should be dropped due to being too specific or could be done using other existing mechanisms.

That same proposal message is not much longer than the quotation above, but does add some details regarding "evaluation criteria" that would be used to determine which investigated features and improvements warranted a JEP for eventual integration into the JDK baseline.

The responses to the Project Atlantis proposal have generally been enthusiastic with a few key discussion points related to expressed concerns:

• Fragmentation of tools related to performance monitoring and analysis.
  • Ensuring that Java Flight Recorder is updated related to Project Atlantis improvements.
• Relationship of Project Atlantis and its mailing list to the OpenJDK Serviceability Group and its mailing list.
• Availability of interested and appropriate personnel to maintain Project Atlantis improvements merged into the JDK baseline.

Beyler provides a summary of the thread's questions and concerns. In the concluding portion of that summary, he writes:

... this project is an attempt to have a venue to create conversations about current internal systems or non-existent ones and that we can see what "sticks" and what doesn't; where it would "stick" potentially, what it would like and then how could be best push it forward with the support of the whole community.

There are several aspects of this proposal that intrigue me. Perhaps the most immediately relevant for me would be anything that could be done to make it easier to analyze large heap dumps. Jonathan Lu has interest in similar capability and writes, "I'm especially interested in the heap dumping part. It costs a lot to create, transfer, and analyze huge heap dump files for my applications."

Based on the initial proposal and discussion, it seems to me that Project Atlantis could be of great benefit to users of OpenJDK (and likely the numerous JDKs that will be based on OpenJDK).

JDK 12's Files.mismatch Method

JDK 12 introduces a new method to the Files class. The method, Files.mismatch(Path,Path), has been introduced to JDK 12 via JDK-8202302 and is available in JDK 12 Early Access Build 20 (same early access build that supports the new {@systemProperty} Javadoc tag).

JDK-8202302 ["(fs) New Files.mismatch method for comparing files"] adds the Files.mismatch(Path,Path) method "to compare the contents of two files to determine whether there is a mismatch between them" and can be used to determine "whether two files are equal." There was talk at one time of adding a Files.isSameContent() method, but it was decided to use Files.mismatch(Path,Parh) because of its consistency "with the Arrays.mismatch and Buffer.mismatch methods."

The next code listing contains a simple Java class that demonstrates the new Files.mismatch(Path,Path) and contrasts it with Files.isSameFile(Path,Path). The full source code is available on GitHub.

package dustin.examples.jdk12.files;

import java.nio.file.Files;
import java.nio.file.Path;

import static java.lang.System.out;

/**
 * Demonstrate {@code Files.mismatch(Path,Path)} introduced with JDK 12
 * and useful for determining if two files have the same content even
 * if they're not the same files.
 */
public class FilesDemo
{
   public static void main(final String[] arguments) throws Exception
   {
      if (arguments.length < 2)
      {
         out.println("USAGE: FilesDemo <file1Name> <file2Name>");
         return;
      }

      final String file1Name = arguments[0];
      final Path file1Path = Path.of(file1Name);
      final String file2Name = arguments[1];
      final Path file2Path = Path.of(file2Name);

      out.println("\nFiles '" + file1Name + "' and '" + file2Name + "' are "
         + (Files.isSameFile(file1Path, file2Path) ? "the" : "NOT the")
         + " same.\n\n");
      out.println("\nFiles '" + file1Name + "' and '" + file2Name + "' are "
         + (Files.mismatch(file1Path, file2Path) == -1 ? "the" : "NOT the")
         + " same content.\n\n");
   }
}

When the above code is executed against various combinations of files, it provides results that are captured in the next table. Note that the Files.mismatch method does not itself return a boolean, but the code shown above adapts its response to the appropriate boolean.

Results of Code Snippet Using Files.isSameFile() and Files.mismatch()
Files Relationship	Files.isSameFile(Path,Path)	Files.mismatch(Path,Path)
Same File	"Same" (true)	"Same" (-1)
Copied File	"Not Same" (false)	"Same" (-1)
Different Files	"Not Same" (false)	"Not Same" (positive integer)
Soft-linked	"Same" (true)	"Same" (-1)
Hard-linked	"Same" (true)	"Same" (-1)
Either File Not Found	FileNotFoundException	FileNotFoundException

The addition of Files.mismatch(Path,Path) is another step in accomplishing JDK-6852033 ["Inputs/Outputs methods to make common I/O tasks easy to do"] and makes it easier to determine when two files that are not the same file are still "equal" or have the same content.

JDK 12 Javadoc Tag for System Properties

JDK 12 Early Access Build 20 (2018/11/15) is available and can be used to try out the new Javadoc tag {@systemProperty}. The new {@systemProperty} Javadoc tag is discussed in the core-libs-dev mailing list message "FYI: new javadoc tag to document system properties" and was introduced in response to JDK-5076751 ["System properties documentation needed in javadocs"].

The {@systemPropery} Javadoc tag displays its contents as normal text in its generated output and makes the contents available to the Javadoc search introduced with JDK 9. This tag is intended to be used for documenting an application's system properties.

The following simple class will be used to demonstrate the new JDK 12 Javadoc tag {@systemProperty}:

package dustin.examples.jdk12.properties;

import static java.lang.System.out;

/**
 * Class with sole purpose to illustrate JDK 12's
 * support for {@literal {@systemProperty}}.
 */
public class PropertiesDemo
{
   /**
    * {@systemProperty blog.title} can be specified to
    * provide a blog title.
    */
   private final static String PROPERTY_NAME = "blog.title";

   public static void main(final String[] arguments)
   {
      final String property = System.getProperty(PROPERTY_NAME);
      out.println("Property " + PROPERTY_NAME + ": " + property);
   }
}

The above code example applies {@systemProperty} to the private attribute PROPERTY_NAME. Because the field if private, the Javadoc tool must be executed with the -private flag to have documentation generated for this field.

The next screen snapshot demonstrates the documentation generated for the simple class using the javadoc command-line tool included in JDK 12 Early Access Build 12 (which did not have support for {@systemProperty}).

The red ovals in the previous screen snapshot show that the {@systemProperty} tag is not handled properly in earlier versions of the JDK. The contents of that tag are NOT displayed and the "search" functionality does not match on the system property name.

The next screen snapshot demonstrates the documentation generated for this same class using the command-line javadoc that comes with JDK 12 Early Access Build 20.

The green ovals in the previous screen snapshot show that {@systemProperty} is better supported in Early Access Build 20 of OpenJDK JDK 12. The contents of that tag are correctly displayed in the Javadoc itself and the search capability now matches on the system property name. It is also worth noting that the text within {@systemProperty} is presented in the HTML with appearance similar to which {@code} is presented.

The addition of {@systemProperty} potentially makes it easier for developers to find relevant descriptions of system properties for an application among Javadoc-generated documentation. The aforementioned post "FYI: new javadoc tag to document system properties" discusses other Javadoc enhancements that could possibly be made to take advantage of this tag. The potential enhancements include "a 'summary page' that lists all the system properties", adding "information regarding the 'scope' of the definition", and allowing "a short description to be included in the {@systemProperty} tag" that "could be included in the search index, the A-Z index, and the summary page."

The Jonathan Gibbons FYI mailing list message that introduces {@systemProperty} also spells out its recommended usage:

Where should the tag be used? The tag should be used in the text of the defining instance of the property. This is where the characteristics of the system property are described, which may include information like: "what is the property for", "how and when is it set", "can it be modified", and so on.

The addition of {@systemProperty} to the Javadoc tool with JDK 12 Early Access Build 20 is a minor thing, but will allow developers to make documentation of important system properties more readily accessible for fellow developers.

Amazon Corretto: Another No-Cost JDK

In the blog post "A Tale of Two Oracle JDKs," I compared and contrasted the two JDKs provided by Oracle: Oracle OpenJDK and Oracle JDK (Java SE). There are numerous other JDK offerings available, most of which are based on OpenJDK. One of these is Amazon Corretto, which is the subject of this post.

Today's "What's New with AWS" post "Introducing Amazon Corretto (Preview)" announces Amazon Corretto as "a no-cost, multiplatform, production-ready distribution of the Open Java Development Kit (OpenJDK)." Amazon Corretto's main page describes what it has to offer: "Corretto comes with long-term support that will include performance enhancements and security fixes." This is significant because it advertises that Amazon will provide performance enhancements and security fixes to its JDK offerings past the 6 months for which Oracle has committed to making performance enhancements and security fixes to each new OpenJDK version.

The first version of Amazon Corretto is currently a preview version and is called Amazon Corretto 8 (which, of course, associates it with OpenJDK 8). The "List of Patches for Amazon Corretto 8 Preview" lists "the patches applied to OpenJDK for the Amazon Corretto 8 Preview." These currently include patches related to OpenJDK issues (including backports from OpenJDK 11) such as JDK-8187123 ["(reflect) Class#getCanonicalName and Class#getSimpleName is a part of performance issue"] and JDK-8213198 ["Not triggering mixed GCs in G1 leaves string table cleanup deferred"].

The Amazon Corretto FAQs presents the question, "What is included in Corretto's long-term support?" The answers to this question communicate that Amazon plans to "provide security updates for Corretto 8 until at least June 2023" and to "support Corretto 11 with quarterly updates until at least August 2024." That same answer tells us that plans are to release Amazon Corretto 11 in the first half of 2019. Current updates for both Amazon Corretto 8 and for Amazon Corretto 11 are planned to be made quarterly until at least their advertised ending month of support (June 2023 for Corretto 8 and August 2024 for Corretto 11).

There has been a lot of miscommunication recently about the required cost of purchasing Java in the future. Amazon Corretto is another supporte JDK distribution that will be free of cost. The Amazon Corretto FAQs feature the question, "Is there any cost associated with using Corretto?" The answers associated with that question are:

Corretto is distributed by Amazon under an Open Source license at no cost to you. It is licensed under the terms of the GNU Public License version 2 with the Class Path Exception (GPLv2 with CPE). Amazon does not charge for its use or distribution.

I highly recommend the Amazon Corretto FAQs to anyone potentially interested in adopting Amazon Corretto. In addition to providing answers about likely questions related to long-term support, licenses, and costs, these FAQs also explain the types of patches Amazon intends to make to OpenJDK distributions, explain the degree of drop-in replaceability of other OpenJDK-based distributions by Amazon Corretto, list the operating systems supported for running Amazon Corretto, and explain how Amazon Corretto can be used with or without use of Amazon Web Services (AWS).

Conclusion

These has been continuing confusion surrounding whether Java is still free or not. Amazon Corretto is another example of an OpenJDK-based JDK binary available free of cost with long-term support advertised.

Additional Resources

• Amazon Corretto
• Amazon Corretto FAQs
• Downloads for Amazon Corretto 8
• Getting Started with Amazon Corretto 8 on Docker images
• aprasadh/corretto-8-docker
• GitHub: corretto/corretto-8
• GitHub: corretto/corretto-8-docker
• AWS debuts Amazon Corretto, a free version of the OpenJDK Java kit
• Amazon Web Services promises to support OpenJDK through 2023 with release of internal tool as new open source project
• Amazon announces Corretto, a *free*, production-ready distribution of OpenJDK with LTS