Monday, July 17, 2017

Java Command-Line Interfaces (Part 8): Argparse4j

Argparse4j is a "Java command-line argument parser library" that its main page describes as "a command line argument parser library for Java based on Python's argparse module." In this post, I will look briefly at using Argparse4j 0.7.0 to process command-line arguments similar to those parsed in the seven earlier posts in this series on command-line processing in Java.

The arguments "definition" stage of command-line processing with Argparse4j can be accomplished via the ArgumentParser interface and its addArgument(String...) method. The return type of the addArgument(String...) method is an instance of the Argument interface. Implementations of that interface (usually ArgumentImpl) provide methods for setting the characteristics of each argument. Because each of these methods returns an instance of Argument, these calls can be chained together in a highly fluent manner. This is demonstrated in the next screen snapshot.

"Definition" Stage with Argparse4j

final ArgumentParser argumentParser =
   ArgumentParsers.newArgumentParser("Main", true);
argumentParser.addArgument("-f", "--file")
              .help("Path and name of file");
argumentParser.addArgument("-v", "--verbose")
              .help("Enable verbose output.");

In the above code listing, an instance of ArgumentParser is instantiated with a static initialization method that expects a String argument representing the script or program name that will be included in usage/help output. The second argument to the ArgumentParsers's newArgumentParse(String, boolean) method specifies that "help" options -h and --help will automatically be supported.

The first argument defined in the above code listing allows for a file path and name to be specified on the command line. The strings "-f" and "--file" are passed to the addArgument(String...) method, meaning that either -f or --file can be used on the command line to specify the file path and name. Three additional methods [dest(String), required(boolean), and help(String)] are called on the instances of Argument created as part of the specification of this first argument. These three methods respectively specify a name by which the argument can be referenced in the code, that the argument must be present on the command line, and the string to be displayed when help is requested for that argument.

The second argument defined in the above code listing passes the strings "-v" and "--verbose" to the addArgument(String...) method to allow this argument to be represented on the command line with either the short or long option name. Like the first argument, this one has the name it will be referenced by in the code set by the dest(String) method and has its string for "help" output specified with the help(String) method. This second argument is not required and so the required(boolean) method is unnecessary here.

The second argument's definition has a few additional methods on Argument called. I used type(Class<T>) to demonstrate the ability to explicitly specify the Java data type expected for the argument. I also needed to specify the combination of the nargs(String) and setConst(Object) methods to specify that the verbosity argument does not need a value provided with the flag. This allows me to specify -v or --verbose with no "true" or "false" after those options expected to be explicitly stated.

The "parsing" stage of command-line processing is supported in argparse4j with a call to the ArgumentParser's parseArgs(String[]) method. The next code listing demonstrates this.

"Parsing" Command Line Arguments with Argparse4j

final Namespace response = argumentParser.parseArgs(arguments);

Parsing requires only a single statement and returns an instance of Namespace.

The "interrogation" stage of command line processing with Argparse4j involves accessing the parsed command-line arguments from the Map that the Namespace instance wraps. The keys of this map are the strings specified with the dest(String) method and the values of the map are the values associated with those argument names. Interrogating these values is demonstrated in the next code listing.

"Interrogating" Command Line Arguments with Argparse4j

final String filePathAndName = response.getString("file");
final Boolean verbositySet = response.getBoolean("verbose");

     "Path/name of file is '" + filePathAndName
   + "' and verbosity is "
   + (Boolean.TRUE.equals(verbositySet) ? "SET" : "NOT SET")
   + ".");

In the just listed code, the keys of "file" and "verbose" were used because those same strings were provided with the dest(String) method when defining the expected arguments.

The full source code from which the code snippets above were extracted can be seen on GitHub.

The next screen snapshot demonstrates running the simple Java application without any arguments and the message that is displayed regarding the missing required "file" argument.

The all uppercase "FILE" shown in the above screen snapshot comes from the string that was specified in the dest(String) method when defining the expected argument. In other words, that dest(String) specification set both the string by which the argument mapping is keyed internally and the target string displayed in the help/usage.

The next screen snapshot demonstrates several variations of typical uses of the "file" and "verbose" options.

The final screen snapshot demonstrates that help information that is provided for -h or --help options because the original instance of ArgumentParser was created with the "addHelp" argument set to true.

Here are some additional characteristics of Argparse4j to consider when selecting a framework or library to help with command-line parsing in Java.

  • Argparse4j is open source and licensed with the MIT License.
  • The argparse4j-0.7.0.jar (December 2015) is approximately 89 KB in size and has no additional third-party library dependencies.
  • Argparse4j does not make use of annotations.
  • The online documentation includes a Clojure example.
  • I suspect that Java developers who write their scripts in Python (particularly if they use argparse) would experience advantages when using argparse4j in their Java applications that need to parse command-line arguments.
    • (I find Apache Commons CLI to be intuitive when processing command-line arguments in Java because I much more frequently parse command line arguments in Groovy than in Java and Groovy supplies built-in Apache Commons CLI support)
  • Argparse4j inspired the development of argparse4s for Scala.

Argparse4j is just one of many Java-based command line processing libraries. The characteristic of Argparse4j that most sets it apart from the numerous other options is its argparse heritage. Given that, I believe that the Java developers most likely to select Argparse4j for their Java command line processing needs would be those developers who frequently parse command line arguments in Python-based scripts and tools using argparse or who prefer command parsing semantics of Python and argparse.

Additional References

Monday, July 10, 2017

Java Command-Line Interfaces (Part 7): JCommander

This is the seventh post in my series that briefly introduces various libraries for processing command-line arguments in Java. This post returns to coverage of an annotation-based library that seems to be one of the better known and more popular of the numerous available libraries for processing command line arguments from Java: JCommander.

JCommander's web page states, "Because life is too short to parse command line parameters" and the Overview introduces JCommander as "a very small Java framework that makes it trivial to parse command line parameters." The code examples and associated screen snapshots of the executing code in this post are based on JCommander 1.72 (June 2017). The full code for the demonstrations shown here is available on GitHub.

JCommander uses annotations to implement the "definition" stage of command-line processing. This is demonstrated in the next code listing snippet.

"Definition" Stage with JCommander

 * Demonstrates use of JCommander for Java-based command-line processing.
public class Main
              description="Enable verbose logging")
   private boolean verbose;

              description="Path and name of file to use",
   private String file;

   @Parameter(names={"-h", "--help"},
   private boolean help;

   // . . .

final JCommander commander
   = JCommander.newBuilder()
              .programName("JCommander Demonstration")

The just-shown code listing demonstrates use of JCommander's @Parameter annotation to define the command-line options via annotation of class fields. The examples demonstrate specification of names to indicate multiple option flags to be associated with a single option, description to provide a description of each option, required=true to enforce presence of a command-line argument, and help=true to indicate a "help" or "usage" command-line argument (instructs JCommander to not throw exception if required arguments are not also provided).

With the class attributes annotated with @Parameter annotations, an instance of the class with annotated fields can be used to create an instance of the JCommander class. In the code example above, I took advantage of the JCommander.Builder for the greater fluency and other advantages associated with use of builders. In particular, the instance with annotated class fields is added via the addObject(Object) method.

The "parsing" stage of command-line processing with JCommander is accomplished via a single line invocation of the parse(String...) method on the instance of JCommander that was just instantiated. This is demonstrated in the next code listing.

"Parsing" Stage with JCommander


The "interrogation" stage of command-line processing with JCommander involves simply accessing the annotated fields of the instance passed to the JCommander class instantiation. This is demonstrated in the next code listing.

"Interrogation" Stage with JCommander

if (help)
      "The file name provided is '" + file + "' and verbosity is set to " + verbose);

The last code listing demonstrates the ability to determine if the boolean attribute with name help was set by the specification of --help or -h. Because it's a simple boolean, it can be used in the conditional and, if true, the help/usage information is presented. In the case when the "help" flag was not set, values associated with the other command line options ("verbose"/-v/--verbose and "file"/-f/--file) are accessed.

The most recent code listing also demonstrates writing the usage information to standard output via an invocation of the method usage() on the instance of the JCommander class. It's worth noting that ParameterException also has a usage() method.

The next series of screen snapshots demonstrate using JCommander with a simple application that includes the above code snippets. The first image shows running the JCommander-based application without any arguments and shows the ParameterException that is displayed in that case because the required --file/-f option was not specified.

The next screen snapshot demonstrates "normal" execution when the expected command line arguments are provided.

The next screen snapshot demonstrates use of the "help" option. Because this was annotated with help=true, the absence of the required "file" command-line argument does not lead to an exception and the automatically generated help/usage information is written to standard output.

JCommander provides a feature that I really like for developing with and learning JCommander. One can specify increased verbosity of the JCommander parsing by invoking the method verbose(int) on JCommandBuilder.

Increasing JCommander's Verbosity

final JCommander commander
   = JCommander.newBuilder()
               .programName("JCommander Demonstration")

With the increased verbosity, greater insight into what JCommander is doing related to command-line processing can be discovered and this is demonstrated in the following two screen snapshots.

Here are some additional characteristics of JCommander to consider when selecting a framework or library to help with command-line parsing in Java.

Additional References

Saturday, July 8, 2017

Java Command-Line Interfaces (Part 6): JOpt Simple

The main web page for JOpt Simple calls this Java-based library "a Java library for parsing command line options, such as those you might pass to an invocation of javac," that "attempts to honor the command line option syntaxes of POSIX getopt() and GNU getopt_long()." This is the sixth post of my series of command-line arguments processing in Java and its focus is on JOpt Simple.

Most of the libraries I've reviewed in this series of command-line processing in Java take use annotations in some way. JOpt Simple, like Apache Commons CLI, does not use annotations. JOpt Simple supports "fluent interfaces" instead. This original post's examples (code listings) and output (screen snapshots) are based on compiling and running against JOpt Simple 4.9, but they have worked similarly for me (and without code changes) when compiling them and running them with JOpt Simple 5.0.3.

The next code listing demonstrates the "definition" stage of command-line processing with JOpt Simple and this example is intentionally similar to that used in the previous posts on command-line processing in Java.

Defining Command-line Options in JOpt Simple

final OptionParser optionParser = new OptionParser();
final String[] fileOptions = {"f", "file"};
optionParser.acceptsAll(Arrays.asList(fileOptions), "Path and name of file.").withRequiredArg().required();
final String[] verboseOptions = {"v", "verbose"};
optionParser.acceptsAll(Arrays.asList(verboseOptions), "Verbose logging.");
final String[] helpOptions = {"h", "help"};
optionParser.acceptsAll(Arrays.asList(helpOptions), "Display help/usage information").forHelp();

This code listing demonstrates use of the "fluent API" approach to defining command-line options. An OptionParser is instantiated and then one of its overloaded acceptsAll methods is called for each potential command-line option. The use of acceptsAll allows multiple flag/option names to be associated with a single option. This support for option synonyms allows for use of "-f" and "--file" for the same option.

The code above demonstrates that a command-line option can be specified as required with the .required() method invocation. In this case, a "file" is required. If an argument is expected to be placed on the command line in association with the option/flag, the withRequiredArg() method can be used. The "help" option in the above code listing takes advantage of the forHelp() method to tell JOpt Simple to not throw an exception if a required option is not on the command-line if the option associated with the forHelp() is on the command-line. This works, in my example, to ensure that the operator could run the application with -h or --help and without any other required option and avoid an exception being thrown.

The JOpt Simple Examples of Usage page provides significant details about the many different possibilities available when defining command-line options and uses JUnit-based assertions to demonstrate how these different tactics for defining command-line options configure differently what is parsed. My code listing shown above only shows a minor subset of what's available. Note that the Javadoc comments for the OptionParser class also contain significant details.

The code above can be even more concise if one statically imports the Arrays.asList and passes the potential command-line options' names as strings directly to that asList(String...) method instead of using the approach I used of creating an array of Strings first and then converting them to a list. I used this approach in this introductory post to make it very clear what was happening, but it's likely that the version of the code associated with this post on GitHub will be changed to use the static import approach.

The "parsing" stage of command-line processing with JOpt Simple is, well, simple:

final OptionSet options = optionParser.parse(arguments);

"Parsing" with JOpt Simple entails invocation of the method OptionParser.parse(String ...)

The "interrogation" stage of command-line processing with JOpt Simple is also simple and is demonstrated in the next code listing.

out.println("Will write to file " + options.valueOf("file") + " and verbosity is set to " + options.has("verbose"));

The single line of code demonstrates that interrogation consists of calling convenient methods on the instance of OptionSet returned by the parsing call. In this case, two demonstrated methods called on OptionSet are OptionSet.valueOf(String) and OptionSet.has(String).

JOpt Simple also supports automatic generation of a usage/help statement. The next code listing demonstrates doing this.


The single line of code just shown writes the usage/help information generated by the instance of OptionParser to the output stream provided to it via its printHelpOn(OutputStream) method.

With the most significant code needed for applying JOpt Simple shown, it's time to see how the simple application that uses this code behaves. The following screen snapshots demonstrate the code in action. The first screen snapshot demonstrates the MissingRequiredOptionsException printed when the required "file" command-line option is not provided.

The next screen snapshot demonstrates specifying the "file" and "verbose" options on the command lines.

The automatic usage/help message provided by JOpt Simple is demonstrated in the next screen snapshot.

Here are some additional characteristics of Apache Commons CLI to consider when selecting a framework or library to help with command-line parsing in Java.

  • JOpt Simple is open source and is licensed under the MIT License.
  • As of this writing, the latest versions of JOpt Simple are 5.0.3 and 6.0 Alpha 1; JOpt Simple 4.9 (latest version currently listed in the JOpt Simple change log and version currently shown in Maven dependency example) was used in this post.
  • The jopt-simple-4.9.jar is approximately 65 KB in size and has no compile-time dependencies on any third-party libraries.
  • JOpt Simple has been or is used by several influential libraries and frameworks. These include Spring Framework (optional compile dependency) and JMH (compile dependency).
    • The main page for the JOpt Simple web page quotes Mark Reinhold, "I thought you might be interested to know that we're using your jopt-simple library in the open-source Java Development Kit. Thanks for writing such a nice little library! It's far cleaner than any of the other alternatives out there."
  • JOpt Simple has been available for several years, but appears to still be maintained (latest on Maven Central is December 2016).
  • JOpt Simple does not use annotations and instead relies on fluent API calls.
  • JOpt Simple supports relationships between command-line options such as required dependent options.

It is typically a positive sign of a library's usefulness when other well-received and useful tools and libraries make use of that library. JOpt Simple's selection as the command-line processing library of choice for some such tools and libraries definitely speaks well of JOpt Simple. JOpt Simple provides a useful and powerful alternative to Apache Commons CLI for those who prefer Java command-line processing that does not use annotations. JOpt Simple provides significant more capability than that shown in this post and this capability is best discovered by reading the unit test-based "tour through JOpt Simple's features."

Additional References

Friday, June 30, 2017

Java Command-Line Interfaces (Part 5): JewelCli

After looking at command-line processing in Java with Apache Commons CLI, args4j, jbock, and Commandline in previous posts, I turn attention in this post to using JewelCli to accomplish similar processing of command-line arguments in Java.

Several Java command-line processing libraries use annotations to define the command-line options. Three of the four libraries covered in this series of posts so far use annotations and so does JewelCli. JewelCli is unique among the libraries I've covered so far because its annotations are applied on a Java interface rather than on a Java class or class's constructs. The next code listing demonstrates how to use annotations on a Java interface to implement the "definition" stage of command-line parsing with JewelCli.

JewelCli "Definition" Implemented with Annotated Interface

package examples.dustin.commandline.jewelcli;

import com.lexicalscope.jewel.cli.Option;

 * Interface defining JewelCli-friendly command-line parameters.
public interface MainCommandLine
   @Option(shortName="f", description="Name and path of file to be used.")
   String getFile();

   @Option(shortName="v", description="Indicate whether status should be reported verbosely.")
   boolean isVerbose();

   @Option(helpRequest=true, description="Usage details on command-line arguments.")
   boolean getHelp();

The simple interface shown above packs a lot related to command-line processing. The options have their single-hyphen short names explicitly specified with shortName annotation type element and implicitly specified via the name of the "get" method (though a longName annotation type element is available for explicitly specifying the long name [double hyphens] version of the switch). The command-line options also have their respective descriptions provided via the Option annotation. The use of helpRequest=true describes what command-line switch should be used to display usage/help information. In this case, because the annotation method is named getHelp(), the --help switch will display usage information. Had I named the method getDustin() and annotated it with @Option(helpRequest=true), the switch would be --dustin to display usage.

JewelCli takes advantage of convention over configuration in cases besides the long name of the switch matching the method names. With the command-line options' corresponding interface method definitions annotated as shown above, the verbosity switch (which returns a boolean) is optional. The file name switch is required because its corresponding getFile() method returns a String. If I wanted to make the file name optional, I could provide a defaultValue to the @Option annotation on the getFile() method such as @Option(defaultValue="").

With the interface (named MainCommandLine in this case) annotated with JewelCli @Option annotations, we can move to the "parsing" stage with JewelCli. This is demonstrated, along with the "interrogation" stage, in the next code listing for Main.

"Parsing" and "Interrogation" Stages with JewelCli

package examples.dustin.commandline.jewelcli;

import static java.lang.System.out;

import com.lexicalscope.jewel.cli.CliFactory;

 * Demonstrates use of JewelCli for parsing command-line
 * parameters in Java.
public class Main
   public static void main(final String[] arguments)
      final MainCommandLine main = CliFactory.parseArguments(MainCommandLine.class, arguments);
      out.println("You specified file '" + main.getFile() + "' with verbosity setting of '" + main.isVerbose() + "'.");

The Main class just shown has one line that "parses" [the call to CliFactory.parseArguments(Class<T>, String...)] and one line that "interrogates" [the line that accesses the methods defined on the JewelCli-annotated interface shown earlier].

The following three screen snapshots demonstrate the JewelCli-based code examples in action. The first image demonstrates use of --help to see usage (notice that a stack trace is included in the output). The second image shows different combinations of long (-) and short (--) option switches. The third image shows the output message and associated stack trace that are presented when a required command-line argument (--file or -f in this case) is not provided.

The code listings for both classes used in this post to demonstrate application of JewelCli are available on GitHub.

Here are some additional characteristics of JewelCli to consider when selecting a library to help with command-line parsing in Java.

  • JewelCli is open source and licensed under an Apache Software License, Version 2.
  • The current JewelCli (0.8.9) JAR (jewelcli-0.8.9.jar / February 2014) is approximately 542 KB in size.
  • No additional libraries are needed to use JewelCli.
  • As shown in the example above, JewelCli uses annotations on Java interfaces for the "definition" stage. Any attempt to annotate class "get" methods in a similar way results in a message such as "IllegalArgumentException: ... is not an interface" at runtime.
  • JewelCli allows interfaces to inherit from super interfaces and @Options defined in parent interfaces will be supported in the inheriting interfaces.
  • The return data types of the methods annotated in the interface provide type enforcement of the command-line options' values. Enums can even be used as return data types to narrow down the possible command-line option types to a finite set of possibilities.

JewelCli is easy to use and, thanks to its convention over configuration approach, requires very little code to define, parse, and interrogate command-line arguments. I find the recommended approach of annotating an interface for defining the parseable command-line options to be aesthetically pleasing as well.

Additional Resources

Wednesday, June 28, 2017

Java Command-Line Interfaces (Part 4): Commandline

This fourth part of my series on command-line parsing in Java features Commandline, which is described as "a Java library to parse command line arguments" that "is based on a mapping from the command line arguments onto objects, by using annotations."

Like previously covered args4j and jbock, Commandline employs annotations to provide the "definition" of potential command-line options. However, while args4j does this via annotations on class fields and jbock does this via annotations on the constructor and its parameters, Commandline uses annotations on "set" (mutator) methods. In this post, I use a Main class example as in the previous posts on Java-based command-line processing, but in normal situations, I'd typically prefer to have a special class representing command-line arguments.

The following code listing demonstrates use of Commandline annotations on "get" methods to implement the "definition" stage of Commandline's command-line processing.

Commandline "Definition" Stage of Command-line Processing

public class Main
   /** Is verbosity enabled? */
   private boolean verbose;

   /** Name/path of applicable file. */
   private String fileName;

   public void setVerbose(final boolean newVerbose)
      verbose = newVerbose;

   public void setFileName(final String newFileName)
      fileName = newFileName;

The code listing above shows the use of Commandline annotation @Option along with other annotations that customize the defined option (@ShortSwitch and @LongSwitch for short and long arguments, @Required for mandatory arguments, @SingleArgument to specify one argument associated with switch, and @Toggle to indicate that the presence or absence of the switch is what's significant [no argument associated with that switch]).

The next code listing demonstrates the "parsing" and "interrogation" stages of command-line parsing with Commandline.

"Parsing" and "Interrogating" with Commandline

   // "Parsing" stage.
   final Main main = CommandLineParser.parse(
      Main.class, arguments, OptionStyle.LONG_OR_COMPACT);

   // "Interrogation" stage.
   out.println("You provided file name of '" + main.fileName
      + "' and verbose is set to '" + main.verbose + "'.");
catch (IllegalAccessException | InstantiationException | InvocationTargetException exception)
   out.println("ERROR: Unable to parse command-line arguments: " + exception);

The last code example demonstrates that parsing is accomplished with the single CommandLineParser.parse(Class<T>, String[], OptionStyle) call and interrogation is as simple as accessing the members of the instance returned by that method. The third argument provided to the parse method is significant because it instructs the parser how to expect the switches to be presented.

The code example just shown uses OptionStyle.LONG_OR_COMPACT, which the documentation describes: "Long switches are prepended by two dashes. Short switches are prepended with a single dash, and may be concatenated into one switch." In contrast, OptionStyle.SIMPLE instructs the parser to expect "All switches have to be standalone" and "all (both long and short) needs to be prepended with a single dash on the command line."

The next screen snapshot demonstrates this simple application in action.

When a required argument is not provided, a message (including stack trace) like that shown in the next screen snapshot is presented.

I don't include an example of using help or usage based on Commandline here because, as the project's GitHub page states, "Generating a help text" is "functionality that is currently not supported." The complete code listing for the class used in this post to demonstrate use of Commandline is available on GitHub.

Here are some additional characteristics of Commandline to consider when selecting a library to help with command-line parsing in Java.

Commandline is another of the plethora of open source command-line processing libraries available to Java developers and, like several of the others, uses annotations to do most of the heavy lifting.

Additional References

Monday, June 26, 2017

Java Command-Line Interfaces (Part 3): jbock

In the first two posts of this series on command-line parsing in Java, I looked at the Apache Commons CLI and args4j libraries. In this third post in the series, I look at jbock, the self-described "curiously simple CLI parser."

My posts on command-line parsing in Java have used examples based on providing a required file name and an optional verbose flag to the Java application. The same approach is used in this post to demonstrate jbock 1.8. The full source code for the example class is available on GitHub, but the code generated by jbock (Main_Parser) is not available as it can be generated.

The approach jbock uses for command-line processing is different than that used by the two previously covered parsing libraries. The previously covered libraries required Java code for parsing command-line arguments to be built against and executed against the libraries' JARs. In other words, the libraries' JARs needed to be on both the compile-time (javac) classpath and on the runtime Java launcher (java) classpath. The jbock approach instead relies on inclusion of the jbock JAR only at compile time. The jbock approach generates Java source code that is completely independent of the jbock library. One could, for example, choose to run jbock to generate these Java source code files once and then version control those generated files and only build and run against the generated files from that point on without needing to build or run against jbock's JAR. The only time the jbock JAR is required is when the generated Java source needs to be regenerated. Because the generated code is generated based on annotations on custom Java classes, it is likely that the jbock code generation would be executed in most cases as part of a normal build rather than version controlling the generated source.

In most situations, I'd use a custom class with a name such as "Arguments" or "CommandLine" when using jbock to parse command-line arguments. However, for this post, I am using a simple Main class to be more similar of an example to the approach used with the other command-line parsing libraries in other posts in this series. Like args4j, jbock uses annotations for the "definition" phase of command-line processing. However, jbock's annotations are on the class's constructor and its arguments rather than args4j's approach of annotating class fields. The jbock constructor-based annotations approach is demonstrated in the next code listing.

jbock "Definition" of Command-Line Options

public Main(
   @ShortName('v') @LongName("verbose") @Description("Verbosity enabled?")
   final boolean newVerbose,
   @ShortName('f') @LongName("file") @Description("File name and path")
   final Optional<String> newFileName)
   verbose = newVerbose;
   file = newFileName.orElse("");
// . . .

The "parsing" stage of command-line processing with jbock is demonstrated in the next code listing.

"Parsing" Command-line Options with jbock

final Main_Parser parser = new Main_Parser();
final Main_Parser.Binder binder = parser.parse(arguments);
final Main main = binder.bind();

The Main_Parser class shown in the above code listing is generated by jbock based on the annotations shown in the first code listing. The jbock library processes the annotations of the Main class to determine how to build the Main_Parser class. The generated class's name is based on the name of the class with jbock annotations and concatenated with _Parser. For example, had my class with jbock annotated constructor and constructor arguments been named "Arguments", the generated class would be named "Arguments_Parser".

After the instance of the generated Main_Parser class has had parse invoked on the command-line arguments, that instance's bind() method is invoked to return an instance of the original annotated Main class. The "interrogation" process at this point consists solely of accessing the attributes of that Main instance via its public "get" methods. This is demonstrated in the next code listing.

"Interrogation" Stage of Command-line Processing with jbock

out.println("The file '" + main.getFile() + "' was provided and verbosity is set to '"
   + main.isVerbose() + "'.");

The screen snapshot that follows demonstrates the code in action using jbock to parse the command-line options.

If help or usage information is desired, this can be retrieved from the generated *_Parser (Main_Parser in this case) class as well. Specifically, the generated *_Parser class includes a nested Option enum representing the various options. One can iterate over those option's enum values to retrieve metadata about each option. In the code listing below, the describe(int) method is invoked on each option's enum value (the passed-in integer is the number of spaces to indent).

Obtaining Usage Details with jbock

final Main_Parser parser = new Main_Parser();
if (arguments.length < 1)
   for (final Main_Parser.Option option : Main_Parser.Option.values())

The screen snapshot shown next demonstrates this code in action to print out the options and their descriptions.

The source code discussed in this post is available on GitHub.

Here are some additional characteristics of jbock to consider when selecting a framework or library to help with command-line parsing in Java.

  • jbock is available as open source.
    • UPDATE: jbock creator h908714124 has pointed out in the feedback that jbock is "just a concept project" that is "intentionally minimalistic, so you can actually grok all the parsing rules."
    • UPDATE: h908714124 also invites users of jbock to "open a github issue or make a pull request" if an issue is identified.
  • The current version of jbock (1.8) requires Java SE 8.
  • jbock has no third-party or external dependencies.
  • The jbock 1.8 JAR (jbock-1.8.jar) is approximately 131 KB in size, but this is not as significant as for similar libraries because this JAR is not required at runtime (generated code is independent of the JAR).
  • I did not demonstrate jbock's enforcement of the presence of required command-line parameters because it intentionally does not support that feature. The README states, "Deliberately simple: No converters, default values or required checking. With java 8, it's easy to add this stuff by hand."

The most obvious characteristic of jbock that distinguishes it from most other Java-based command-line parsing libraries is the generation of parsing code entirely at compile time that leaves no runtime dependencies on the jbock library. This would be an obvious advantage in situations where there is concern about the number of classes loaded or the size of the expressed classpath. The README lists multiple items that "set [jbock] apart." These include "no reflection, purely static analysis" and "convenient, flexible property binding via constructor."

Additional References

Thursday, June 22, 2017

Java Command-Line Interfaces (Part 2): args4j

In my previous post, I looked at parsing command-line arguments in Java applications using Apache Commons CLI. In this post, I look at doing the same using a different library: args4j.

args4j takes a different approach to specifying which command-line arguments the Java application should expect than that used by Commons CLI. While Commons CLI expects objects representing the options to be individually and explicitly instantiated, args4j uses custom annotations to facilitate this "definition" stage of command-line arguments processing. Command-line options are expected to be instance-level fields on the class and are annotated with the @org.kohsuke.args4j.Option annotation. The characteristics of each command-line argument are included as attributes of this @Option annotation.

The simple application demonstrated in this post is similar to that used in my previous post and focuses on an optional and valueless -v option for specifying verbosity and a required -f option which expects a value that represents the file path and name. The next code listing demonstrates use of args4j's @Option annotation to set up these command-line argument as annotation on class data members.

args4j Definition of Command-line Arguments via @Option Annotations

@Option(name="-v", aliases="--verbose", usage="Print verbose status.")
private boolean verbose;

@Option(name="-f", aliases="--file", usage="Fully qualified path and name of file.", required=true)
private String fileName;

As the above code listing demonstrates, it is easy to specify the name of the options, their usage, and whether they are required or not (default is optional). The presence of the private modifier above makes it obvious that these are attributes defined at a class level. Because there is no static modifier, we see that these are instance variables that have been annotated.

To parse the command-line options, one simply needs to instantiate a CmdLineParser and pass the command-line arguments to its parseArguments(String...) method:

Parsing Command-line Arguments in args4j

final CmdLineParser parser = new CmdLineParser(this);
catch (CmdLineException clEx)
   out.println("ERROR: Unable to parse command-line options: " + clEx);

In the first line of Java code just shown, this is the reference to the instance of the class in which the member variables shown above are defined and annotated with the @Option annotation. In this case, I used this because the same class that defines those options is the one calling this parsing method. To do this in the same class, I needed to have an instance (non-static) method called doMain defined in the class and invoked by the class's main function (this is shown in the complete code listing toward the end of this post). The command-line arguments as received from the class's main(final String[]) function are the array of Strings passed to the parseArguments(String[]) method.

The next two screen snapshots demonstrate application of the described code based on args4j to parsing the command-line arguments. The first image shows combinations of the short and long options for the two options. The second image shows the automatic reporting of the case where a required command-line argument was not provided.

An important feature of a command line parsing library is the ability to display usage or help information. The next code listing demonstrates an example of doing this with args4j's CmdLineParser.printUsage(OutputStream) method.

Printing Usage Information with args4j

final CmdLineParser parser = new CmdLineParser(this);
if (arguments.length < 1)

The usage information printed out by default by args4j is depicted in the next screen snapshot.

This post has demonstrated using arg4j to achieve some of the most common functionality related to command-line parsing in Java applications including option "definition", command-line arguments "parsing", "interrogation" of the parsed command-line arguments, and help/usage details related to the command-line arguments. The full code listing for the class partially represented above in code listings is shown now (also available on GitHub, which version is subject to tweaks and improvements).

Full Code Listing for args4j Demonstration

package examples.dustin.commandline.args4j;

import static java.lang.System.out;

import org.kohsuke.args4j.CmdLineException;
import org.kohsuke.args4j.CmdLineParser;
import org.kohsuke.args4j.Option;


 * Demonstrate args4j.
public class Main
   @Option(name="-v", aliases="--verbose", usage="Print verbose status.")
   private boolean verbose;

   @Option(name="-f", aliases="--file", usage="Fully qualified path and name of file.", required=true)
   private String fileName;

   private void doMain(final String[] arguments) throws IOException
      final CmdLineParser parser = new CmdLineParser(this);
      if (arguments.length < 1)
      catch (CmdLineException clEx)
         out.println("ERROR: Unable to parse command-line options: " + clEx);
      out.println("The file '" + fileName + "' was provided and verbosity is set to '" + verbose + "'.");

    * Executable function demonstrating Args4j command-line processing.
    * @param arguments Command-line arguments to be processed with Args4j.
   public static void main(final String[] arguments)
      final Main instance = new Main();
      catch (IOException ioEx)
         out.println("ERROR: I/O Exception encountered: " + ioEx);

Here are some additional characteristics of args4j to consider when selecting a framework or library to help with command-line parsing in Java.

  • args4j is open source and licensed with the MIT License.
  • Current version of args4j (2.33) requires J2SE 5.
  • args4j does not require any third-party libraries to be downloaded or referenced separately.
  • The args4j 2.33 main JAR (args4j-2.33.jar) is approximately 152 KB in size.
  • The Maven Repository shows 376 dependencies on args4j including OpenJDK's JMH Core and Jenkins (not surprising given Kohsuke Kawaguchi's involvement in both).
  • args4j has been around for a while; its 2.0.3 release was in January 2006 and it's been around in some form since at least 2003.
  • args4j allows a command-line parameter to be excluded from the usage output via "hidden" on the @Option annotation.
  • args4j allows for relationships between command-line arguments to be specified and enforced. This includes the ability to specify when two arguments cannot be supplied at the same time ("forbids") and when the presence of an argument only makes sense when another argument is also provided ("depends").
  • args4j supports use of enum-typed class attributes for cases where a finite set of values is applicable to the option. The @Option documentation describes how to do this under the "Enum Switch" section.
  • args4j provides extensibility and customizability of command-line arguments parsing via its OptionHandler class.
  • args4j provides internationalization and localization support.

The args4j library is easy to use and allows for highly readable code. Perhaps the biggest consideration when deciding whether to use args4j is deciding how comfortable one is with using annotations for specifying the command-line parameters' definitions.

Additional References

Tuesday, June 20, 2017

Java Command-Line Interfaces (Part 1): Apache Commons CLI

Although I typically use Groovy to write JVM-hosted scripts to be run from the command-line, there are times when I need to parse command-line parameters in Java applications and there is a plethora of libraries available for Java developers to use to parse command-line parameters. In this post, I look at one of the best known of these Java command line parsing libraries: Apache Commons CLI.

I have blogged on Apache Commons CLI before, but that post is over eight years old and describes Apache Commons CLI 1.1. Two classes that I demonstrated in that post, GnuParser and PosixParser, have since been deprecated. The examples in this current post are based on Apache Commons CLI 1.4 and use the newer DefaultParser that was introduced with CLI 1.3 to replace GnuParser and PosixParser.

The Apache Commons CLI documentation's "Introduction" explains how Commons CLI accomplishes the "three stages [of] command line processing" ("definition", "parsing", and "interrogation"). These three stages map in Commons CLI to classes Option and Options ("definition"), to interface CommandLineParser ("parsing"), and to class CommandLine ("interrogation").

For the examples built here with Apache Commons CLI, the expected command-line arguments are relatively simple. One argument is optional and, when specified, indicates that verbose output is enabled. The other argument is required and is used to specify a file to be processed by the imaginary application. The optional argument does not have a value associated with the flag and is expressed as -v or --verbose. The required argument should be followed by a value which is the path and name of the file. This flag is either -f or --file. The next code listing demonstrates using Commons CLI's Option.Builder (introduced with Commons CLI 1.3) to build up the expected options as part of the "definition" stage.

Example of Using Apache Commons CLI Option.Builder for "Definition Stage

 * "Definition" stage of command-line parsing with Apache Commons CLI.
 * @return Definition of command-line options.
private static Options generateOptions()
   final Option verboseOption = Option.builder("v")
      .desc("Print status with verbosity.")
   final Option fileOption = Option.builder("f")
      .desc("File to be processed.")
   final Options options = new Options();
   return options;

The "Builder" pattern implemented for Apache Commons CLI as shown in the above example features the benefits of the builder pattern such as creating an Option in a fully completed state in one statement and use of highly readable builder methods to set that instance's various fields. My older post on Apache Commons CLI demonstrates use of the alternate traditional constructor approach to instantiating Option instances.

With the command-line options defined, it's time to move to the "parsing" stage and the next code listing demonstrates how to parse with Apache Commons CLI by simply invoking the method CommandLinePaser.parse().

Parsing Command-line Options with Commons CLI

 * "Parsing" stage of command-line processing demonstrated with
 * Apache Commons CLI.
 * @param options Options from "definition" stage.
 * @param commandLineArguments Command-line arguments provided to application.
 * @return Instance of CommandLine as parsed from the provided Options and
 *    command line arguments; may be {@code null} if there is an exception
 *    encountered while attempting to parse the command line options.
private static CommandLine generateCommandLine(
   final Options options, final String[] commandLineArguments)
   final CommandLineParser cmdLineParser = new DefaultParser();
   CommandLine commandLine = null;
      commandLine = cmdLineParser.parse(options, commandLineArguments);
   catch (ParseException parseException)
           "ERROR: Unable to parse command-line arguments "
         + Arrays.toString(commandLineArguments) + " due to: "
         + parseException);
   return commandLine;

Note that this code using a newer version of Apache Commons CLI instantiates a DefaultParser for doing the parsing rather than a PosxParser or GnuParser as was done in the older code.

With the command-line objects defined and the command-line parsed, it is time for the interrogation stage. The next code listing demonstrates Apache Commons CLI's support for command-line interrogation.

Interrogating Command-line with Commons CLI

final boolean verbose =
final String fileName =
out.println("The file '" + fileName + "' was provided and verbosity is set to '" + verbose + "'.");

The above code listing demonstrates use of CommandLine.hasOption() to determine if an option's particular flag is present without regard for whether a value is provided for that flag (appropriate for -v/--verbose in our example). Likewise, the code shows that CommandLine.getOptionValue() can be used to obtain the value associated with the the provided command-line flag (appropriate for the -f/--file option in our example).

The next screen snapshot demonstrates the output from the simple example whose code listings were shown above and they demonstrate the support for the verbosity and file path/location command-line options described above.

The second screen snapshot demonstrates Commons CLI' output when the command-line parameters don't include a required command-line argument.

A useful piece of functionality for any framework for building Java command-line parsing is the ability to support usage and help information. This is accomplished via Commons CLI's HelpFormatter. The next code listing demonstrates using HelpFormatter for printing help and usage information and the screen snapshot following the code listing demonstrates the appearance of the help and usage when employed.

Acquiring "usage" and "help" Details with Commons CLI

 * Generate usage information with Apache Commons CLI.
 * @param options Instance of Options to be used to prepare
 *    usage formatter.
 * @return HelpFormatter instance that can be used to print
 *    usage information.
private static void printUsage(final Options options)
   final HelpFormatter formatter = new HelpFormatter();
   final String syntax = "Main";
   final PrintWriter pw  = new PrintWriter(out);
   formatter.printUsage(pw, 80, syntax, options);

 * Generate help information with Apache Commons CLI.
 * @param options Instance of Options to be used to prepare
 *    help formatter.
 * @return HelpFormatter instance that can be used to print
 *    help information.
private static void printHelp(final Options options)
   final HelpFormatter formatter = new HelpFormatter();
   final String syntax = "Main";
   final String usageHeader = "Example of Using Apache Commons CLI";
   final String usageFooter = "See for further details.";
   formatter.printHelp(syntax, usageHeader, options, usageFooter);

This post has demonstrated using Apache Commons CLI to achieve some of the most common functionality related to command-line parsing in Java applications including option "definition", command-line arguments "parsing", "interrogation" of the parsed command-line arguments, and help/usage details related to the command-line arguments. Here are some additional characteristics of Apache Commons CLI to consider when selecting a framework or library to help with command-line parsing in Java.

  • Apache Commons CLI is open source and licensed with the Apache License, Version 2.0.
  • Current version of Apache Commons CLI (1.4) requires J2SE 5 or later.
  • Apache Commons CLI does not require any third-party libraries to be downloaded or referenced separately.
  • The Apache Commons CLI 1.4 main JAR (commons-cli-1.4.jar) is approximately 53 KB in size.
  • Apache Groovy provides out-of-the-box command-line parsing capabilities based on Apache Commons CLI via CliBuilder.
  • The Maven Repository shows almost 1800 dependencies on Apache Commons CLI including Apache Groovy.
  • Apache Commons CLI has been around for a while; its initial 1.0 release was in November 2002.
  • Commons CLI supports both long and short syntax for command-line arguments. Its main page lists support for these types of option formats (and that page includes examples of each style):
    • "POSIX like options" (single hyphen)
    • "GNU like long options" (double hyphen)
    • "Java like properties" (using -D)
    • "Short options with value attached"
    • "Long options with single hyphen"

For me, one of the biggest advantages of Apache Commons CLI when implementing command-line interfaces in simple Java applications is that I'm already familiar with Groovy's built-in use of CliBuilder. Because I use Groovy far more often for simple command-line based scripts and tools than I use Java, this Groovy familiarity with the basic Apache Commons CLI usage is helpful when moving back to Java.

Additional References

Monday, June 5, 2017

jhsdb: A New Tool for JDK 9

I like to use the command-line tools provided with the JDK in the early steps of analyzing performance and other issues with Java-based applications and have blogged on tools such as jcmd, jps, jstat, jinfo, jhat and jmap, jrunscript, jstack, and jdeps. JDK 9 is bringing new command-line tools with multiple tools specifically related to new JDK 9 features such as modularity (jlink and jmod) and enhanced deprecation (jdeprscan). In this post, I focus on a new command-line tool delivered with JDK 9 for dealing with performance and serviceability issues: jhsdb.

The jhsdb tool is described on its Oracle JDK 9 Documentation Early Access page, "You use the jhsdb tool to attach to a Java process or to launch a postmortem debugger to analyze the content of a core-dump from a crashed Java Virtual Machine (JVM)." The tool comes with several "modes" and several of these modes correspond in name and function with individual command-line tools available in previous JDK distributions. The jhsdb tool not only provides a single tool that encompasses functionality of multiple other tools, but it also provides a single, consistent approach to applying these different functions. For example, the jhsdb command-line syntax for getting help for each of the "modes" is identical.

The jhsdb tool can be attached and applied to a running JVM (including one that is hanging) via its process identifier (PID) similar to how several other tools (including jcmd) work. The jhsdb tool can also be used to analyze core information associated with a crashed JVM if the core file and executable are provided. As an example of the consistency jhsdb provides, all of its mode support the "common options" --pid (to specify target JVM's process ID), --exe (to specify target executable), --core (to specify target core dump file), and --help (to display options specific to each mode).

The next series of snapshots demonstrates use of the --help option with the main jhsdb command and with each of several of jhsdb's "modes." One observation that can be made is that the common options --pid, --core, and --exe are offered by all the modes. The obvious inference from this is that the specific functions supported in each mode are those other than those "common" options.

jhsdb "jstack" Mode

The --help for the jhsdb mode jstack has two specific functionality options: --locks and --mixed. These are demonstrated in the following two screen snapshots.

The screen snapshots just shown demonstrate that the jstack mode of the jhsdb tool provides us with deadlock detection details, information on thread locks, and an overview of the native frames and Java frames.

jhsdb "jmap" Mode

The --help for jhsdb mode jmap shows several functions supported by that mode. When jhsdb jmap is executed with only the --pid or only with the --exe/--core combination, the output is similar to that provided by the Linux pmap command.

As one would expect, the jmap mode of the jhsdb provides functions similar to those provided by the separate but similarly named jmap command. These include heap dump (--heap), class histogram (--histo), classloader statistics (--clstats), and finalizer information (--finalizerinfo) and are demonstrated in the following four screen snapshots.

jhsdb "jinfo" Mode

Not surprisingly, the jinfo mode of the jhsdb command provides functionality that overlaps with that provided by the jinfo command. Specifically, the jhsdb's jinfo mode allows one to see the targeted JVM's flags and system properties.

There are three main options used with jhsdb jinfo: --flags to see JVM flags, --sysprops to see the system properties, or no argument to see both the flags and the system properties. The next two screen snapshots demonstrate use of jhsdb jinfo --flags and jhsdb jinfo --sysprops. Running jhsdb jinfo without any arguments shows the system properties first followed by the flags, but is not shown here.

jhsdb "jsnap" Mode

The jhsdb mode jsnap provides access to information previously provided by the internal class which was previously available in lib/sa-jdi.jar and which has been added to jhdsdb for JDK 9. There are two options for output from jhsdb jsnap based on whether no mode-specific argument is provided or if the --all mode-specific argument is provided. The next two screen snapshots demonstrate these two options.

These screenshots demonstrate that jhsdb jsnap with no mode-specific option lists information such as events related to threads and class loading/unloading along with core JVM properties. Adding the --all option lists these same properties, but in addition adds far more properties and, according to the documentation, "Prints all performance counters."

By the way, Marcus Hirt's Using the JVM Performance Counters provides an interesting look at how to apply JMX and custom MBeans to achieve a tool "similar to the PerformanceCounters MBean available in JRockit." I believe that jhsdb jsnap --all brings simple ability to see the same type of information in HotSpot as Hirt talked about being available in JRockit with jrcmd -l.

jhsdb Debug Modes

The three jhsdb modes hsdb (graphical user interface for interactive debugging), clhsdb (command-line interface for interactive debugging), and debugd (remote debug server) are related to debug operations. I may take a closer look at these modes in a future post, but for now I simply show some screen snapshots that demonstrate the graphical interaction using jhsdb hsdb. The GUI was started with jhsdb hsdb --pid <pid> and most of the options displayed here were run by selecting the specific menu option under "Tools".

As can be seen in the article HotSpot's Hidden Treasure, the serviceability debugger GUI has been available before JDK 9 and jhsdb, but this article also shows how much more difficult it was to find and start this tool before JDK 9's introduction of jhsdb.

Relationship of jhsdb to jcmd and to Other Command-line JDK Tools

I summarized the relationship of general-purpose tool jcmd to other JDK-provided command-line tools in the blog post jcmd: One JDK Command-Line Tool to Rule Them All. I adapt that table here to add jhsdb to the mix.

FunctionalityjhsdbjcmdSimilar Tool
Listing Java Processes N/A1 jcmd jps -lm
Heap Dumps jhsdb jmap --binaryheap jcmd <pid> GC.heap_dump jmap -dump <pid>
Heap Usage Histogram jhsdb jmap --histo jcmd <pid> GC.class_histogram jmap -histo <pid>
Thread Dump jhsdb jstack --locks
(subset of locked thread frames)
jcmd <pid> Thread.print jstack <pid>
List System Properties jhsdb jinfo --sysprops jcmd <pid> VM.system_properties jinfo -sysprops <pid>
List VM Flags jhsdb jinfo --flags jcmd <pid> VM.flags jinfo -flags <pid>

1 You use jcmd or jps -lm to identify PID upon which to have jhsdb, jcmd, and many other tools act if working against a running JVM. I used jcmd in this post to identify the PID, but the current jhsdb documentation demonstrates using jps to acquire the JVM PID.

The jhsdb tool is a mostly command-line tool that does also have an optional interactive GUI available that supports reporting of many of the commonly desired attributes of a JVM that is hung or has crashed. It provides a consistent interface across its modes and the command-line interaction allows for interactive help requests such that very little syntax must be known or remembered before applying the tool. If one can remember "jhsdb", one can start using the tool effectively. The jhsdb tool is new to JDK 9, but brings functionality into one tool that was previously available from several different tools.