Tuesday, February 8, 2011

Java and Oracle, One Year Later

It's been just over a year since Oracle closed the deal to purchase Sun. With that in mind, the forthcoming Oracle Technology Network (OTN) TechCast "Java and Oracle, One Year Later," is aptly named. It is scheduled for 10 am (Pacific Time) on Tuesday, February 15, 2011. Ajay Patel, VP of Product Development for Application Grid Products, presents a "special live conversation" regarding "changes that have come to Java and Oracle since the Sun acquisition."

Additional information on this TechCast can be found online, but I also received an e-mail message with additional details on this TechCast. The e-mail message adds more details on what will be discussed:
  • "Highlights, challenges and what we learned over the past year"
  • "The Future of Java and its importance to Oracle and the community"
  • "Oracle’s Application Grid product portfolio today"
The e-mail message states that Justin Kestelyn (Director of OTN) will also be involved and that "attendees" will be able to ask questions. There is encouragement to register even if the currently scheduled time does not work "so we can send you the replay information."

Groovy Scripts Master Their Own Classpath with RootLoader

Groovy's RootLoader is a handy class that can be used to encapsulate a Groovy script's reference to external classpath dependencies within the script itself. The documentation for RootLoader states, "It's possible to add urls to the classpath at runtime through addURL(URL)." The ability to add to a Groovy script's classpath at runtime allows the script developer to write a Groovy script that can take care of its own bootstrapping and removes the need to necessarily start a Groovy script from a shell script that provides the classpath with the -classpath (or -cp) option.

This post is not my first to focus on using Groovy's RootLoader. My previous post Viewing Groovy Application's Classpath demonstrated using RootLoader to detect all of the resources already on Groovy's classpath. In this post, I look at using this same RootLoader to allow a Groovy script to bootstrap itself with appropriate classpath dependencies.

I am going to use a simple Groovy script with a dependency on the Oracle JDBC driver to illustrate. That script is called printEmployees.groovy and is shown next.

printEmployees.groovy
// printEmployees.groovy
import groovy.sql.Sql
sql = Sql.newInstance("jdbc:oracle:thin:@localhost:1521:orcl", "hr", "hr",
                      "oracle.jdbc.pool.OracleDataSource")
sql.eachRow("SELECT employee_id, last_name, first_name FROM employees")
{
   println "The employee's name is ${it.first_name} ${it.last_name}."
}

Supposing that the appropriate Oracle JDBC driver is located at C:\app\Dustin\product\11.1.0\db_1\jdbc\lib\ojdbc6.jar, this Groovy script could be run by specifying the Oracle JDBC driver on the command line like this:

groovy -cp C:\app\Dustin\product\11.1.0\db_1\jdbc\lib\ojdbc6.jar printEmployees

The next screen snapshot shows the beginning of the script's output when this is done against the 'hr' sample schema supplied with the Oracle database.


The above output is much better than that which is provided when no classpath is provided. The result of not specifying where the Groovy script can find the Oracle JDBC driver is shown in the next screen snapshot.


Using Groovy's -classpath (or -cp) option is not the only way to tell the Groovy script about a resource it requires. In Dustin's Blog (I like the name!), the Dustin Whitney post Groovy Classpath concisely describes another way to place a resource on a Groovy script's classpath. He simply states (I have added the emphasis): "You can place jars in your ${user.home}/.groovy/lib directory to have them automatically loaded into your classpath."

The next screen snapshot demonstrates that I have now tried this by placing the appropriate Oracle JAR file in the appropriate directory (C:\Users\Dustin\.groovy\lib in my case).


The next screen snapshot shows that the script can now be run without explicitly specifying the classpath on the command line. I added a line of Groovy to the original script to print out the location of "user.home": println "${System.getProperty('user.home')}". It prints out C:\Users\Dustin.


There is another directory common to all Groovy scripts in which dependent JARs can also be placed. This is the %GROOVY_HOME%\lib or $GROOVY_HOME/lib directory. Although I don't show it here, placing the Oracle JAR file in the Groovy distribution's lib directory makes it available to all Groovy scripts run from that distribution regardless of the user running the Groovy script.

There is a single file provided with the Groovy installation for controlling use of the directories for files placed automatically on the classpath. The directory for this configuration is %GROOVY_HOME%\conf or $GROOVY_HOME/conf and the file is named groovy-starter.conf. The next screen snapshot shows this on my current environment.


Within this file, there are two lines that set the directories where Groovy automatically looks for classpath entries. There are comments indicating which line "load[s] required libraries" (load !{groovy.home}/lib/*.jar) and which line "load[s] user specific libraries" (load !{user.home}/.groovy/lib/*.jar). A typical configuration is to have the user specific line commented out, but the comment character (#) can be removed so that the user directory's contents will be automatically on the classpath. As a complete example, here is the current groovy-starter.conf file in my environment:

##############################################################################
##                                                                          ##
##  Groovy Classloading Configuration                                       ##
##                                                                          ##
##############################################################################

##
## $Revision: 9225 $ $Date: 2007-11-15 21:17:45 +0100 (Do, 15. Nov 2007) $
##
## Note: do not add classes from java.lang here. No rt.jar and on some
##       platforms no tools.jar
##
## See http://groovy.codehaus.org/api/org/codehaus/groovy/tools/LoaderConfiguration.html
## for the file format

    # load required libraries
    load !{groovy.home}/lib/*.jar

    # load user specific libraries
    load !{user.home}/.groovy/lib/*.jar
    
    # tools.jar for ant tasks
    load ${tools.jar}

Note also that this configuration file refers to the URL http://groovy.codehaus.org/api/org/codehaus/groovy/tools/LoaderConfiguration.html, where the syntax of this file and its makeup are more comprehensively defined. The RootLoader receives prominent mention in this document.

The script works without explicitly specifying the classpath on the command line when the dependent JAR is placed either in the user's specific .groovy/lib directory or in the general Groovy distribution's $GROOVY_HOME/lib directory. The difference between the two is that one minimizes the dependent JAR's availability to classpaths of Groovy scripts run by the specific user while the other makes the dependent JAR available to classpaths of all Groovy scripts run from that distribution. In either case, the dependent JARs are necessarily present on either all of the user's Groovy scripts or all of the Groovy scripts run from a particular Groovy installation and are not limited to a single script.

The downside (or upside depending on perspective) is that this affects all Groovy scripts. It's akin to setting the environment variable CLASSPATH or to placing the JAR in a directory used for all Java applications rather than explicitly setting the classpath when running Java applications. In general, the Java equivalent is considered bad form and the Groovy version suffers the same potential drawbacks. Incidentally, setting the CLASSPATH environment variable to include whatever is required by the Groovy script works for Groovy as well as for Java.

Specifying the classpath used by a Groovy script using the groovy command's -classpath (or -cp) requires the person running the script to either type that in or requires a shell or other "outer" script to invoke the Groovy script. Even using the approach of placing a dependent JAR file in the .groovy/lib subdirectory of the user directory requires the person running the script to have placed the JAR there. The best solution is often the one that allows the script to contain its own classpath references. This is discussed next.

When it is less than desirable to have a shell script kick off a Groovy script and it is also undesirable to pollute other Groovy scripts' classpaths by placing dependent JARs in the CLASSPATH environment variable or in the %JAVA_HOME%\lib directory or in the user's .groovy/lib directory, the most desirable solution may be to dynamically add a dependency to the classpath using RootLoader. The next Groovy code listing shows the previously shown script amended to use the RootLoader to dynamically load the Oracle JDBC driver JAR and append it to the script's classpath.

// printEmployees.groovy
this.class.classLoader.rootLoader.addURL(
   new URL("file:///C:/app/Dustin/product/11.1.0/db_1/jdbc/lib/ojdbc6.jar"))
import groovy.sql.Sql
sql = Sql.newInstance("jdbc:oracle:thin:@localhost:1521:orcl", "hr", "hr",
                      "oracle.jdbc.pool.OracleDataSource")
sql.eachRow("SELECT employee_id, last_name, first_name FROM employees")
{
   println "The employee's name is ${it.first_name} ${it.last_name}."
}

The above script does not require the Oracle JDBC driver JAR to be explicitly specified on the command line and does not require the JAR to be in any Groovy-specific directory. This frees the script developer and the script's users from these command-line and directory dependencies and removes the risk of polluting other Groovy scripts' classpaths.


Conclusion

I often use shell scripts to start my Groovy scripts. Not only can the shell scripts properly specify the -classpath or -cp option when running the Groovy script, but they can also set JAVA_OPTS environment variable appropriately (such as for setting JVM heap sizing). However, there are times when it seems unnecessary or less than desirable to need two scripts (a shell and the Groovy script) to accomplish a single script's job. In such cases, the ability to dynamically append resources to the script's classpath via RootLoader is welcome.

Monday, February 7, 2011

Finding Multiple Class Definitions with Jarminator

One of the advantages of being a Java developer is the wide assortment of tools, libraries, and frameworks available to make Java development easier. Several months ago, a colleague introduced my to Jarminator, a little open source produce that is surprisingly (pleasantly) helpful in Java development. It has also been referenced on DZone. In this post, I look at Jarminator and how it can be helpful to the Java developer.

The main project page for Jarminator v0.15 is filled with screen snapshots demonstrating Jarminator in action. A Java developer who is aware of this product's existence and accesses this main page should have no trouble using Jarminator. The usage can also be found by running one of the batch files (jarminator_debug) with the -? option as shown in the next screen snapshot.


The single ZIP file can be downloaded at http://sourceforge.net/projects/jarminator/files/ and version 0.15 is a tad over 50kb in size. Once downloaded, its "installation" is simple (unzip it into whatever location you like). From that point, it can be run in Windows with the provided batch files. One simply needs to change directory to the directory into which the contents of the jarminator.zip were extracted and type "jarminator" on the command line.

Most Java developers can recount numerous stories of situations where their applications ran into exceptions and other misbehavior such as ClassNotFoundException and NoClassDefFoundError. These are sometimes trivial to fix. For example, a ClassNotFoundException is easy to address if it is determined that the particular class is not on the classpath. Errors and exceptions and other misbehavior can also occur when more than one definition of the same class are on the classpath. I remember many traumatic episodes dealing with different versions of Xerces supplied by different frameworks several years ago. All of these issues are more easily dealt with when Jarminator is applied.

Jarminator aids the Java developer by providing a simple, graphically-based view of classes associated with specified JARs. It makes the developer's job of determining which classes are on the classpath and which are not on the classpath much easier. It makes duplicate classes obvious by listing the duplicates in a gray font.

To demonstrate this, I have loaded Groovy JAR files from both the Groovy distribution installed on my machine and from a Spring Framework distribution installed on my machine into Jarminator as shown in the next screen snapshot.


One way to specify the JARs for Jarminator load is to click on the "Browse" button and to select the appropriate JAR. However, I find it often most useful to simply copy-and-paste my specified classpath (whatever follows -cp or -classpath) into that "Source" field. Once the source JARs are specified (either by using the "Browse" button to select them or by pasting or typing them in), clicking on the "Load" button instructors Jarminator to load metadata about the JARs' contents.

Once the JARs' contents have been loaded into Jarminator, the JARs that have been loaded successfully are listed in the "Jars" tab as shown in the above screen snapshot. This is useful to ensure that the classpath typed in was valid and was for a valid JAR that Jarminator understands. The "Classes" tab is the one that is most helpful. This tab lists all of the classes contained in the JARs specified in the "Source" field. An example of this is shown in the next screen snapshot.


The preceding screen snapshot shows that the various packages of classes found in the loaded JARs are displayed in the "Classes" tab. Clicking on these packages expands them to show the classes that reside within the packages. This is shown in the next screen snapshot.


Thanks to Jarminator, it is easy to identify that many of these classes are duplicate (the ones in gray indicate duplicates). Although it's nice to know there are duplicates, it is far more useful to know where the duplicate definitions are coming from. This can be determined by clicking on the duplicate classes. Once clicked on, Jarminator indicates where each class definition source is. This is shown in the next two screen snapshots which show the duplicate Closure class being defined in the Groovy distribution and in the Spring Framework distribution. With this knowledge, the developer can then remove the inappropriate source definition from the classpath.



The bottom left corner of the Jarminator GUI indicates the source of the class definition that is highlighted. In the case shown in the screen snapshots above, the Spring Framework supplied version is considered the duplicate, though, of course, that decision is really up to the developer based on the context of the application.

I previously mentioned that the "Jars" tab is useful for ensuring that JARs have been appropriately loaded into Jarminator. It also can be used to delve down into the classes provided on a per JAR basis. This is shown in the next screen snapshot.


The Jarminator GUI also features a "Filter" button. As might be expected given its name, this field allows a filter to be specified to only show entries matching the filter. For example, if the developer only cares about .class files and not about other non-class items such as images, the .class filter can be specified and is applied when the "Apply" button is clicked. To see this in action, the next two screen snapshots show Jarminator's display for the case of no filter applied and then having the ".class" filter applied. When no filter is applied, the Grape package contents are shown with the included XML file. When the ".class" filter is applied, the XML file is no longer shown.



Besides demonstrating the utility of Jarminator's filtering capability, the last two screen snapshots also demonstrate the different icons Jarminator uses for different file types within JARs. Java classes (.class files) have a C within a green circle and most of the icons are familiar to Java developers who have used the Eclipse IDE. A summary of the Jarminator icons is shown next.



Jarminator v0.15 also supports a console mode. This is executed with the command jarminator_debug -c followed by the path to be loaded and searched. I have found that if there is more than one JAR file specified (and therefore separated by semicolon) that it is necessary to put the multiple paths within quotes. An example of this is shown in the next two screen snapshots with the first image showing the command used from the console with the beginning of the output and the second image showing the end of the output with a summary line.



Conclusion

As I stated earlier in this post, the main Jarminator page is almost exclusively devoted to coverage of the features of Jarminator. Jarminator is an easy tool to use, but can be a very powerful tool when the Java developer needs to determine classpath conflicts involving insufficient or too many class definitions.

Saturday, February 5, 2011

Determining Level of Java Debug in Class File via javap

The Java Class File Disassembler (javap) is a useful tool for the Java developer that I have referenced in previous blog posts covering a variety of contexts such as detecting the innards of a Groovy class, comparing to the output of javac -Xprint, investigating synthetic methods, and looking at code offsets leading to "code too large" problems. In this post, I look at how javap might be used to determine what level of debugging a particular Java class was compiled with.

The Sun/Oracle javac implementation provides the -g option to control the level of debug support built into the compiled .class file. When -g is not specified at all during the compilation process, the javac default is that "only line number and source file information is generated." Specifying -g by itself enables inclusion of "local variable debugging information" in addition to source code and line numbers. The -g option can also be used to include only one of the three pieces of debugging information. The option -g:lines includes line numbers, -g:source includes source information, and -g:vars includes local variable debugging information. More than one keyword (lines, source, and vars) can be specified with a single -g option as long as the multiple keywords are separated by a comma.

An obvious way to determine which of these types of debug has been included in a compiled class is to actually attempt to access debug information via a debugger or other tool that requires debug information to be available. However, in cases where we'd like to know before firing up such tools, javap provides an alternative approach for determining what types of debug information are included in a given .class file.

For the demonstrations of using javap to determine the types of debug supported in a given class, I'm using a very simple Java class called SimpleClass.

SimpleClass.java
import static java.lang.System.out;

import java.util.Date;

/**
 * Simple class whose primary reason for existence is to demonstrate how javap
 * can be used to determine if a class was compiled with debug options enabled.
 */
public class SimpleClass
{
   private Date startDate;

   public SimpleClass() {}

   public void printTodaysDate()
   {
      out.println("Today is " + new Date());
   }

   public static void main(final String[] arguments)
   {
      final SimpleClass me = new SimpleClass();
      me.printTodaysDate();
   }
}

With a simple class defined in Java source code, I can now use javac with various -g options to compile .class files with correspondingly different debug support. I show the javap output for each of these and use the differences to explain how to tell what levels of debug javap is telling us the compiled class supports.


Compiling with javac Without -g Option

As described above, using javac without explicitly specifying the -g option leads to the default debug information of line numbers and source information being included in the compiled class. For this example, the command javac SimpleClass.java generates a SimpleClass.class file that javap disassembles as shown next.

Compiled from "SimpleClass.java"
public class SimpleClass extends java.lang.Object{
public SimpleClass();
  LineNumberTable: 
   line 13: 0



public void printTodaysDate();
  LineNumberTable: 
   line 17: 0
   line 18: 31



public static void main(java.lang.String[]);
  LineNumberTable: 
   line 22: 0
   line 23: 8
   line 24: 12



}

The first line in the javap output above (Compiled from "SimpleClass.java") is evidence that the "source code information" was included in the debug. The several lines with line numbers (such as "line 22") are similar evidence of line numbers being included in the compiled class. As expected, javap's output tells us that line numbers and source code information were compiled into the class.


Compiling with javac and -g:none Option

When the command javac -g:none SimpleClass.java is executed, there is no debug information included in the compiled class. The javap output, which is shown next, is therefore not surprising:

public class SimpleClass extends java.lang.Object{
public SimpleClass();



public void printTodaysDate();



public static void main(java.lang.String[]);



}

The javap output proves that a Java class compiled with javac -g:none has neither the source information (no "Compiled from") nor the line numbers that are provided by default.


Compiling with javac and -g:source Option

The javap output for a class compiled with javac -g:source has the beginning line that states where the .class was compiled from:

Compiled from "SimpleClass.java"
public class SimpleClass extends java.lang.Object{
public SimpleClass();



public void printTodaysDate();



public static void main(java.lang.String[]);



}


Compiling with javac and -g:lines Option

The javap output for a class compiled with javac -g:lines is shown next. It has the line numbers as did the default case, but lacks the "Compiled from" source information of the default case.

public class SimpleClass extends java.lang.Object{
public SimpleClass();
  LineNumberTable: 
   line 13: 0



public void printTodaysDate();
  LineNumberTable: 
   line 17: 0
   line 18: 31



public static void main(java.lang.String[]);
  LineNumberTable: 
   line 22: 0
   line 23: 8
   line 24: 12



}


Compiling with javac and -g:vars Option

The one type of debug output we have not yet seen is that for variable debugging information provided when javac -g:vars is used to compile the class. This javap output is shown next. The LocalVariableTable output here is the sign that the class in question was compiled with -g:vars specified. The variables' names are also in the output.

public class SimpleClass extends java.lang.Object{
public SimpleClass();

  LocalVariableTable: 
   Start  Length  Slot  Name   Signature
   0      5      0    this       LSimpleClass;


public void printTodaysDate();

  LocalVariableTable: 
   Start  Length  Slot  Name   Signature
   0      32      0    this       LSimpleClass;


public static void main(java.lang.String[]);

  LocalVariableTable: 
   Start  Length  Slot  Name   Signature
   0      13      0    arguments       [Ljava/lang/String;
   8      5      1    me       LSimpleClass;


}


Conclusion

It is easy to use javap to determine what types of debug information were compiled into a generated .class file. When the first line of the javap output states "Compiled from," we know that source information was included. When we see various lines of the output with the keyword "line," we know that line number information was included. Finally, when we see "LocalVariableTable" in the javap output, we know that the class was compiled with variable debugging information included.


Additional Reference

There was a JDC Tech Tip called Getting Started with javap published on 29 August 2000 that apparently is no longer available in its original form. Fortunately, it is available in an Apache mail forum thread.

Generating XML Schema with schemagen and Groovy

I have previously blogged on several utilitarian tools that are provided with the Java SE 6 HotSpot SDK such as jstack, javap, and so forth. I focus on another tool in the same $JAVA_HOME/bin (or %JAVA_HOME%\bin directory: schemagen. Although schemagen is typically used in conjunction with web services and/or JAXB, it can be useful in other contexts as well. Specifically, it can be used as an easy way to create a starting point XML Schema Definition (XSD) for someone who is more comfortable with Java than with XML Schema.

We'll begin with a simple Java class called Person to demonstrate the utility of schemagen. This is shown in the next code listing.

package dustin.examples;

public class Person
{
   private String lastName;

   private String firstName;

   private char middleInitial;

   private String identifier;

   /**
    * No-arguments constructor required for 'schemagen' to create XSD from
    * this Java class.  Without this "no-arg default constructor," this error
    * message will be displayed when 'schemagen' is attempted against it:
    *
    *      error: dustin.examples.Person does not have a no-arg default
    *      constructor.
    */
   public Person() {}

   public Person(final String newLastName, final String newFirstName)
   {
      this.lastName = newLastName;
      this.firstName = newFirstName;
   }

   public Person(
      final String newLastName,
      final String newFirstName,
      final char newMiddleInitial)
   {
      this.lastName = newLastName;
      this.firstName = newFirstName;
      this.middleInitial = newMiddleInitial;
   }

   public String getLastName()
   {
      return this.lastName;
   }

   public void setLastName(final String newLastName)
   {
      this.lastName = newLastName;
   }

   public String getFirstName()
   {
      return this.firstName;
   }

   public void setFirstName(final String newFirstName)
   {
      this.firstName = newFirstName;
   }

   public char getMiddleInitial()
   {
      return this.middleInitial;
   }
}

The class above is very simple, but is adequate for the first example of employing schemagen. As the comment on the no-arguments constructor in the above code states, a constructor without arguments (sometimes called a "default constructor") must be available in the class. Because other constructors are in this class, it is required that a no-args constructor be explicitly specified. I also intentionally provided get/set (accesor/mutator) methods for some of the fields, only an accessor for one of the fields, and neither for a field to demonstrate that schemagen requires get/set methods to be specified if the schema it generates includes a reference to those attributes.

The next screen snapshot demonstrates the most simple use of schemagen in which the generated XML schema file (.xsd) is generated with the default name of schema1.xsd (there is no current way to control this directly with schemagen) and is placed in the same directory from which the schemagen command is run (output location can be dictated with the -d option).


The generated XSD is shown next.

schema1.xsd
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<xs:schema version="1.0" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:complexType name="person">
    <xs:sequence>
      <xs:element name="firstName" type="xs:string" minOccurs="0"/>
      <xs:element name="lastName" type="xs:string" minOccurs="0"/>
    </xs:sequence>
  </xs:complexType>
</xs:schema>

This is pretty convenient, but is even easier with Groovy. Suppose that one wanted to generate an XSD using schemagen and did not care about or need the original Java class. The following very simple Groovy class could be used. Very little effort is required to write this, but it's compiled .class file can be used with schemagen.

package dustin.examples;

public class Person2
{
   String lastName;

   String firstName;

   char middleInitial;

   String identifier;
}

When the above Groovy class is compiled with groovyc, its resulting Person2.class file can be viewed through another useful tool (javap) located in the same directory as schemagen. This is shown in the next screen snapshot. The most important observation is that get/set methods have been automatically generated by Groovy.


When the groovyc-generated .class file is run through schemagen, the XSD is generated and is shown next.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<xs:schema version="1.0" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:complexType name="person2">
    <xs:sequence>
      <xs:element name="firstName" type="xs:string" minOccurs="0"/>
      <xs:element name="identifier" type="xs:string" minOccurs="0"/>
      <xs:element name="lastName" type="xs:string" minOccurs="0"/>
      <xs:element name="middleInitial" type="xs:unsignedShort"/>
    </xs:sequence>
  </xs:complexType>
</xs:schema>

Because I did not explicitly state that Groovy's automatic get/set methods should not be applied, all attributes are represented in the XML. Very little Groovy, but XSD nonetheless.

It is interesting to see what happens when the attributes of the Groovy class are untyped. The next Groovy class listing does not explicitly type the class attributes.

package dustin.examples;

public class Person2
{
   def lastName;

   def firstName;

   def middleInitial;

   def identifier;
}

When schemagen is run against the above class with untyped attributes, the output XSD looks like this:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<xs:schema version="1.0" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:complexType name="person2">
    <xs:sequence>
      <xs:element name="firstName" type="xs:anyType" minOccurs="0"/>
      <xs:element name="identifier" type="xs:anyType" minOccurs="0"/>
      <xs:element name="lastName" type="xs:anyType" minOccurs="0"/>
      <xs:element name="middleInitial" type="xs:anyType" minOccurs="0"/>
    </xs:sequence>
  </xs:complexType>
</xs:schema>

Not surprisingly, the Groovy class with the untyped attributes leads to an XSD with elements of anyType. It is remarkably easy to generate Schema with schemagen from a Groovy class, but what if I don't want an attribute of the class to be part of the generated schema? Explicitly specifying an attribute as private communicates to Groovy to not automatically generate get/set methods and hence schemagen will not generate XSD elements for those attributes. The next Groovy class shows two attributes explicitly defined as private and the resultant XSD from running schemagen against the compiled Groovy class is then shown.

package dustin.examples;

public class Person2
{
   String lastName;

   String firstName;

   /** private modifier prevents auto Groovy set/get methods */
   private String middleInitial;

   /** private modifier prevents auto Groovy set/get methods */
   private String identifier;
}

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<xs:schema version="1.0" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:complexType name="person2">
    <xs:sequence>
      <xs:element name="firstName" type="xs:string" minOccurs="0"/>
      <xs:element name="lastName" type="xs:string" minOccurs="0"/>
    </xs:sequence>
  </xs:complexType>
</xs:schema>

Groovy makes it really easy to generate an XSD. The Groovy code required to do so is barely more than a list of attributes and their data types.


Conclusion

The schemagen tool is a highly useful tool most commonly used in conjunction with web services and with JAXB, but I have found several instances where I have needed to create a "quick and dirty" XSD file for a variety of purposes. Taking advantage of Groovy's automatically generated set/get methods and other Groovy conciseness makes it really easy to generate a simple XSD.

Wednesday, February 2, 2011

Java News: Java Hanging and Java Desktop Popularity

There have been two significant stories in the world of Java this week and I briefly summarize them in this post.

Hanging javac and java

Rick Regan posted Java Hangs When Converting 2.2250738585072012e-308 to look at an issue which causes the javac compiler and java launcher both to hang. His post is based on Konstantin Preißer's feedback comment to Regan's previous post on a similar issue in PHP. The Regan post lists simple examples of Java code that respectively hang the compiler and the launcher. I include slightly adapted (to be more Javaese) versions of these next.

RuntimeHang.java (Hangs java Launcher)
package dustin.examples;

import static java.lang.System.out;

/**
 * Example that breaks Java runtime adapted from 
 */
public class RuntimeHang
{
   /**
    * Main function demonstrating Java runtime hang.
    *
    * @param args Command-line arguments: none expected.
    */
   public static void main(String[] args)
   {
      out.println("Test of Hanging Double Issue...");
      final double d = Double.parseDouble("2.2250738585072012e-308");
      out.println("Value: " + d);
   }
}

CompileHang.java (Hangs javac Compiler)
package dustin.examples;

import static java.lang.System.out;

/**
 * This class currently hands when javac is used to attempt to compile it.
 */
public class CompileHang
{
   /**
    * Main executable function.
    *
    * @param args Command-line arguments: none expected.
    */
   public static void main(final String[] args)
   {
      final double d = 2.2250738585072012e-308;
      out.println("Value: " + d);
   }
}

This story has received significant Java blogosphere attention. It has been referenced at DZone and is currently the most popular link (see next screen snapshot).


This post is also featured at reddit/Programming and there are already over 350 comments on the story there. Some of the comments point out that various tools can be used to find out a little more about what's happening with the hung application. For example, use of jstack and forcing the printing of the stack trace are mentioned.

The Groovy Programming post Java hangs when converting 2.2250738585072012e-308 — prevent DOS attacks with Groovy talks about preventing a potential denial of service (DoS) attack based on this issue using Groovy. In other alternative JVM language related posts, Charles Nutter demonstrates Working Around the Java Double.parseDouble Bug for JRuby.


Popularity of the Java Desktop

In the post Poll Result: Java on the Desktop Is in Desperate Need of Attention, Java.net editor Kevin Farnham analyzes results of the recent Java.net poll question, "Which area of Java/JVM technology most desperately needs serious attention in 2011?" The high number of responses (600+) and the high number of comments (10+) are both significantly higher than is typical for these poll questions, suggesting serious interest in this particular question.

I was not surprised at all at the feature that Java developers want some love to be shown to: "Java on the desktop." Some were surprised and that is understandable because many of the online resources (blogs, articles, and so forth) don't truly reflect the "typical Java developer." Although there are obviously exceptions, many blogs and articles focus on "newer" things and less on "older things." There are many reasons for this, including the perception that older things are already well covered or are not interesting to anyone. The fact that Sun/Oracle has shown little interest in the desktop outside of JavaFX also probably reduces the number of articles and blog posts on the subject.

One of the reasons I'm not surprised that nearly 1/3 of the respondents said "Java on the desktop" needs some love is that I continually hear Java developers who don't like Flex say they don't want to use Flex because "it's not Java." That's true, but I point out that JavaFX is hardly Java either (though this could change dramatically if Oracle delivers on what was offered at JavaOne 2010). The point is that many developers want the advantages of modern user interface technologies in a truly Java-based and standardized toolkit. There are some nice third-party Java-based toolkits out there, but they lack the commonality and widespread usage that comes with standardization.

On a related but different note, I see this as evidence of something I have long maintained: blog posts and software development social sites (such as DZone and reddit/programming) attract a subset of the software developers that is not necessarily representative of the greater software development community. Developers who choose to write and read posts and to post comments and get involved in discussions on these sites are not representative of all developers. Therefore, it is dangerous to assume that the number of recent posts on a particular subject reflect the true popularity or usage of that subject. Many posts and discussions are fueled by rabid software development enthusiasts and/or consultants needing to provide some new insights. Not all developers fall into this category. I believe more software developers should write blogs so that we can have a better overall picture of the industry via better sampling of the community.


Conclusion

The news about Java's compiler and application launcher hanging when trying to deal with a certain double representation is big news. However, my best guess is that even though many people reading this blog post will have already seen that news, there will be many times that number of Java developers who have not seen this article and may never even be aware of this issue because it will be fixed before they ever hear or read about it. Not all Java developers scour the headlines each day or even each week to see what's happening. The Java.net survey asking Java developers what they want to see receive the most attention seems to be overwhelming evidence that we as a community still wish to see Java on the desktop get some attention and TLC.

Happy Groundhog Day!

Monday, January 31, 2011

Groovy, JMX, and the Attach API

One of the exciting new features of the (then Sun) HotSpot Java SE 6 release was support for the Attach API. The Attach API is "a Sun Microsystems extension that provides a mechanism to attach to a Java virtual machine." Tools such as JConsole and VisualVM "use the Attach API to attach to a target virtual machine and load its tool agent into that virtual machine." Custom Java and Groovy clients can likewise use the Attach API to monitor and manage JVMs.

There are several online resources that demonstrate Java client code that uses the Attach API. These include Daniel Fuchs's code listing for a JVMRuntimeClient, the "Setting up Monitoring and Management Programmatically" section of "Monitoring and Management Using JMX Technology," the Core Java Technology Tech Tip called "The Attach API," and the Javadoc API documentation for the class com.sun.tools.attach.VirtualMachine. These examples generally demonstrate using VirtualMachine.attach(String) to attach to the virtual machine based on its process ID in String form. This is generally followed by loading the appropriate agent with VirtualMachine.loadAgent(String), where the String parameter represents the path to the JAR file containing the agent. The VirtualMachine.detach() method can be called to detach from the previously attached JVM.

All of the previously mentioned examples demonstrate use of the Attach API from Java clients. In this post, I demonstrate use of the Attach API via Groovy. The three code listings that follow present three pieces of Groovy code but all work together as a single script. The main script is embodied in the Groovy file getJvmThreadInfo.groovy and is a simple script file that calls the other two Groovy script files (attachToVirtualMachine.groovy and displayMxBeanDerivedInfo.groovy) to attach to the virtual machine and to display details regarding that virtual machine via its MXBeans.

getJvmDetails.groovy
#!/usr/bin/env groovy
// getJvmDetails.groovy
//
// Main script for extracting JVM details via Attach API and JMX.
// Accepts single parameter which is the process ID (pid) of the Java application
// whose JVM is to be connected to.
//
import static attachToVirtualMachine.retrieveConnector
import static displayMxBeanDerivedInfo.*

def serverConnection = attachToVirtualMachine.retrieveServerConnection(args[0])

displayMxBeanDerivedInfo.displayThreadInfo(serverConnection)
displayMxBeanDerivedInfo.displayOperatingSystemInfo(serverConnection)
displayMxBeanDerivedInfo.displayRuntimeInfo(serverConnection)
displayMxBeanDerivedInfo.displayMemoryInfo(serverConnection)

attachToVirtualMachine.groovy
// attachToVirtualMachine.groovy
//
// Provide an MBeanServerConnection acquired via the Attach API.

import javax.management.MBeanServerConnection
import javax.management.remote.JMXConnector
import javax.management.remote.JMXConnectorFactory
import javax.management.remote.JMXServiceURL

import com.sun.tools.attach.VirtualMachine


/**
 * Provide an MBeanServerConnection based on the provided process ID (pid).
 *
 * @param pid Process ID of Java process for which MBeanServerConnection is
 *    desired.
 * @return MBeanServerConnection connecting to Java process identified by pid.
 */
def static MBeanServerConnection retrieveServerConnection(String pid)
{
   println "Get JMX Connector for pid ${pid}!"
   def connectorAddressStr = "com.sun.management.jmxremote.localConnectorAddress"
   def jmxUrl = retrieveUrlForPid(pid, connectorAddressStr)
   def jmxConnector = JMXConnectorFactory.connect(jmxUrl)
   return jmxConnector.getMBeanServerConnection()
}


/**
 * Provide JMX URL for attaching to the provided process ID (pid).
 *
 * @param @pid Process ID for which JMX URL is needed to connect.
 * @param @connectorAddressStr String for connecting.
 * @return JMX URL to communicating with Java process identified by pid.
 */
def static JMXServiceURL retrieveUrlForPid(String pid, String connectorAddressStr)
{
   // Attach to the target application's virtual machine
   def vm = VirtualMachine.attach(pid)

   // Obtain Connector Address
   def connectorAddress =
      vm.getAgentProperties().getProperty(connectorAddressStr)

   // Load Agent if no connector address is available
   if (connectorAddress == null)
   {
      def agent = vm.getSystemProperties().getProperty("java.home") +
          File.separator + "lib" + File.separator + "management-agent.jar"
      vm.loadAgent(agent)

      // agent is started, get the connector address
      connectorAddress =
         vm.getAgentProperties().getProperty(connectorAddressStr)
   }

   return new JMXServiceURL(connectorAddress);
}

displayMxBeanDerivedInfo.groovy
// displayMxBeanDerivedInfo.groovy
//
// Display details regarding attached virtual machine and associated MXBeans.

import java.lang.management.ManagementFactory
import java.lang.management.MemoryMXBean
import java.lang.management.OperatingSystemMXBean
import java.lang.management.RuntimeMXBean
import java.lang.management.ThreadMXBean
import javax.management.MBeanServerConnection

/**
 * Display thread information based on ThreadMXBean associated with the provided
 * MBeanServerConnection.
 *
 * @param server MBeanServerConnection to use for obtaining thread information
 *    via the ThreadMXBean.
 */
def static void displayThreadInfo(MBeanServerConnection server)
{
   def remoteThreadBean = ManagementFactory.newPlatformMXBeanProxy(
                             server,
                             ManagementFactory.THREAD_MXBEAN_NAME,
                             ThreadMXBean.class);

   println "Deadlocked Threads: ${remoteThreadBean.findDeadlockedThreads()}"
   println "Monitor Deadlocked Threads: ${remoteThreadBean.findMonitorDeadlockedThreads()}"
   println "Thread IDs: ${Arrays.toString(remoteThreadBean.getAllThreadIds())}"
   def threads = remoteThreadBean.dumpAllThreads(true, true);
   threads.each
   {
      println "\t${it.getThreadName()} (${it.getThreadId()}): ${it.getThreadState()}"
   }
}


/**
 * Display operating system information based on OperatingSystemMXBean associated
 * with the provided MBeanServerConnection.
 *
 * @param server MBeanServerConnection to use for obtaining operating system
 *    information via the OperatingSystemMXBean.
 */
def static void displayOperatingSystemInfo(MBeanServerConnection server)
{
   def osMxBean = ManagementFactory.newPlatformMXBeanProxy(
                     server,
                     ManagementFactory.OPERATING_SYSTEM_MXBEAN_NAME,
                     OperatingSystemMXBean.class)
   println "Architecture: ${osMxBean.getArch()}"
   println "Number of Processors: ${osMxBean.getAvailableProcessors()}"
   println "Name: ${osMxBean.getName()}"
   println "Version: ${osMxBean.getVersion()}"
   println "System Load Average: ${osMxBean.getSystemLoadAverage()}"
}


/**
 * Display operating system information based on RuntimeMXBean associated with
 * the provided MBeanServerConnection.
 *
 * @param server MBeanServerConnection to use for obtaining runtime information
 *    via the RuntimeMXBean.
 */
def static void displayRuntimeInfo(MBeanServerConnection server)
{
   def remoteRuntime = ManagementFactory.newPlatformMXBeanProxy(
                          server,
                          ManagementFactory.RUNTIME_MXBEAN_NAME,
                          RuntimeMXBean.class);

   println "Target Virtual Machine: ${remoteRuntime.getName()}"
   println "Uptime: ${remoteRuntime.getUptime()}"
   println "Classpath: ${remoteRuntime.getClassPath()}"
   println "Arguments: ${remoteRuntime.getInputArguments()}"
}


/**
 * Display operating system information based on MemoryMXBean associated with
 * the provided MBeanServerConnection.
 *
 * @param server MBeanServerConnection to use for obtaining memory information
 *    via the MemoryMXBean.
 */
def static void displayMemoryInfo(MBeanServerConnection server)
{
   def memoryMxBean = ManagementFactory.newPlatformMXBeanProxy(
                         server,
                         ManagementFactory.MEMORY_MXBEAN_NAME,
                         MemoryMXBean.class);
   println "HEAP Memory: ${memoryMxBean.getHeapMemoryUsage()}"
   println "Non-HEAP Memory: ${memoryMxBean.getNonHeapMemoryUsage()}"
}

The three Groovy code listings above together form a script that will use the Attach API to contact to an executing JVM without host or port specified and solely based on the provided process ID. The examples demonstrate use of several of the available MXBeans built into the virtual machine. Because it's Groovy, the code is somewhat more concise than its Java equivalent, especially because no checked exceptions must be explicitly handled and there is no need for explicit classes.

Much more could be done with the information provided via the Attach API and the MXBeans. For example, the Groovy script could be adjusted to persist some of the gathered details to build reports, Java mail could be used to alert individuals when memory constraints or other issues requiring notice occurred, and nearly anything else that can be done in Java could be added to these client scripts to make it easier to monitor and manage Java applications.

Running with the Attach API

The main implementation class of the Attach API, VirtualMachine, is located in the ${JAVA_HOME}\lib\tools.jar or %JAVA_HOME\lib\tools.jar JAR file included with the HotSpot SDK distribution. This file typically needs to be explicitly placed on the classpath of the Java client that uses the Attach API unless it is otherwise placed in a directory that is part of that inherent classpath. This is typically not required when using Groovy because it's normally already in Groovy's classpath. I briefly demonstrated this in the post Viewing Groovy Application's Classpath.

Conclusion

The Attach API makes it easier for the Java (or Groovy) developer to write clients that can communicate with, manage, and monitor Java processes. The Attach API provides the same benefits to the developer of custom JMX clients that JConsole and VisualVM leverage.