Tuesday, July 29, 2014

Book Review: Spring MVC Beginner's Guide

Spring MVC Beginner's Guide features the subtitle "Your ultimate guide to building a complete web application using all the capabilities of Spring MVC." It is published by Packt Publishing and authored by Amuthan G. Spring MVC Beginner's Guide is comprised of ten chapters and two appendices spanning over 280 pages of substantive content.

What is Spring MVC?

In case you're wondering what Spring MVC is, tutorialspoint's Spring MVC Framework Tutorial describes Spring MVC: "The Spring web MVC framework provides model-view-controller architecture and ready components that can be used to develop flexible and loosely coupled web applications. The MVC pattern results in separating the different aspects of the application (input logic, business logic, and UI logic), while providing a loose coupling between these elements." In the JavaWorld article Mastering Spring MVC, Steven Haines describes Spring MVC as "a full MVC implementation that follows the patterns and paradigms that Spring is known for, including [dependency injection]."

Preface

The Preface of Spring MVC Beginner's Guide opens by stating the objective of the book: "to introduce you to the incredible simplicity and power of Spring MVC." This Preface follows the pattern established in many other Packt books of presenting short (2-4 sentences each) summaries of its ten chapters and two appendices, presenting what readers need to try out the examples (Java SE 7 JDK Update 45 as stated in Preface, but it is stated in Chapter 1 that "Java 6 or any higher version is also sufficient"; Maven 3.1.0; Apache Tomcat 7.0; and Eclipse-based Spring Tool Suite 3.4.0), and describing the intended audience ("ideally suited for beginners and intermediate developers" with no assumed knowledge of the Spring Framework). I felt that my Spring experience made the early chapters of Spring MVC Beginner's Guide much more approachable and would recommend that readers of Spring MVC Beginner's Guide with no experience with Spring Framework at least read Overview of Spring Framework before reading this book.

Chapter 1: Configuring a Spring Development Environment

As its title suggests, the first chapter of Spring MVC Beginner's Guide is about setting up the environment for Spring-related development. The chapter includes sections on Windows installation of the JDK, Maven, Tomcat, and Spring Tool Suite (STS). These sections include text and screen snapshots, are highly detailed, and are easy to follow, but they only cover Windows installation of these products. A screen snapshot in the section on installing Java shows Java SE Development Kit 7 Update 51 being installed and the text instructions and output show that same Java 7 Update 51 installed.

Chapter 1 indicates how to download, install, and individually configure JDK, Maven, Tomcat, and Spring Tool Suite. It then demonstrates how to configure these tools to work together for a complete, integrated Spring development environment. The chapter then moves onto preliminary steps of delivering on the book's stated theme: "developing a simple e-commerce site step-by-step."

Chapter 2: Spring MVC Architecture – Architecting Your Web Store

Although the end of the first chapter introduced a simple Spring MVC-based application, Chapter 2 of Spring MVC Beginner's Guide explains the concepts behind this and presents a "deep-dive into Spring MVC architecture." The chapter introduces the Spring MVC DispatcherServlet in more detail (including discussion of the web.xml file and @RequestMapping annotation) than in the first chapter. The discussion of Spring MVC web configuration in the second chapter specifically discusses the XML elements <mvc:annotation-driven /> and <context:component-scan />.

Chapter 3: Control Your Store with Controllers

The initial two chapters of Spring MVC Beginner's Guide are primarily high-level, general, and relatively introductory, but Chapter 3 marks the beginning of the more focused chapters. This chapter focused on Controllers opens up with the statement, "Controllers are presentation layer components that are responsible for responding to user actions." In the course of this chapter's coverage of Spring MVC Controllers, topics such as the @Controller annotation and @RequestMapping annotation. The discussion under the heading "The role of a controller in Spring MVC" well into the third chapter (page 74) discusses the role the Controller plays in Spring MVC.

The third chapter also discusses general web application architecture and describes the typical "layers" in a web application: presentation, domain/model, services, and persistence. Each layer is described in text and then illustrated with code samples. Other discussed topics include handler mappings, URI template patterns, matrix variables, and handling GET request parameters.

Chapter 4: Working with Spring Tag Libraries

Chapter 4 of Spring MVC Beginner's Guide focuses on "predefined [JSP] tag libraries of Spring" that enable "[binding of] form elements to model data" and "support various other common functionalities, such as, externalizing messages and error handling." The chapter introduces the JSP Standard Tag Library (JSTL) and then introduces Spring Form Tags. Several pages of descriptive text and code examples demonstrate how @ModelAttribute can be used to bind a Java object to a Spring MVC form. The chapter also discusses and demonstrates how to customize Spring MVC binding using WebDataBinder and @InitBinder.

Moving text out of the JSP and into external properties files in Spring MVC forms is another subject covered in Chapter 4. This section demonstrates and explains how to apply <spring:messsage>. This chapter on applying Spring Tags concludes with a demonstration of adding a small portion of Spring Security to the Spring MVC application to enable authentication and authorization.

Chapter 5: Working with View Resolver

The fifth chapter of Spring MVC Beginner's Guide begins with the assertion that "Spring MVC maintains a high level of decoupling between the view and controller" and then demonstrates how this is accomplished using View Resolvers and Views. The chapter provides in-depth descriptions and code examples demonstrating use of the RedirectView, access of static resources with <mvc:resources>, parsing of request into multiple parts, and serving "different representations of the same resource" with ContentNegotiatingViewResolver.

One of the main points of Chapter 5 is that Spring MVC separates the view approach from the controller. This is demonstrated with examples of using InternalResourceViewResolver for JSP views, MappingJacksonJsonView for JSON views, and MarshallingView for XML views. Chapter 5 also discusses and demonstrates use of the ResponseStatusExceptionResolver and ExceptionHandlerExceptionResolver implementations of the HandlerExceptionResolver interface for "[resolving] exceptions that are thrown during controller mapping or execution."

Chapter 6: Intercept Your Store with Interceptor

Chapter 6 of Spring MVC Beginner's Guide is about Spring MVC interceptors, which the chapter states are "used to intercept actual web requests before or after they are processed" that can be related to the "filter concept of servlet programming." The chapter demonstrates through text and code how to write an implementation of the HandlerInterceptor interface (including referencing HandlerInterceptorAdapter as an easier way to implement HandlerInterceptor). The chapter also demonstrates application of standard provided interceptor for internationalization (LocaleChangeInterceptor) as well as providing two custom interceptors for performing audit logging and for performing conditional redirection.

Chapter 7: Validate Your Products with a Validator

Spring MVC Beginner's Guide's seventh chapter explains the "basics of setting up validation with Spring MVC." The chapter begins with a discussion of Java Bean Validation (JSR-303). The first example demonstrates "how to validate a form submission in a Spring MVC application" and uses Hibernate Validator and standard annotations-based constraints defined in package javax.validation.constraints. The next example expands on this to demonstrate customized constraints. The example demonstrates writing a custom constraint annotation and applying that annotation.

Chapter 7 transitions from JSR-303 validation to "classic Spring validation." The example in this section illustrates using "classic Spring validation" to validate the relationships between fields rather than individual field values ("cross-field validation"). The chapter concludes with a discussion and demonstration on how to combine JSR 303 Bean Validation and classic Spring Validation in the same application.

Chapter 8: Give REST to Your Application with Ajax

The eighth chapter of Spring MVC Beginner's Guide is on REST and Ajax. The chapter provides a very brief introduction to REST before moving into an example of applying @RequestBody and @ResponseBody to handle transformations between the serialization format and Java objects. The chapter discusses using Postman Chrome extension as client for testing the REST application just written.

The second part of Chapter 8 discusses using Ajax for communication between client and the REST service. This section begins with brief background on Ajax and then moves onto an example that uses trendy Angular.js.

Chapter 9: Apache Tiles and Spring Web Flow in Action

Chapter 9 of Spring MVC Beginner's Guide focuses on two teaching objectives: "Developing flow-based applications using Spring Web Flow" and "Decomposing pages using reusable Apache Tiles templates." The chapter emphasizes that Spring Web Flow and Apache Tiles are two completely independent web frameworks.

The section of Chapter 9 on using Spring Web Flow states about Spring Web Flow, "Spring Web Flow facilitates us to develop a flow-based web application easily" and adds, "A flow in a web application encapsulates a series of steps that guides a user through the execution of a business task." The initial example demonstrates writing an XML-based flow control file. The chapter uses that example XML to explain the contents of a Spring Web Flow XML file including its six possible states and transitions between states. There are a couple very lengthy examples to further explain details of Spring Web Flow.

Chapter 9's section on Apache Tiles describes Tiles as a "template composition framework" that allows "developers to define reusable page fragments (tiles) that can be assembled into a complete web page at runtime." Just as the section on Spring Web Flow focuses on the SWF's web flow XML file, the Apache Tiles section focuses on tile-definition.xml. The example demonstrates using Apache Tiles to implement JSPs consisting of composite templates in a Spring MVC application.

Chapter 10: Testing Your Application

The final chapter of Spring MVC Beginner's Guide opens with an overview of and a brief discussion regarding the benefits and challenges of unit testing web applications. The chapter then explains and illustrates testing a Spring MVC application's domain object (unit testing with JUnit), validator (integration testing with Spring TestContext Framework), controller (Spring Test Context framework mocking), and REST-exposed service.

Appendix A and Appendix B

Appendix A ("Using the Gradle Build Tool") briefly describes Gradle and how to use Gradle to build the book's examples. The examples throughout the book use Maven for the builds, but the presence of this appendix focused on Gradle is a sign of the trend in Java builds. Appendix B contains the answers to the "Pop Quiz" questions that occur throughout the book.

General Observations
  • Spring MVC Beginner's Guide jumps right into discussing how to apply Spring MVC without explaining what it is.
    • On the page labeled 43 that exists well into the second chapter, there is some discussion on "Model View Controller" in general before contrasting "MVC" and "web MVC" (the latter of which is implemented with Spring MVC).
    • The page labeled 45 in Spring MVC Beginner's Guide has a graphic under the heading "An overview of the Spring MVC request flow" that should probably be understood early in the reading of this book.
    • This is likely not a problem in most cases as a reader interested in reading about with "Spring MVC" in its title is likely to know what Spring MVC is.
    • A reader who is not familiar with what Spring MVC is might read a quick introduction before reading this book.
    • I would recommend that a reader of this book have some awareness of servlets and of the Spring Framework. The "Beginner's" part of the title is in reference to Spring MVC, but this book is not an introduction to the general Spring Framework and will be far easier to read for those with some Spring development experience.
  • Many of the examples heavily use Spring Tool Suite and many of the steps in the examples are made simpler because of that Eclipse-based tool. Readers with little experience using Spring or Spring MVC are well-served to use Spring Tool Suite to work through these examples.
  • The PDF version of Spring MVC Beginner's Guide that I reviewed has full-color screen snapshots.
  • Code samples tend to be well organized and indented, but have neither color coded syntax nor line numbers.
  • Multiple versions of the JDK are referenced in Spring MVC Beginner's Guide. The Preface states that JDK 7 Update 45 or newer is required, but Chapter 1 states that any JDK 6 release or later should be sufficient and a screen snapshot in Chapter 1 indicates JDK 7 Update 51 being installed. As far as I can determine, Java SE 6 JDK is probably sufficient for examples in this book, but I cannot be certain of that.
Conclusion

Spring MVC Beginner's Guide is a comprehensive resource on Spring MVC. Although its title includes "Beginner's", I think most readers would be well-advised to have some prior experience with the Spring Framework and with JavaServer Pages and Java servlets to get the most out of this book. The "Beginner's" portion of the title applies specifically to Spring MVC rather than to the more general Spring Framework or JSP/servlet programming.

Monday, July 28, 2014

Book Review: Creating Flat Design Websites

Creating Flat Design Websites was recently published by Packt Publishing. Written by António Pratas, this book's subtitle is "Design and develop your own flat design websites in HTML." The book features six chapters and is fewer than 100 pages in substantive length. In case you're wondering what the term flat design means, I include a definition here provided in the first chapter of Creating Flat Design Websites:

[Flat design] is characterized by a really minimalistic look, focused on removing all extra elements and effects from a design, such as bevels, shadows, lighting effects, depth, texture, and every element that creates and gives an extra dimension to these elements.
Preface

Staying true to the pattern established in other Packt titles, the Preface of Creating Flat Design Websites contains useful overview information such as a chapter-by-chapter summary of what the book covers, what is required for applying the book's examples (primarily PhotoShop, a text editor, jQuery, and a web browser), and who the intended audience is ("designers and developers who are interested in creating websites").

Chapter 1: What is Flat Design?

The initial chapter of Creating Flat Design Websites defines "flat design" (see definition shown above) and talks about the history of flat design. The coverage of flat design's history included reference to The Flat Design Era post and includes examples of how Microsoft and other vendors have adopted flat designs in recent years.

Chapter 1 contrasts Skeuomorphic and flat designs and lists advantages and disadvantages of each. As part of this, Skeuomorphic design is defined: "Skeuomorphic design, also known as realism, consists of creating visual elements that represent their original, physical counterpart." The chapter also includes a detailed example of creating buttons in both styles using PhotoShop that enables the differences between the styles to be made more obvious. The chapter concludes with side-by-side examples of well-known logos that have been changed from skeuomorphic style to flat style.

Chapter 2: Designing in Flat

The second chapter of Creating Flat Design Websites explains how flat design is more than just minimalistic because flat design has a "specific look and certain rules." The chapter discusses some of these rules about what is "allowed" and "not allowed" for a design to be considered a flat design. For example, shadows are allowed while gradients and textures are not. The chapter discusses applying photographs and illustrations in flat designs and introduces 960 Grid System (fixed grid) and Unsemantic (responsive grid) during its discussion of application of grids in flat designs. The chapter lists five "freely available [predesigned flat] icon sets online."

Typography and fonts in flat design are also covered in the second chapter. This section includes a list of six "font families suggested for use in flat design." The section on flat colors introduces and describes how to use the Flat UI Colors site to select flat colors. The chapter concludes with several pages of screen snapshots demonstrating good flat designs that could be used to inspire readers' own flat designs.

I was happy to see that the author states that flat design is not always the answer. Most books covering a topic act as if the tool or approach being covered always works better than the alternatives and it's always more credible to me when an author evangelizes an approach while also recognizing its limitations.

Chapter 3: Creating a Flat and Usable Interface

Chapter 3 of Creating Flat Design Websites addresses maintaining usability features in flat designs, which can be an issue because flat "style may create several usability problems given its simple look." The chapter outlines reasons that usability is important in web pages and sites and then discusses balancing functional and appearance aspects. The chapter then provides an example of applying some of this chapter's principles to migrate a web site to flat design.

The third chapter explains how flat design can lead to removing of some of the normal cues of the existence of web form and then goes through an example of how to convert a web form to a flat design in which it is still obvious that a web form is involved. The chapter concludes with a discussion regarding "flat readability" and advises to "always look to achieve the best contrast between text and background."

Chapter 4: Designing Your Own Flat Website

Creating Flat Design Websites's fourth chapter describes steps to take to design one's own flat website. Topics include planning the website based on anticipated audience, designing the sections of the website, and using tools such as PhotoShop, uiFaces, flatuicolors.com, Lightbox, and Dribbble to create an attractive web page adhering to flat design principles. The chapter introduces Designmodo's Flat UI Free as a tool for quickly jump-starting the website design process.

Chapter 5: Developing Your Site

Chapter 5 of Creating Flat Design Websites extends the example started in Chapter 4. Topics covered here include laying out the project's directory structure, preparing images and generating image assets with PhotoShop, developing the website with Bootstrap-based Designmodo Flat UI Free, and applying HTML, CSS, and JavaScript/jQuery to the website.

Chapter 6: Creating Your Own Flat UI Kit

The final chapter of Creating Flat Design Websites references Alexandre Crenn's FREE Pack UI .PSD as an example of a personal Flat UI kit and then explains how the reader can create his or her own kit. The chapter covers creating elements with PhotoShop, creating sprites with Sprite Cow, writing HTML, writing CSS (includes introduction to CSS3 Generator), and documenting the developed code.

General Observations
  • The screen snapshots in the PDF edition I reviewed are full color, which is important for illustrating design principles and examples.
  • I liked the frequent use and mention of freely available online resources and tools for building flat design websites.
  • It seems difficult to really explain flat design and differentiate it from other minimalistic design styles with words alone. The many examples of existing flat designs and of implementing flat designs helped make the subject clearer.
  • Although Creating Flat Design Websites includes content related to design and code development, it seems more focused on design than on code development. For me, this was a positive aspect as I am more comfortable with how to implement the code than I am with how to design the page. I appreciated the tips of flat design and design in general.
Conclusion

Creating Flat Design Websites introduces the concept of flat web designs and provides descriptions, code examples, and visual examples of implementing flat designs. This relatively short book provides a nice introduction into what constitutes flat design and how the reader can begin creating his or her own flat design websites.

Saturday, July 26, 2014

Applying S.T.O.P. To Software Development

The acronym STOP (or STOPP) is used by several organizations (United States Army, Hunter's Ed, Mountain Rescue, Search and Rescue, Boy Scouts of America), often for describing how to cope with wilderness survival situations or other situations when one is lost (especially outdoors). The "S" typically stands for "Stop" (some say it stands for "Sit"), the "T" stands for "Think" (some say "Take a Breath"), the "O" stands for "Observe", and the "P" stands for "Plan" (some say it stands for "Prepare"). When there is a second "P", that typically stands for "Proceed." In other words, the best approach to use for wilderness survival is to stop, think, observe, and plan before proceeding (taking action). Proceeding without a plan based on thinking and observation is rarely a good idea in for those in a survival situation.

Our approaches to developing software and fixing issues with existing software can benefit from the general guidance STOP provides. In this, my 1000th blog post, I look at applying the principles of STOP to software development.

Developing New Software

For many of us who consider ourselves software developers, programmers, or even software engineers, it can be difficult to ignore the impulse to jump right in and write some code. This is especially true when we're young and relatively inexperienced with the costs associated with that approach. These costs can include bad (no) overall design and spaghetti code. Code written with this approach often suffers from "stream of consciousness programming" syndrome, in which the code comes out in the way one is thinking it. The problem with "steam of consciousness" programming is that it may only be coherent to the author at that particular moment and not later when outside of that "stream of consciousness." It is likely not to be coherent by anyone else. By first considering at least at a high level how to organize the code, the developer is more likely to build something that he or she and others will understand later. At some point, we all write lines of code based on our "stream of consciousness," but that's much more effective if it's implementing a small number of lines in well-defined methods and classes.

When implementing a new feature, the software developer generally benefits from taking the following general steps:

  • Stop:
    • Be patient and don't panic.
    • Don't allow schedule pressure to force you into hasty decisions that may not save any time in the long-run and can lead to problematic code that you and others will have to deal with over the long run.
    • Gather available facts such as what the desired functionality is (customer requirements or expressed desires).
  • Think:
    • Consider what portions of the desired new feature might already be provided.
    • Consider alternative approaches that might be used to implement the desired feature.
    • Consider which existing tools, libraries, and people are already available to might satisfy the need or help satisfy it.
    • Consider design and architectural implications related to existing functionality and likely potential future enhancements.
  • Observe:
    • Confirm available existing tools and libraries that might be used to implement the new feature or which could be expanded to work with the new feature.
    • If necessary, search for blogs, forums, and other sources of information on approaches, libraries, and tools that might be used to implement the new feature.
    • Use others' designs and code as inspiration for how to implement similar features to what they implemented (or, in some cases, how not to implement a similar feature).
  • Plan:
    • "Design" implementation. In simpler cases, this may simply be a mental step without any formal tools or artifacts.
    • If adhering to test-driven development principles, plan the tests to write first. Even if not strictly applying TDD, make testability part of your consideration of how to design the software.
    • Allocate/estimate the time needed to implement or effort needed to accomplish this, even if you call it by a fancy name such as Story Points.
  • Proceed:
    • Implement and test and implement and test functionality.
    • Get feedback on implemented functionality from customers and other stakeholders and repeat cycle as necessary.

The above are just some of the practices that can go into applying the STOP principle to new software development. There are more that could be listed. These steps, especially for simpler cases, might take just a few minutes to accomplish, but those extra few minutes can lead to more readable and maintainable code. These steps can also prevent pollution of an existing baseline and can, in some cases, be the only way to get to a "correct" results. More than once, I have found myself un-doing a bunch of stream of consciousness programming (or doing some significant code changing) because I did not apply these simple steps before diving into coding.

Fixing and Maintaining Software

When fixing a bug in the software, it is very easy to make the mistake of fixing a symptom rather than the root cause of the problem. Fixing the symptom might bring short-tem benefit of addressing an obviously wrong behavior or outcome, but often hides a deeper problem that may manifest itself with other negative symptoms or, even worse, might contribute to other undetected but significant problems. Applying STOP to fixing bugs can help address these issues.

  • Stop:
    • Be patient and don't panic.
    • Don't allow schedule pressure to force you into merely covering up a potentially significant problem. Although a bad enough (in terms of financial loss or loss of life) problem may require you to quickly address the symptom, ensure that the root cause is addressed in a timely fashion as well.
  • Think:
    • Consider anything you or your team recently added to the baseline that may have introduced this bug or that may have revealed a pre-existing bug.
    • Consider the effects/costs of this bug and determine where fixing the bug falls in terms of priority.
    • Consider whether this bug could be related to any other issues you're already aware of.
    • Consider whether this "bug" is really a misunderstood feature or a user error before fixing something that wasn't broken.
  • Observe:
    • Evaluate appropriate pieces of evidence to start determining what went wrong. These might be one or more of the following: reading code itself and thinking through its flows, logs, debugger, tools (IDEs warnings and hints and Java examples include VisualVM, jstack, jmap, JConsole), application output, defect description, etc.
    • Building on the Thinking step:
      • Ensure that unit tests and other tests ran without reporting any breakage or issue.
      • Evaluate revision history in your configuration management system to see if anything looks suspicious in terms of being related to the bug (same class or dependency class changed, for example).
      • Evaluate whether any existing bugs/JIRAs in your database seem to be related. Even resolved defects can provide clues as to what is wrong or may have been reintroduced.
  • Plan:
    • Plan new unit test(s) that can be written to find this type of defect in the future in case it is reintroduced at some point and as a part of your confirmation that you have resolved the defect.
    • Plan/design solution for this defect. At this point, if the most thorough solution is considered prohibitively expensive, you may need to choose a cheaper solution, but you are doing so based on knowledge and a deliberate decision rather than just doing what's easiest and sweeping the real problem under the rug. Document in the DR's/JIRA's resolution that this decision was made and why it was made.
    • Plan for schedule time to implement and test this solution.
  • Proceed
    • Implement and test and implement and test functionality.
    • Get feedback on implemented functionality from customers and other stakeholders and repeat cycle as necessary.

There are other tactics and methodologies that might be useful in resolve defects in our code as part of the STOP approach. The important thing is to dedicate at least a small amount of time to really thinking about the problem at hand before diving in and ending up in some cases "fixing" the problem multiple times until the actual and real problem (the root cause) is really fixed.

Conclusion

Most software developers have a tendency to dive right in and implement a new feature or fix a broken feature as quickly as possible. However, even a small amount of time applying S.T.O.P. in our development process can bring benefits of more efficiency and a better product. Stopping to think, observe, and plan before proceeding is as effective in software development as it is in wilderness survival. Although the stakes often aren't as high in software development as they are in wilderness survival, there is no reason we cannot still benefit from remembering and adhering to the principles of S.T.O.P.

Thursday, July 24, 2014

Book Review: WildFly Performance Tuning

Packt Publishing recently published Arnold Johansson's and Anders Welén's WildFly Performance Tuning with the subtitle, "Develop high-performing server applications using the widely successful WildFly platform." As I discussed in my post Video Review: JBoss EAP Configuration, Deployment, and Administration, WildFly is the new name for the freely downloaded (community edition) version of the JBoss Application Server ("next generation of JBoss AS"). This post is my review of the PDF version of WildFly Performance Tuning.

Preface: Summary of Book's Contents and Who Book is Intended For

The Preface of Packt books is typically a useful place to quickly determine whether a Packt book is likely to address the subjects one is interested in and this continues to be the case with this book. The Preface of WildFly Performance Tuning contains brief summaries of what each of the book's ten chapters covers. It also explains what readers need to run many of the examples covered in the book (the book does cover some performance metrics tools not on this list because they are considered "optional" by the authors): Java SE 7 (I assume Java SE 8 will be okay), VisualVM (examples are based on VisualVM 1.3.6), and JMeter.

The "Who this book is for" section of the Preface describes the audience of this book and is a pretty good indicator of the content of the book: "This book is for anyone interested in what performance tuning is all about and how to do it using Java technologies, the WildFly Application Server, and other Open Source software." I also like that this section highlights something I was going to highlight in this review: Chapter 1 covers very general performance tuning principles, Chapter 2 and Chapter 3 cover performance tuning principles general to Java applications, and the remainder of the chapters focus on WildFly-specific performance tuning concepts and principles.

Chapter 1: The Science of Performance Tuning

The authors state in the initial chapter of WildFly Performance Tuning that "performance testing and tuning is a science." The chapter introduces, and describes in some detail, key performance-related terms such as response time, throughput, and scalability. The first chapter also covers some common high-level "performance tuning anti-patterns," many of which are more process and organizational in nature than technical in nature. Chapter 1 describes the software development cycle and how performance considerations need to be made in architecture and design phases and not simply just before shipping. There is a small amount of introductory WildFly-specific and Java/JMX-specific discussion in this chapter, but most of the chapter is on concepts at a higher level than a specific application server or even specific programming language or platform.

Chapter 2: Tools of the Tuning Trade

The second chapter of WildFly Performance Tuning discusses using tools "from the open source ecosystem" for JVM profiling and sampling, for operating system monitoring, and for load testing. The tools covered include VisualVM (including remote access with jstatd and JMX), top, netstat, vmstat (and vm_stat), OS X Activity Monitor, WildFly Command Line Interface (CLI), WildFly Management Console (HAL), and Apache JMeter.

Although the focus of this second chapter is on tools for profiling, monitoring, and load testing, there is more WildFly specific content in this chapter than in the first chapter. Several of the concepts and tools described are done so from a WildFly perspective.

Chapter 3: Tuning the Java Virtual Machine

Chapter 3 of WildFly Performance Tuning takes on the ambitious task of explaining basics of the JVM. It's particular focus is on tuning Oracle's HotSpot JVM and the authors state, "We focus on Java SE 7 based on the Oracle JDK 7 distribution in this book, we'll follow that trail by looking at its VM implementation named Hotspot." One consequence of the Java 7 JVM being covered is that changes to the JVM in Java 8 (such as removal of permanent generation) are not covered. A single chapter is necessarily briefer coverage than one would find in a full book such as Java Performance.

Chapter 3's JVM coverage begins by describing the relationship between concepts described by the terms Java Virtual Machine (JVM), Java Runtime Environment (JRE), and Java Development Kit (JDK). I would expect readers of a book on performance tuning related to a Java-based application server to already know the difference between the JDK and JRE, but this material is there for those who aren't aware of the difference. I think Oracle's JRE 8 Download page has about as clear of an explanation of the difference between the JDK and JRE as I have seen for non-developers:

Do you want to run JavaTM programs, or do you want to develop Java programs? If you want to run Java programs, but not develop them, download the Java Runtime Environment, or JRETM. If you want to develop applications for Java, download the Java Development Kit, or JDKTM. The JDK includes the JRE, so you do not have to download both separately.

The third chapter talks about the JVM stacks, the JVM heap, and garbage collection (GC) with a HotSpot JVM focus. The chapter introduces some of the most commonly used HotSpot JVM options such as -XX:+PrintFlagsFinal, -client and -server, -Xss, -Xms, and -Xmx. After reviewing these options, the chapter looks at how to determine through analysis and monitoring what the settings should be for initial heap size and maximum heap size. The chapter looks at using heap options -XX:MaxNewSize, -XX:NewSize, and -XX:NewRatio together to manage the size of the young generation portion of the heap.

Chapter 3's coverage of balancing eden space and survivor space on the JVM heap introduces the options -XX:+PrintTenuringDistribution, -XX:InitialTenuringThreshold, -XX:MaxTenuringThreshold, -XX:SurvivorRatio, and -XX:TargetSurvivorRatio. There is also a short description of permanent generation heap space with a side note that it is removed in Java 8.

The JVM option -XX:+UseLargePages for large memory pages is covered in Chapter 3 before moving onto discussion of OutOfMemoryError. This portion of the chapter has a highly useful break-down of several of the different specific types of OutOfMemoryError and how to deal with them.

There is also Chapter 3 discussion on memory leaks in Java and how they can be identified and resolved. An example is presented to illustrate using VisualVM to diagnose and address a memory leak.

The section of Chapter 3 that focuses on garbage collection strategies introduces and briefly describes the four "most common strategies of the Hotspot VM" (serial collector, parallel collector, concurrent collector, and G1 ["incremental parallel compacting"] collector). This section also briefly compares these collectors and describes the situations in which each fits best.

Chapter 3 introduces three ways to specify JVM options for the JVM in which an instance of WildFly is running. The chapter then goes onto introduce and briefly describe four virtual machine logging parameters that the authors assert "should always be set in your production environment to log relevant information" (-verbose:gc, -XX:+PrintGCDetails, -XX:+PrintTenuringDistribution, and -Xloggc). The authors list several available tools (such as GCViewer and the tools provided with the Oracle JDK) for analyzing this log output.

Chapter 4: Tuning WildFly

Each of the first three chapters provided a little more WildFly specific information than the one before it, but even the third chapter was mostly more "general Java" than "specifically WildFly." Chapter 4, which begins about one-third of the way into the book, is the first chapter with heavy emphasis on WildFly specifically. It is in this chapter that WildFly background information is provided. For example, the authors state, "WildFly is, as of 2013, the new name of the historically famous and well renowned JBoss Application Server (JBoss AS)."

The fourth chapter of WildFly Performance Tuning provides a history of WildFly as it evolved from Enterprise Java Bean Open Source Software (EJB-OSS) to JBoss Application Server to WildFly. A table in this section associates major releases of JBoss/WildFly with the version of J2EE/Java EE it implemented (albeit not always being certified). This table indicates that WildFly 8 (first version with new name WildFly) implements Java EE 7. There is a brief section of Chapter 4 on WildFly architecture that talks about WildFly's modular design with separate components and subsystems.

Chapter 4 describes each of the WildFly subsystems and their associated pools: thread pool executor with six of its offered thread pools (listed in Table of Contents), resource adapter subsystem (makes use of Java EE Connector Architecture [JCA] and Resource Archives [RAR files]), batch subsystem (implements Java EE 7 Batch API), remoting subsystem (based on NIO and XNIO), transaction subsystem, and logging subsystem. Several pages are devoted to configuring WildFly logging.

Chapter 5: EJB Tuning in WildFly

The fifth through ninth chapters of WildFly Performance Tuning each focus on tuning of a particular Java EE component type within WildFly. Chapter 5's focus is Enterprise JavaBean (EJB) tuning and the chapter begins with a history of EJB and a brief description of the various types of EJBs.

The "Performance tuning EJBs in WildFly" section of the fifth chapter begins by showing the CLI command to instruct WildFly to generate detailed EJB-related statistics. Several of the approaches listed in this chapter are true of Java EE optimization in general (not limited to WildFly). Examples of this include using pass-by-reference (@Local/shallow/not serialized) rather than pass-by-value (@Remote/deep/serialized) when possible and using coarse grained access for remote calls.

Chapter 5 contains numerous examples of WildFly-specific monitoring and tuning of different types of EJBs (stateless, stateful, singleton session, and message-driven). Many of the monitoring examples use CLI commands and JMX/JConsole and the tuning examples tend to be via XML configuration.

Chapter 6: Tuning the Persistence Layer

WildFly Performance Tuning's sixth chapter presents some WildFly-specific tuning information, but much of the information presented in this chapter is more general (database design, SQL statement design, JDBC, etc.). The chapter begins with an overview of normalization in relational database design, database partitioning and indexes, all at a high-level and covered in general terms rather than focusing on a specific relational database implementation.

Chapter 6 describes three approaches to tuning JDBC: connection pooling, setting fetch size [ResultSet.setFetchSize(int) and Statement.setFetchSize(int)], batching [Statement.executeBatch()], and use of PreparedStatements. The sections on connection pooling and using PreparedStatements include information on how to specify and monitor these things in WildFly. There are short sections in this chapter on database isolation and low-level JDBC network tuning.

A significant portion of Chapter 6 is focused on tuning Java Persistence API (JPA) and Hibernate. The authors state, "WildFly 8.0.0.Final makes use of the JPA 2.1 specification and the bundled module since the provider is Hibernate in Version 4.3.1." Numerous tactics and strategies related to JPA are covered here and the list of them is available in the listing for Chapter 6 in the Table of Contents (includes entity caching and named queries).

Most of Chapter 6 applies to the persistence layer in any Java EE application server, but there are some sections that discuss WildFly specifics.

Chapter 7: Tuning the Web Container in WildFly

Chapter 7 introduces Undertow, described on its web page as "a flexible performant web server written in java, providing both blocking and non-blocking API’s based on NIO." The chapter provides some Undertow implementation details and also mentions several things that are not yet fully implemented, but are planned for the future. The seventh chapter moves onto tuning the Undertow Worker and buffer pool using WildFly CLI and the WildFly Admin Console.

Chapter 7 introduces Jastow, which it describes as "the JSP engine in Undertow" and "a fork of the Apache Jasper project." This section provides a table with a subset of options available for tuning Jastow performance and demonstrates using CLI, JMX, and Admin Console to view these settings.

WildFly Performance Tuning's seventh chapter ends with a discussion of using Apache HTTPD as a "frontend" to WildFly. As part of this discussion, this section introduces Apache JServ Protocol (AJP) as an alternative to HTTP for communication between the Apache HTTPD server and the WildFly instance(s). Examples are included to demonstrate configuring Apache HTTPD and WildFly for this configuration.

Chapter 8: Tuning Web Applications and Services

Chapter 8 begins it coverage of web applications and web services with a history of web application development from Common Gateway Interface (CGI) to Java servlets to JavaServer Pages (JSP) to JavaServer Faces (JSF). There is also some discussion regarding why one might choose to use or not use JSF and the authors mention that "WildFly ships with the Mojorra JSF 2.2 implementation."

Chapter 8 has sections on tuning web applications and tuning web services. The section on tuning web applications presents performance considerations related to servlets and JSPs (with JSTL), JSF, and JSF with PrimeFaces 4.x. The chapter uses a "simple data table" example to illustrate performance considerations for all three approaches to building Java EE web applications. Some of the topics covered along the way include JSP scopes, session timeouts, asynchronous servlets, JSF client state versus server state, and JSF immediate. There are several topics specific to Undertow and JSF component libraries (PrimeFaces and RichFaces) also covered in the web application performance tuning section before brief discussion of Java support for WebSockets.

Chapter 8's section of performance tuning web services covers Java API for XML Web Services (JAX-WS) along with WildFly's implementation of JAX-WS [JBossWS and Apache CXF (JBossWS-CXF)] and JAXB. Recommendations for reducing XML messaging overhead are covered. The section of web services tuning also includs a very short section on REST-based web services [Java API for RESTful Web Services (JAX-RS)].

Chapter 9: JMS and HornetQ

WildFly Performance Tuning's ninth chapter focuses on Java Message Service (JMS) and HornetQ version 2.4.1.Final (WildFly's JMS broker provider). The chapter covers JMS basics such as JMS destination types and JMS message types. It discusses general JMS tuning ideas such as tuning the JMS session and tuning the JMS MessageProducer.

A significant portion of Chapter 9 focuses on performance tuning options specific to HornetQ. These recommendations, which include "using input and output streams for handling large messages, are listed in the Table of Contents. The chapter concludes with examples of using WildFly-provided mechanisms to monitor HornetQ.

Chapter 10: WildFly Clustering

The tenth and final chapter of WildFly Performance Tuning focuses on WildFly clustering. The chapter begins with definitions of clustering and distributed computing terms and discussion of the basics of clustering. It opens up its coverage of WildFly clustering by pointing out that WildFly's clustering is built upon JGroups and Infinispan. The chapter then covers using each of these foundational open source projects. Chapter 10 then runs down the Java EE components and explains which have Java EE standard cluster support and which have WildFly proprietary cluster support.

General Observations
  • WildFly Performance Tuning covers both Java EE and JDBC standard tuning concepts as well as proprietary WildFly tuning concepts. Readers who are familiar with tuning other application servers and JDBC applications will likely be able to skim or skip chapters 1 through 3 and much of chapter 6. Readers who are new to performance tuning of Java EE (and Java SE) applications will likely find the inclusion of these more general chapters useful.
  • Chapter 3 ("Tuning the Java Virtual Machine") has very little WildFly-specific detail in it, but a related consequence of this is that this is a great introductory chapter for anyone interested in learning principles of JVM tuning that apply to various Java EE implementations and even to Java SE applications. This chapter provides a quick preview of the most important concepts in JVM tuning.
  • The PDF edition I reviewed has color screen snapshots.
  • Code examples are generally well-formatted and indented, but do not have any color syntax or line numbers.
  • With rare exceptions, the text flows well and sentences are generally easy to understand.
  • A relatively small number of items discussed in this book have been delayed to WildFly 9 or else are expected in the near future. The book does a nice job of calling these relatively infrequent cases out and making it clear what's in WildFly 8.x and what will be in WildFly 9.x.
Conclusion

WildFly Performance Tuning provides background information, descriptive text, and illustrative examples that allow the reader to learn principles, concepts, and approaches that can be applied to tuning a WildFly-hosted application. WildFly Performance Tuning mixes general JVM and Java EE tuning principles and tools with WildFly-specific tuning principles and tools to provide a composite view of how performance tuning of WildFly applications can be performed.

Monday, July 7, 2014

Custom Cassandra Data Types

In the blog post Connecting to Cassandra from Java, I mentioned that one advantage for Java developers of Cassandra being implemented in Java is the ability to create custom Cassandra data types. In this post, I outline how to do this in greater detail.

Cassandra has numerous built-in data types, but there are situations in which one may want to add a custom type. Cassandra custom data types are implemented in Java by extending the org.apache.cassandra.db.marshal.AbstractType class. The class that extends this must ultimately implement three methods with the following signatures:

public ByteBuffer fromString(final String) throws MarshalException
public TypeSerializer getSerializer()
public int compare(Object, Object)

This post's example implementation of AbstractType is shown in the next code listing.

UnitedStatesState.java - Extends AbstractType
package dustin.examples.cassandra.cqltypes;

import org.apache.cassandra.db.marshal.AbstractType;
import org.apache.cassandra.serializers.MarshalException;
import org.apache.cassandra.serializers.TypeSerializer;

import java.nio.ByteBuffer;

/**
 * Representation of a state in the United States that
 * can be persisted to Cassandra database.
 */
public class UnitedStatesState extends AbstractType
{
   public static final UnitedStatesState instance = new UnitedStatesState();

   @Override
   public ByteBuffer fromString(final String stateName) throws MarshalException
   {
      return getStateAbbreviationAsByteBuffer(stateName);
   }

   @Override
   public TypeSerializer getSerializer()
   {
      return UnitedStatesStateSerializer.instance;
   }

   @Override
   public int compare(Object o1, Object o2)
   {
      if (o1 == null && o2 == null)
      {
         return 0;
      }
      else if (o1 == null)
      {
         return 1;
      }
      else if (o2 == null)
      {
         return -1;
      }
      else
      {
         return o1.toString().compareTo(o2.toString());
      }
   }

   /**
    * Provide standard two-letter abbreviation for United States
    * state whose state name is provided.
    *
    * @param stateName Name of state whose abbreviation is desired.
    * @return State's abbreviation as a ByteBuffer; will return "UK"
    *    if provided state name is unexpected value.
    */
   private ByteBuffer getStateAbbreviationAsByteBuffer(final String stateName)
   {
      final String upperCaseStateName = stateName != null ? stateName.toUpperCase().replace(" ", "_") : "UNKNOWN";
      String abbreviation;
      try
      {
         abbreviation =  upperCaseStateName.length() == 2
                       ? State.fromAbbreviation(upperCaseStateName).getStateAbbreviation()
                       : State.valueOf(upperCaseStateName).getStateAbbreviation();
      }
      catch (Exception exception)
      {
         abbreviation = State.UNKNOWN.getStateAbbreviation();
      }
      return ByteBuffer.wrap(abbreviation.getBytes());
   }
}

The above class listing references the State enum, which is shown next.

State.java
package dustin.examples.cassandra.cqltypes;

/**
 * Representation of state in the United States.
 */
public enum State
{
   ALABAMA("Alabama", "AL"),
   ALASKA("Alaska", "AK"),
   ARIZONA("Arizona", "AZ"),
   ARKANSAS("Arkansas", "AR"),
   CALIFORNIA("California", "CA"),
   COLORADO("Colorado", "CO"),
   CONNECTICUT("Connecticut", "CT"),
   DELAWARE("Delaware", "DE"),
   DISTRICT_OF_COLUMBIA("District of Columbia", "DC"),
   FLORIDA("Florida", "FL"),
   GEORGIA("Georgia", "GA"),
   HAWAII("Hawaii", "HI"),
   IDAHO("Idaho", "ID"),
   ILLINOIS("Illinois", "IL"),
   INDIANA("Indiana", "IN"),
   IOWA("Iowa", "IA"),
   KANSAS("Kansas", "KS"),
   LOUISIANA("Louisiana", "LA"),
   MAINE("Maine", "ME"),
   MARYLAND("Maryland", "MD"),
   MASSACHUSETTS("Massachusetts", "MA"),
   MICHIGAN("Michigan", "MI"),
   MINNESOTA("Minnesota", "MN"),
   MISSISSIPPI("Mississippi", "MS"),
   MISSOURI("Missouri", "MO"),
   MONTANA("Montana", "MT"),
   NEBRASKA("Nebraska", "NE"),
   NEVADA("Nevada", "NV"),
   NEW_HAMPSHIRE("New Hampshire", "NH"),
   NEW_JERSEY("New Jersey", "NJ"),
   NEW_MEXICO("New Mexico", "NM"),
   NORTH_CAROLINA("North Carolina", "NC"),
   NORTH_DAKOTA("North Dakota", "ND"),
   NEW_YORK("New York", "NY"),
   OHIO("Ohio", "OH"),
   OKLAHOMA("Oklahoma", "OK"),
   OREGON("Oregon", "OR"),
   PENNSYLVANIA("Pennsylvania", "PA"),
   RHODE_ISLAND("Rhode Island", "RI"),
   SOUTH_CAROLINA("South Carolina", "SC"),
   SOUTH_DAKOTA("South Dakota", "SD"),
   TENNESSEE("Tennessee", "TN"),
   TEXAS("Texas", "TX"),
   UTAH("Utah", "UT"),
   VERMONT("Vermont", "VT"),
   VIRGINIA("Virginia", "VA"),
   WASHINGTON("Washington", "WA"),
   WEST_VIRGINIA("West Virginia", "WV"),
   WISCONSIN("Wisconsin", "WI"),
   WYOMING("Wyoming", "WY"),
   UNKNOWN("Unknown", "UK");

   private String stateName;

   private String stateAbbreviation;

   State(final String newStateName, final String newStateAbbreviation)
   {
      this.stateName = newStateName;
      this.stateAbbreviation = newStateAbbreviation;
   }

   public String getStateName()
   {
      return this.stateName;
   }

   public String getStateAbbreviation()
   {
      return this.stateAbbreviation;
   }

   public static State fromAbbreviation(final String candidateAbbreviation)
   {
      State match = UNKNOWN;
      if (candidateAbbreviation != null && candidateAbbreviation.length() == 2)
      {
         final String upperAbbreviation = candidateAbbreviation.toUpperCase();
         for (final State state : State.values())
         {
            if (state.stateAbbreviation.equals(upperAbbreviation))
            {
               match = state;
            }
         }
      }
      return match;
   }
}

We can also provide an implementation of the TypeSerializer interface returned by the getSerializer() method shown above. That class implementing TypeSerializer is typically most easily written by extending one of the numerous existing implementations of TypeSerializer that Cassandra provides in the org.apache.cassandra.serializers package. In my example, my custom Serializer extends AbstractTextSerializer and the only method I need to add has the signature public void validate(final ByteBuffer bytes) throws MarshalException. Both of my custom classes need to provide a reference to an instance of themselves via static access. Here is the class that implements TypeSerializer via extension of AbstractTypeSerializer:

UnitedStatesStateSerializer.java - Implements TypeSerializer
package dustin.examples.cassandra.cqltypes;

import org.apache.cassandra.serializers.AbstractTextSerializer;
import org.apache.cassandra.serializers.MarshalException;

import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;

/**
 * Serializer for UnitedStatesState.
 */
public class UnitedStatesStateSerializer extends AbstractTextSerializer
{
   public static final UnitedStatesStateSerializer instance = new UnitedStatesStateSerializer();

   private UnitedStatesStateSerializer()
   {
      super(StandardCharsets.UTF_8);
   }

   /**
    * Validates provided ByteBuffer contents to ensure they can
    * be modeled in the UnitedStatesState Cassandra/CQL data type.
    * This allows for a full state name to be specified or for its
    * two-digit abbreviation to be specified and either is considered
    * valid.
    *
    * @param bytes ByteBuffer whose contents are to be validated.
    * @throws MarshalException Thrown if provided data is invalid.
    */
   @Override
   public void validate(final ByteBuffer bytes) throws MarshalException
   {
      try
      {
         final String stringFormat = new String(bytes.array()).toUpperCase();
         final State state =  stringFormat.length() == 2
                            ? State.fromAbbreviation(stringFormat)
                            : State.valueOf(stringFormat);
      }
      catch (Exception exception)
      {
         throw new MarshalException("Invalid model cannot be marshaled as UnitedStatesState.");
      }
   }
}

With the classes for creating a custom CQL data type written, they need to be compiled into .class files and archived in a JAR file. This process (compiling with javac -cp "C:\Program Files\DataStax Community\apache-cassandra\lib\*" -sourcepath src -d classes src\dustin\examples\cassandra\cqltypes\*.java and archiving the generated .class files into a JAR named CustomCqlTypes.jar with jar cvf CustomCqlTypes.jar *) is shown in the following screen snapshot.

The JAR with the class definitions of the custom CQL type classes needs to be placed in the Cassandra installation's lib directory as demonstrated in the next screen snapshot.

With the JAR containing the custom CQL data type classes implementations in the Cassandra installation's lib directory, Cassandra should be restarted so that it will be able to "see" these custom data type definitions.

The next code listing shows a Cassandra Query Language (CQL) statement for creating a table using the new custom type dustin.examples.cassandra.cqltypes.UnitedStatesState.

createAddress.cql
CREATE TABLE us_address
(
   id uuid,
   street1 text,
   street2 text,
   city text,
   state 'dustin.examples.cassandra.cqltypes.UnitedStatesState',
   zipcode text,
   PRIMARY KEY(id)
);

The next screen snapshot demonstrates the results of running the createAddress.cql code above by describing the created table in cqlsh.

The above screen snapshot demonstrates that the custom type dustin.examples.cassandra.cqltypes.UnitedStatesState is the type for the state column of the us_address table.

A new row can be added to the US_ADDRESS table with a normal INSERT. For example, the following screen snapshot demonstrates inserting an address with the command INSERT INTO us_address (id, street1, street2, city, state, zipcode) VALUES (blobAsUuid(timeuuidAsBlob(now())), '350 Fifth Avenue', '', 'New York', 'New York', '10118');:

Note that while the INSERT statement inserted "New York" for the state, it is stored as "NY".

If I run an INSERT statement in cqlsh using an abbreviation to start with (INSERT INTO us_address (id, street1, street2, city, state, zipcode) VALUES (blobAsUuid(timeuuidAsBlob(now())), '350 Fifth Avenue', '', 'New York', 'NY', '10118');), it still works as shown in the output shown below.

In my example, an invalid state does not prevent an INSERT from occurring, but instead persists the state as "UK" (for unknown) [see the implementation of this in UnitedStatesState.getStateAbbreviationAsByteBuffer(String)].

One of the first advantages that comes to mind justifying why one might want to implement a custom CQL datatype in Java is the ability to employ behavior similar to that provided by check constraints in relational databases. For example, in this post, my sample ensured that any state column entered for a new row was either one of the fifty states of the United States, the District of Columbia, or "UK" for unknown. No other values can be inserted into that column's value.

Another advantage of the custom data type is the ability to massage the data into a preferred form. In this example, I changed every state name to an uppercase two-digit abbreviation. In other cases, I might want to always store in uppercase or always store in lowercase or map finite sets of strings to numeric values. The custom CQL datatype allows for customized validation and representation of values in the Cassandra database.

Conclusion

This post has been an introductory look at implementing custom CQL datatypes in Cassandra. As I play with this concept more and try different things out, I hope to write another blog post on some more subtle observations that I make. As this post shows, it is fairly easy to write and use a custom CQL datatype, especially for Java developers.

Friday, June 27, 2014

Next Generation Project Valhalla Proposed

Earlier this week, Brian Goetz proposed Project Valhalla on an OpenJDK mailing list. Goetz's proposal states:

In accordance with the OpenJDK guidelines, this project will provide a venue to explore and incubate advanced Java VM and Language feature candidates such as Value Types, Generic Specialization, enhanced volatiles (and possibly other related topics, such as reified generics.)

Note that Goetz's proposal should not be confused with the 1997 version of Oracle's Project Valhalla that is described as "the code name for Oracle's flexible Java development environment for building, debugging and deploying component-based applications for the network computing platform." This "other" Project Valhalla is also described in the 15 December 1997 edition of InfoWorld: "Code-named Project Valhalla, the Oracle AppBuilder for Java is slated to ship in the first quarter of 1998. The Java tool, which is based on Borland's JBuilder Java IDE, is designed for writing n-tier, thin client applications."

The "other" Project Valhalla became AppBuilder for Java and was included with JDeveloper Suite (15 April 1998). The Oracle Application Server (4.0 at the time) that was part of this same JDeveloper Suite has new company at Oracle with Oracle's acquiring WebLogic with the BEA acquisition and acquiring GlassFish with the Sun acquisition and JDeveloper is an Oracle-provided (free in the "free beer" sense, but not open source) Java IDE.

Returning to the "next generation" of Project Valhalla, the voting on this Goetz proposal closes on 7 July 2014. Several have already expressed "yes" in replies to Goetz's original e-mail post. In that forum, Patrick Wright asked, "how is this/will this be different from the Da Vinci Project/MLVM?" and John Rose answered that "the charter for Da Vinci is to incubate JVM features for languages other than Java ... or for general language support without associated Java language changes" while "Valhalla is intended to support the evolution of Java itself."

I was excited about Project Coin when it was announced and enjoyed reading and hearing about its progress, but Project Valhalla would be much more ambitious and much more exciting if it is investigated and some or all of the proposed features are implemented.

Wednesday, June 25, 2014

Book Review: Penetration Testing with the Bash Shell

This post is my review of the Packt Publishing book Penetration Testing with the Bash shell by Keith Makan. I think it's important to emphasize its subtitle: "Make the most of the Bash shell and Kali Linux's command-line-based security assessment tools." As the main title and sub-title suggest, this book is about penetration testing with Bash using the Kali Linux. The book has approximately 125 pages covering 5 chapters.

The Preface of Penetration Testing with the Bash shell begins with an articulation of why it is important for penetration testers to be familiar with command-line tools. The Preface also provides brief summaries of each of the five chapters in the book and some of the command-line tools covered in each chapter.

Under the "What you need for this book" section of Penetration Testing with the Bash shell's Preface, Makan writes, "The only software requirement for this book is the Kali Linux operating system, which you can download in the ISO format from http://www.kali.org." You don't necessarily need to use Kali Linux as many of the bash examples and described command-line tools are also available with or for other Linux distributions. The "Who this book is for" section of the Preface states, "Command line hacking is a book for anyone interested in learning how to wield their Kali Linux command lines to perform effective penetration testing."

Chapter 1: Getting to Know Bash

The initial chapter explains that "throughout the book, the bash environment or the host operating system that will be discussed will be Kali Linux" and that "Kali Linux is a distribution adapted from Debian." The chapter then goes onto explain with text and examples Bash concepts such as man pages and bash commands such cd, ls, pwd, and find. The chapter also describes using Linux redirection (output and input) and pipes before concluding with a discussion of grep and regular expressions. As the author suggests, this chapter could be generally be skipped by developers familiar with Linux, but would be must-read information for those new to Linux. Although Kali Linux is used for the examples, I didn't see anything in this initial chapter that seemed specific to Kali Linux.

Chapter 2: Customizing Your Shell

The second chapter of Penetration Testing with the Bash shell begins with coverage of terminal control sequences to change appearance of text in the terminal. The chapter builds upon this information to demonstrate customization of the terminal prompt. Chapter 2 also introduces aliases, customizing command history, and customizing tab completion. Like the first chapter, this second chapter covers additional general Bash concepts rather than specifics of penetration testing.

Chapter 3: Network Reconnaissance

Chapter 3 of Penetration Testing with the Bash shell transitions from coverage of Linux and bash to "move on to using the shell and the Kali Linux command-line utilities to collect information about the networks you find yourself in." The chapter examines tools such as whois, dig, dnsmap, Nmap, and arping.

Chapter 4: Exploitation and Reverse Engineering

Penetration Testing with the Bash shell's fourth chapter focuses on reverse engineering and tools that "may enable you to discover memory corruption, code injection, and general data- or file-handling flaws that may be used to instantiate arbitrary code execution vulnerabilities." Tools covered in this chapter include Metasploit (including command-line msfcli, mixing msfcli with bash commands and other command line tools, msfpayload, Meterpreter), objdump, and GDB.

Chapter 5: Network Exploitation and Monitoring

The fifth and final chapter of Penetration Testing with the Bash shell focuses "on the network exploitation available in Kali Linux and how to take advantage of it in the modern bash shell environment." This relatively longer chapter begins with discussion of "Media Access Control (MAC) spoofing" and "Address Resolution Protocol (ARP) abuse." The chapter looks at tools such as macchangeer, ifconfig, arpspoof.

Chapter 5 describes man-in-the-middle (MITM) attacks and provides a detailed introduction to ettercap. The chapter also explains server interrogation and describes tools for Simple Network Management Protocol (SNMP) interrogation (snmpwalk and Metasploit's snmp_enum and snmp_login) and for Simple Mail Transfer Protocol (SMTP) interrogation (smtp-user-enum).

There is a section in Chapter 5 on brute force authentication that demonstrates use of Medusa. The section of Chapter 5 on traffic filtering demonstrates use of TCPDump. SSLyze is demonstrated as a tool for "assess[ing] SSL/TLS implementations" and SkipFish and Arachni are demonstrated as tools for scanning web pages and web applications for vulnerabilities.

General Observations
  • The electronic version of Penetration Testing with the Bash shell that I reviewed included numerous colored screen snapshots.
  • I liked the "Further Reading" sections at the end of each chapter that provided links to online information with more detail about subjects covered in the chapter.
  • The first two chapters of Penetration Testing with the Bash shell have very little information specific to penetration testing but provide an overview of some bash features used in later chapters. The third, fourth, and fifth chapters are heavily focused on penetration testing and build upon the bash basics covered in the first two chapters.
  • Penetration Testing with the Bash shell provides background information on various security and assessment concepts as its illustrates the tools available that are related to those concepts.
  • There are some awkward sentence structures in Penetration Testing with the Bash shell and several of the sentences are too long for my taste (especially early in the book), but I was generally able to make out the intent of the writing. An example of the occasional awkward but understandable text is on the first page of the first chapter: "Why are discussing the bash shell?"
  • Although this book does mention Kali Linux specifically and frequently, the majority of the described tools are available as built-in tools or as tools that can be installed separately with other flavors of Linux. With no or just a a bit of effort, one could run the examples in different Linux implementations. It's also worth pointing out that the author repeatedly mentions that several commands runs with no special effort in Kali Linux because things are run as root. In other Linux implementations, you may need to use sudo to apply these tools.
  • Although I am fairly familiar with bash, I still learned a few new things from the first two chapters (primarily the second chapter). I am far less experienced with penetration testing and learned quite a bit from the other three chapters.
Conclusion

Penetration Testing with the Bash shell: Make the most of the Bash shell and Kali Linux's command-line-based security assessment tools outlines how bash in general and Kali Linux in particular provide command-line security assessment tools. The book introduces the tools and how to apply them and explains security-related concepts along the way.