Adopt OpenJDK & Java community: how can you help Java !

Introduction

I want to take the opportunity to show what we have been doing in last year and also what we have done so far as members of the community. Unlike other years I have decided to keep this post less technical compare to the past years and compared to the other posts on Java Advent this year.

InTheBeginning

This year marks the fourth year since the first OpenJDK hackday was held in London (supported by LJC and its members) and also when the Adopt OpenJDK program was started. Four years is a small number on the face of 20 years of Java, same goes to the size of the Adopt OpenJDK community which forms a small part of the Java community (9+ million users). Although the post is non-technical in nature, the message herein is fairly important for the future growth and progress of our community and the next generation developers.

Creations of the community

Creations from the community

Over the many months a number of members of our community contributed and passed on their good work to us. In no specific order I have enlisted these picking them from memory. I know there are more to name and you can help us by sharing those with us (we will enlist them here).  So here are some of those that we can talk about and be proud of, and thank those who were involved:

  • Getting Started page – created to enabled two way communication with the members of the community, these include a mailing list, an IRC channel, a weekly newsletter, a twitter handle, among other social media channels and collaboration tools.
  • Adopt OpenJDK project: jitwatch – a great tool created by Chris Newland, its one of its kind, ever growing with features and helping developers fine-tune the performance of your Java/JVM applications running on the JVM.
  • Adopt OpenJDK: GSK – a community effort gathering knowledge and experience from hackday attendees and OpenJDK developers on how to go about with OpenJDK from building it to creating your own version of the JDK. Many JUG members have been involved in the process, and this is now a e-book available in many languages (5 languages + 2 to 3 more languages in progress).
  • Adopt OpenJDK vagrant scripts – a collection of vagrant scripts initially created by John Patrick from the LJC, later improved by the community members by adding more scripts and refactoring existing ones. Theses scripts help build OpenJDK projects in a virtualised container i.e. VirtualBox, making building, and testing OpenJDK and also running and testing Java/JVM applications much easier, reliable and in an isolated environment.
  • Adopt OpenJDK docker scripts – a collection of docker scripts created with the help of the community, this is now also receiving contributions from a number of members like Richard Kolb (SA JUG). Just like the vagrant scripts mentioned above, the docker scripts have similar goals, and need your DevOps foo!
  • Adopt OpenJDK project: mjprof – mjprof is a Monadic jstack analysis tool set. It is a fancy way to say it analyzes jstack output using a series of simple composable building blocks (monads). Many thanks to Haim Yadid for donating it to the community.
  • Adopt OpenJDK project: jcountdown – built by the community that mimics the spirit of ie6countdown.net. That is, to encourage users to move to the latest and greatest Java! Many thanks to all those involved, you can already see from the commit history.
  • Adopt OpenJDK CloudBees Build Farm – thanks to the folks at CloudBees for helping us host our build farm on their CI/CD servers. This one was initially started by Martijn Verburg and later with the help of a number of JUG members have come to the point that major Java projects are built against different versions of the JDK. These projects include building the JDKs themselves (versions 1.7, 1.8, 1.9, Jigsaw and Shenandoah). This project has also helped support the Testing Java Early project and Quality  Outreach program.

These are just a handful of such creations and contributions from the members of the community, some of these projects would certainly need help from you. As a community one more thing we could do well is celebrate our victories and successes, and especially credit those that have been involved whether as individuals or a community. So that our next generation contributors feel inspired and encourage to do more good work and share it with us.

Contributions from the community

We want to contribute

In a recent tweet and posts to various Java / JVM and developer mailing lists, I requested the community to come forward and share their contribution stories or those from others with our community. The purpose was two-fold, one to share it with the community and the other to write this post (which in turn is shared with the community). I was happy to see a handful of messages sent to me and the mailing lists by a number of community members. I’ll share some of these with you (in the order I have received them).

Sebastian Daschner:

I don’t know if that counts as contribution but I’ve hacked on the
OpenJDK compiler for fun several times. For example I added a new
thought up ‘maybe’ keyword which produces randomly executed code:
https://blog.sebastian-daschner.com/entries/maybe_keyword_in_java

Thomas Modeneis:

Thanks for writing, I like your initiative, its really good to show how people are doing and what they have been focusing on. Great idea.
From my part, I can tell about the DevoxxMA last month, I did a talk on the Hacker Space about the Adopt the OpenJDK and it was really great. We had about 30 or more attendees, it was in a open space so everyone that was going to any talk was passing and being grabbed to have a look about the topic, it was really challenging because I had no mic. but I managed to speak out loud and be listen, and I got great feedback after the session. I’m going to work over the weekend to upload the presentation and the recorded video and I will be posting here as soon as I have it done! 🙂

Martijn Verburg:

Good initiative.  So the major items I participated in were Date and Time and Lambdas Hackdays (reporting several bugs), submitted some warnings cleanups for OpenJDK.  Gave ~10 pages of feedback for jshell and generally tried to encourage people more capable than me to contribute :-).

Andrii Rodionov:

Olena Syrota and Oleg Tsal-Tsalko from Ukraine JUG: Contributing to JSR 367 test code-base (https://github.com/olegts/jsonb-spec), promoting ‘Adopt a JSR’ and JSON-B spec at JUG UA meetings (http://jug.ua/2015/04/json-binding/) and also at JavaDay Lviv conference (http://www.slideshare.net/olegtsaltsalko9/jsonb-spec).

Contributors

Contributors gathering together

As you have seen that from out of a community of 9+ million users, only a handful of them came forward to share their stories. While I can point you out to another list of contributors who have been paramount with their contributions to the Adopt OpenJDK GitBook, for example, take a look at the list of contributors and also the committers on the git-repo. They have not just contributed to the book but to Java and the OpenJDK community, especially those who have helped translate the book into multiple languages. And then there are a number of them who haven’t come forward to add their names to the list, even though they have made valuable contributions.
Super heroes together

From this I can say contributors can be like unsung heroes, either due their shy or low-profile nature or they just don’t get noticed by us. So it would only be fair to encourage them to come forward or share with the community about their contributions, however simple or small those may be. In addition to the above list I would like to also add a number of them (again apologies if I have missed out your name or not mentioned about you or all your contributions). These names are in no particular order but as they come to my mind as their contributions have been invaluable:

  • Dalibor Topic (OpenJDK Project Lead) & the OpenJDK team
  • Mario Torre & the RedHat OpenJDK team
  • Tori Wieldt (Java Community manager) and her team
  • Heather Vancura & the JCP team
  • NightHacking, vJUG and RebelLabs (and the great people behind them)
  • Nicolaas & the team at Cloudbees
  • Chris Newland (JitWatch developer)
  • Lucy Carey, Ellie & Mark Hazell (Devoxx UK & Voxxed)
  • Richard Kolb (JUG South Africa)
  • Daniel Bryant, Richard Warburton, Ben Evans, and a number of others from LJC
  • Members of SouJava (Otavio, Thomas, Bruno, and others)
  • Members of Bulgarian JUG (Ivan, Martin, Mitri) and neighbours
  • Oti, Ludovic & Patrick Reinhart
  • and a number of other contributors who for some reason I can’t remember…

I have named them for their contributions to the community by helping organise Hackdays during the week and weekends, workshops and hands-on sessions at conferences, giving lightening talks, speaking at conferences, allowing us to host our CI and build farm servers, travelling to different parts of the world holding the Java community flag, writing books, giving Java and advance-level training, giving feedback on new technologies and features, and innumerable other activities that support and push forward the Java / JVM platform.

How you can make a difference ? And why ?

Make a difference

You can make a difference by doing something as simple as clicking the like button (on Twitter, LinkedIn, Facebook, etc…) or responding to a message on a mailing list by expressing your opinion about something you see or read about –as to why you think about it that way or how it could be different.

The answer to the question “And why ?” is simple, because you are part of a community and ‘you care’ and want to share your knowledge and experience with others — just like the others above who have spared free moments of their valuable time for us.

Is it hard to do it ? Where to start ? What needs most attention ?

important-checklist The answer is its not hard to do it, if so many have done it, you can do it as well. Where to start and what can you do ? I have written a page on this topic. And its worth reading it before going any further.

There is a dynamic list of topics that is worth considering when thinking of contributing to OpenJDK and Java. But recently I have filtered this list down to a few topics (in order of precedence):

We need you!

With that I would like to close by saying:

i_need_you_duke3

Not just “I”, but we as a community need you.

This post is part of the Java Advent Calendar and is licensed under the Creative Commons 3.0 Attribution license. If you like it, please spread the word by sharing, tweeting, FB, G+ and so on!

Project Jigsaw Hands-On Guide

Project Jigsaw will bring modularization to the Java platform and according to the original plan it was going to be feature complete on the 10th of December. So here we are but where is Jigsaw?

Surely a lot happened in the last six months: The prototype came out, the looming removal of internal APIs caused quite a ruckus, the mailing list is full of critical discussions about the project’s design decisions, and JavaOne saw a series of great introductory talks by the Jigsaw team. And then Java 9 got delayed for half year due to Jigsaw.

But let’s ignore all of that for now and just focus on the code. In this post we’ll take an existing demo application and modularize it with Java 9. If you want to follow along, head over to GitHub, where all of the code can be found. The setup instructions are important to get the scripts running with Java 9. For brevity, I removed the prefix org.codefx.demo from all package, module, and folder names an this article.

The Application Before Jigsaw

Even though I do my best to ignore the whole Christmas kerfuffle, it seemed prudent to have the demo uphold the spirit of the season. So it models an advent calendar:

  • There is a calendar, which has 24 calendar sheets.
  • Each sheet knows its day of the month and contains a surprise.
  • The death march towards Christmas is symbolized by printing the sheets (and thus the surprises) to the console.

Of course the calendar needs to be created first. It can do that by itself but it needs a way to create surprises. To this end it gets handed a list of surprise factories. This is what the main method looks like:

public static void main(String[] args) {
    List<SurpriseFactory> surpriseFactories = Arrays.asList(
            new ChocolateFactory(),
            new QuoteFactory()
    );
    Calendar calendar =
        Calendar.createWithSurprises(surpriseFactories);
    System.out.println(calendar.asText());
}

The initial state of the project is by no means the best of what is possible before Jigsaw. Quite the contrary, it is a simplistic starting point. It consists of a single module (in the abstract sense, not the Jigsaw interpretation) that contains all required types:

  • “Surprise API” – Surprise and SurpriseFactory (both are interfaces)
  • “Calendar API” – Calendar and CalendarSheet to create the calendar
  • Surprises – a couple of Surprise and SurpriseFactory implementations
  • Main – to wire up and run the whole thing.

Compiling and running is straight forward (commands for Java 8):

# compile
javac -d classes/advent ${source files}
# package
jar -cfm jars/advent.jar ${manifest and compiled class files}
# run
java -jar jars/advent.jar

Entering Jigsaw Land

The next step is small but important. It changes nothing about the code or its organization but moves it into a Jigsaw module.

Modules

So what’s a module? To quote the highly recommended State of the Module System:

A module is a named, self-describing collection of code and data. Its code is organized as a set of packages containing types, i.e., Java classes and interfaces; its data includes resources and other kinds of static information.

To control how its code refers to types in other modules, a module declares which other modules it requires in order to be compiled and run. To control how code in other modules refers to types in its packages, a module declares which of those packages it exports.

So compared to a JAR a module has a name that is recognized by the JVM, declares which other modules it depends on and defines which packages are part of its public API.

Name

A module’s name can be arbitrary. But to ensure uniqueness it is recommended to stick with the inverse-URL naming schema of packages. So while this is not necessary it will often mean that the module name is a prefix of the packages it contains.

Dependencies

A module lists the other modules it depends on to compile and run. This is true for application and library modules but also for modules in the JDK itself, which was split up into about 80 of them (have a look at them with java -listmods).

Again from the design overview:

When one module depends directly upon another in the module graph then code in the first module will be able to refer to types in the second module. We therefore say that the first module reads the second or, equivalently, that the second module is readable by the first.

[…]

The module system ensures that every dependence is fulfilled by precisely one other module, that no two modules read each other, that every module reads at most one module defining a given package, and that modules defining identically-named packages do not interfere with each other.

When any of the properties is violated, the module system refuses to compile or launch the code. This is an immense improvement over the brittle classpath, where e.g. missing JARs would only be discovered at runtime, crashing the application.

It is also worth to point out that a module is only able to access another’s types if it directly depends on it. So if A depends on B, which depends on C, then A is unable to access C unless it requires it explicitly.

Exports

A module lists the packages it exports. Only public types in these packages are accessible from outside the module.

This means that public is no longer really public. A public type in a non-exported package is as hidden from the outside world as much as a non-public type in an exported package. Which is even more hidden than package-private types are today because the module system does not even allow reflective access to them. As Jigsaw is currently implemented command line flags are the only way around this.

Implementation

To be able to create a module, the project needs a module-info.java in its root source directory:

module advent {
    // no imports or exports
}

Wait, didn’t I say that we have to declare dependencies on JDK modules as well? So why didn’t we mention anything here? All Java code requires Object and that class, as well as the few others the demo uses, are part of the module java.base. So literally every Java module depends on java.base, which led the Jigsaw team to the decision to automatically require it. So we do not have to mention it explicitly.

The biggest change is the script to compile and run (commands for Java 9):

# compile (include module-info.java)
javac -d classes/advent ${source files}
# package (add module-info.class and specify main class)
jar -c \
    --file=mods/advent.jar \
    --main-class=advent.Main \
    ${compiled class files}
# run (specify a module path and simply name to module to run)
java -mp mods -m advent

We can see that compilation is almost the same – we only need to include the new module-info.java in the list of classes.

The jar command will create a so-called modular JAR, i.e. a JAR that contains a module. Unlike before we need no manifest anymore but can specify the main class directly. Note how the JAR is created in the directory mods.

Utterly different is the way the application is started. The idea is to tell Java where to find the application modules (with -mp mods, this is called the module path) and which module we would like to launch (with -m advent).

Splitting Into Modules

Now it’s time to really get to know Jigsaw and split that monolith up into separate modules.

Made-up Rationale

The “surprise API”, i.e. Surprise and SurpriseFactory, is a great success and we want to separate it from the monolith.

The factories that create the surprises turn out to be very dynamic. A lot of work is being done here, they change frequently and which factories are used differs from release to release. So we want to isolate them.

At the same time we plan to create a large Christmas application of which the calendar is only one part. So we’d like to have a separate module for that as well.

We end up with these modules:

  • surpriseSurprise and SurpriseFactory
  • calendar – the calendar, which uses the surprise API
  • factories – the SurpriseFactory implementations
  • main – the original application, now hollowed out to the class Main

Looking at their dependencies we see that surprise depends on no other module. Both calendar and factories make use of its types so they must depend on it. Finally, main uses the factories to create the calendar so it depends on both.

jigsaw-hands-on-splitting-into-modules

Implementation

The first step is to reorganize the source code. We’ll stick with the directory structure as proposed by the official quick start guide and have all of our modules in their own folders below src:

src
  - advent.calendar: the "calendar" module
      - org ...
      module-info.java
  - advent.factories: the "factories" module
      - org ...
      module-info.java
  - advent.surprise: the "surprise" module
      - org ...
      module-info.java
  - advent: the "main" module
      - org ...
      module-info.java
.gitignore
compileAndRun.sh
LICENSE
README

To keep this readable I truncated the folders below org. What’s missing are the packages and eventually the source files for each module. See it on GitHub in its full glory.

Let’s now see what those module infos have to contain and how we can compile and run the application.

surprise

There are no required clauses as surprise has no dependencies. (Except for java.base, which is always implicitly required.) It exports the package advent.surprise because that contains the two classes Surprise and SurpriseFactory.

So the module-info.java looks as follows:

module advent.surprise {
    // requires no other modules
    // publicly accessible packages
    exports advent.surprise;
}

Compiling and packaging is very similar to the previous section. It is in fact even easier because surprises contains no main class:

# compile
javac -d classes/advent.surprise ${source files}
# package
jar -c --file=mods/advent.surprise.jar ${compiled class files}

calendar

The calendar uses types from the surprise API so the module must depend on surprise. Adding requires advent.surprise to the module achieves this.

The module’s API consists of the class Calendar. For it to be publicly accessible the containing package advent.calendar must be exported. Note that CalendarSheet, private to the same package, will not be visible outside the module.

But there is an additional twist: We just made Calendar.createWithSurprises(List<SurpriseFactory>) publicly available, which exposes types from the surprise module. So unless modules reading calendar also require surprise, Jigsaw will prevent them from accessing these types, which would lead to compile and runtime errors.

Marking the requires clause as public fixes this. With it any module that depends on calendar also reads surprise. This is called implied readability.

The final module-info looks as follows:

module advent.calendar {
    // required modules
    requires public advent.surprise;
    // publicly accessible packages
    exports advent.calendar;
}

Compilation is almost like before but the dependency on surprise must of course be reflected here. For that it suffices to point the compiler to the directory mods as it contains the required module:

# compile (point to folder with required modules)
javac -mp mods \
    -d classes/advent.calendar \
    ${source files}
# package
jar -c \
    --file=mods/advent.calendar.jar \
    ${compiled class files}

factories

The factories implement SurpriseFactory so this module must depend on surprise. And since they return instances of Surprise from published methods the same line of thought as above leads to a requires public clause.

The factories can be found in the package advent.factories so that must be exported. Note that the public class AbstractSurpriseFactory, which is found in another package, is not accessible outside this module.

So we get:

module advent.factories {
    // required modules
    requires public advent.surprise;
    // publicly accessible packages
    exports advent.factories;
}

Compilation and packaging is analog to calendar.

main

Our application requires the two modules calendar and factories to compile and run. It has no API to export.

module advent {
    // required modules
    requires advent.calendar;
    requires advent.factories;
    // no exports
}

Compiling and packaging is like with last section’s single module except that the compiler needs to know where to look for the required modules:

#compile
javac -mp mods \
    -d classes/advent \
    ${source files}
# package
jar -c \
    --file=mods/advent.jar \
    --main-class=advent.Main \
    ${compiled class files}
# run
java -mp mods -m advent

Services

Jigsaw enables loose coupling by implementing the service locator pattern, where the module system itself acts as the locator. Let’s see how that goes.

Made-up Rationale

Somebody recently read a blog post about how cool loose coupling is. Then she looked at our code from above and complained about the tight relationship between main and factories. Why would main even know factories?

Because…

public static void main(String[] args) {
    List<SurpriseFactory> surpriseFactories = Arrays.asList(
            new ChocolateFactory(),
            new QuoteFactory()
    );
    Calendar calendar =
        Calendar.createWithSurprises(surpriseFactories);
    System.out.println(calendar.asText());
}

Really? Just to instantiate some implementations of a perfectly fine abstraction (the SurpriseFactory)?

And we know she’s right. Having someone else provide us with the implementations would remove the direct dependency. Even better, if said middleman would be able to find all implementations on the module path, the calendar’s surprises could easily be configured by adding or removing modules before launching.

This is indeed possible with Jigsaw. We can have a module specify that it provides implementations of an interface. Another module can express that it uses said interface and find all implementations with the ServiceLocator.

We use this opportunity to split factories into chocolate and quote and end up with these modules and dependencies:

  • surpriseSurprise and SurpriseFactory
  • calendar – the calendar, which uses the surprise API
  • chocolate – the ChocolateFactory as a service
  • quote – the QuoteFactory as a service
  • main – the application; no longer requires individual factories

jigsaw-hands-on-services

Implementation

The first step is to reorganize the source code. The only change from before is that src/advent.factories is replaced by src/advent.factory.chocolate and src/advent.factory.quote.

Lets look at the individual modules.

surprise and calendar

Both are unchanged.

chocolate and quote

Both modules are identical except for some names. Let’s look at chocolate because it’s more yummy.

As before with factories the module requires public the surprise module.

More interesting are its exports. It provides an implementation of SurpriseFactory, namely ChocolateFactory, which is specified as follows:

provides advent.surprise.SurpriseFactory
    with advent.factory.chocolate.ChocolateFactory;

Since this class is the entirety of its public API it does not need to export anything else. Hence no other export clause is necessary.

We end up with:

module advent.factory.chocolate {
    // list the required modules
    requires public advent.surprise;
    // specify which class provides which service
    provides advent.surprise.SurpriseFactory
        with advent.factory.chocolate.ChocolateFactory;
}

Compilation and packaging is straight forward:

javac -mp mods \
    -d classes/advent.factory.chocolate \
    ${source files}
jar -c \
    --file mods/advent.factory.chocolate.jar \
    ${compiled class files}

main

The most interesting part about main is how it uses the ServiceLocator to find implementation of SurpriseFactory. From its main method:

List surpriseFactories = new ArrayList<>();
ServiceLoader.load(SurpriseFactory.class)
    .forEach(surpriseFactories::add);

Our application now only requires calendar but must specify that it uses SurpriseFactory. It has no API to export.

module advent {
    // list the required modules
    requires advent.calendar;
    // list the used services
    uses advent.surprise.SurpriseFactory;
    // exports no functionality
}

Compilation and execution are like before.

And we can indeed change the surprises the calendar will eventually contain by simply removing one of the factory modules from the module path. Neat!

Summary

So that’s it. We have seen how to move a monolithic application into a single module and how we can split it up into several. We even used a service locator to decouple our application from concrete implementations of services. All of this is on GitHub so check it out to see more code!

But there is lots more to talk about! Jigsaw brings a couple of incompatibilities but also the means to solve many of them. And we haven’t talked about how reflection interacts with the module system and how to migrate external dependencies.

If these topics interest you, watch the Jigsaw tag on my blog as I will surely write about them over the coming months.