I posted this on the java posse googlegroup about 5 minutes after Mark Reinhold’s talk on jigsaw at devoxx ’09. I’m consolidating it here on my blog after some requests to do so.
Notes from his presentation at devoxx, 17:40-18:40 Central European Time, November 19th, 2009.
Section 1: Problems and Solutions
(why jigsaw, what is it trying to solve? Subtext: What is it trying to solve that OSGi can’t?)
jigsaw exists because they want to have modules-all-the-way-down. OSGi depends on java classes, but how can you then modularize the core java classes like java.util.List or even java.lang.String with OSGi? So, the module system needs to be done in the JVM core. OSGi comes with baggage such as lifecycle management that isn’t needed for slicing rt.jar into modules.
Secondly, jar files are a fine distro tool, but a crappy runtime tool, so a new package processing system is required. If everyone versions, then we can go back to the model of installing ‘jars’ (whatever they will be) into the system, without running into version conflicts anymore. By changing the format between running and distro, java can verify at install time, do some ahead-of-time compilation, and store JIT profiles straight into the local module base for future executions of the JVM. This is not something OSGi does (or even should do). Also, class files are great for distribution, but they aren’t that great for efficient loading. The java core libraries are what they are. They can’t be changed now. And yet, e.g. java.io.FileInputStream has a method to get a java.nio.FileChannel. If java.io and java.nio are to be different packages, and you don’t want io to be dependent on nio, then what?
Even worse, some non-util packages contribute a class or two to java.util. OSGi is very much anti split packages. If java were to be designed from scratch with modularization in mind, that might be okay, but java wasn’t. Thus, split packages, in-package optional dependencies are all fairly complicated requirements that are needed for slicing rt.jar into modules.
Big point he keeps making: He looked at OSGi to slice rt.jar, decided it just wasn’t going to work, so came up with a new module system that is mostly not intersecting with OSGi’s bailiwick. And then he decided that, as java code itself will have to have the capability of stating that it doesn’t, for example, need swing, or nio, or corba (heh), it makes sense to allow other people to make modules in this as well. Hence, jigsaw.
Section 2: Jigsaw – technical details
Part A: Grouping
A module system needs to group classes and other resources in a single module.
jigsaw solution: module-info.java file, at the top of the relevant source tree (so at ~/module.info if your class is at ~/com/foo/Whatever.class)
Flipside of coin: Need ability to state dependency, better than import foo;
example:
module com.foo {
requires org.bar.lib;
requires edu.baz.util;
}
and with versions:
module com.foo @ 1.0.0 {
requires org.bar.lib @ 2.1-alpha;
requires edu.baz.util @ 5.2_11;
}
Version syntax is now chosen to be non-descriptive at all. Anything goes. Just like apt-get. Awareness of the need to then look up versioning scheme per module, and there WILL be syntaxes like 2.(1+). Preferable to OSGi forced versioning system. Tokenizes on a limited number of characters (dots and dashes, pretty much), and each subsequence can be matched against with wildcards. It seems to work well for debian, and for rpm (their their algo is slightly different), hence confidence this will be fine, and should make for much easier adoption amongst existing projects, such as Oracle v1.9.12.0.0.0.0.0.0.0.0.0.2 (red: I’m channeling Joe here, not Mark)
As there is to be interop between e.g. rpm and debian, anything more rigid will also not translate very well (e.g. a complex wildcard just couldn’t be translated to a deb file!) requires by default flows through to dependents. So if A requires B, and C requires B, then C has access to A’s stuff. Get around via requires private.
You can also get around this FROM A:
module com.foo.secret {
permits com.foo.lib;
}
means that only com.foo.lib can requires it, and can’t export it (forced to say private, or compiler error).
Optional dependencies: requires optional
Of course, your code will need to deal with the ClassNotFoundErrors and such that could result.
virtual modules: Like apt-get’s virtual java package. It’s a bit like a module interface concept, though it’s just a name, there’s no structure in a virtual module. So:
module com.foo.lib {
}
module com.foo.lib.impl.phone {
provides com.foo.lib;
}
If another module then wants com.foo.lib, it’ll get com.foo.lib.impl.phone.
Part B: Interop with other packaging systems, and module distribution.
You can for example create a debian package (For apt-get, so you can install on ubuntu, debian, etc):
jpkg - m classes deb com.foo.app com.foo.lib
creates a .deb file for apt-get from the modules com.foo.app and com.foo.lib. All dependencies are reflected into the debian package, so if com.foo.app requires foo.bar.baz, this is reflected in the apt file’s Requires block.
For windows, with no such package manager, or in any java-centric environment, or even on phones with a stripped-down linux without rpm or apt-get, you just stick with java’s own. There will be a platform independent transfer format (basically v2.0 of jar files. Won’t be zip on the inside, will be opaque so the format can be revved by sun over time, but it’s mostly the same concept). It will have a new file extension.
Part C: Slicing the JDK.
Here’s a chart of dependencies per package in java core library:
Here’s what base is dependent on in JDK6 (so, almost everything):
Here’s what it looks like now in an experimental JDK7. with base dependent on very little, and nearly acyclic:
And even simpler with optional deps removed (this photo you should be able to read):
JIGSAW BINARIES will go live in a few minutes after this talk!
http://openjdk.java.net/projects/jigsaw
http://dl.openjdk.java.net/jigsaw
only binaries for unbuntu jaunty, but you can always build from source (that’s just because Mark is an ubuntu user). It’s all very raw, but it includes the packaging tools, as well as support for module-info.java and the like.
Part D: OSGi interop.
(Not mentioned in main presentation, but someone asked).
Status unknown, but it’s something they’ll look at. Realization that compatibility is rather difficult due to different requirements, such as split-packages. Realization that having 2 competing module formats, forcing lib devvers to make 2 versions. Something Mark is looking at is extending jigsaw libraries to just wrap around an OSGi library. That way, you can choose to just make an OSGi library and ship that.
NB: There’s a BOF coming up on this at 20:00 Central European about the Modular Java Platform and Jigsaw with Mark, so this isn’t the last word.
NB2: Slicing rt.jar is well on its way, but not quite done yet. Still dependencies exist that will be eliminated before JDK7 RC. Also, so far, no measurable performance benefits (yet). However, new installed-module stuff (ahead-of-time implementation, second version of class file format optimized for reading, jit profiles) aren’t live yet, and should bring significant performance benefits.