Crazy Coding Conundrums

I’m less than pleased with ruby and python so far (tried RoR and django). Yet, in pure language syntax both are miles ahead of java. It’s the enforced type-lessness of it all which really bothers me, and the reason it does so is because I keep having to resort to external documentation to figure out the proper names for all sorts of things, instead of having my IDE just pop up my choices as it occurs in eclipse or netbeans (java).

Learning to use new toolkits is tons easier with such popups, and so far I am still not convinced that you can just throw compile-time checks out the window as long as you have proper unit tests. Given that any serious project spends as much time, if not more time, debugging compared to creating, any move that makes debugger more difficult has got to be a bad one. There’s also the implicit compile-time checked documentation. These beat comments hands down, simply because comments are both difficult to machine read, and they can get out of date.

I’m really looking for a best-of-both-worlds language. Something with static typing (but with an option to escape it when it is unwieldy) yet with the language improvements of python and ruby.

A new language which compiles to Java bytecode and is such as directly ‘pluggable’ into any java system has shown up, and it’s called groovy. Java 1.6 (Mustang) will even support it out of the box. In many ways its trying to be this panacea language.

I’ll first explain what groovy is trying to do, then I’ll dive into some improvements that groovy doesn’t have yet.

Remember first that java is a lot of things: It’s a language (which groovy tries to make better), but also a runtime library, a distribution model, a virtual machine, and a security model. Groovy just tries to fix the language it self, but in my opinion all the other stuff is pretty good already.

Let’s start with a simple custom sort example. This example shows function objects, as well as some basic collections API (creating and manipulating lists of data).

python:

def reverseNumeric(a, b):
    return b-a
a = [5, 1, 3, 2]
a.sort(reverseNumeric)
print a

JAVA5:

Comparator reverseNumeric = new Comparator<Integer>() {
    public int compare(Integer a, Integer b) {
        return b-a;
    }
};
List<Integer> a = Arrays.asList(5, 1, 3, 2);
Collections.sort(a, reverseNumeric);
System.out.println(a);

These little fragments of code do the exact same thing, and in the exact same way. I think it highlights exactly what’s wrong with both python and java. First things first: notice how ‘reverseNumeric’ is a function object in both examples. Suggestions that java doesn’t support function objects are misleading; java doesn’t support functions as direct objects, but you can declare new classes on the fly, which amounts to the same thing. The syntax for this is ridicuously wordy, but we’ll get to that.

What’s wrong with the java code: Arrays are supported ‘natively’ but lists and maps aren’t, leading to ugly hacks like the new Arrays.asList method. There is of course the wordy function object. A third problem arises in java’s strictness – mixin functionality is difficult to pull in java, so we have a Collections.sort, which is a bad thing – it should have been a.sort(reverseNumeric), somehow. I sort of understand why sun hasn’t done it that way, but this does indicate a clear weakness of the language. The fact that it was difficult to implement sort into all relevant collection classes is not good.

The python problem is much easier to identify: It’s all dynamic. There’s less namespacing going on (module members have a namespace in java, ie Gun.shoot() is something else compared to Camera.shoot(), assuming Gun and Camera are classes or interfaces, whereas in python that’s not important and there’s just shoot(). I take it the wording of the method and classes illustrates why this can sometimes lead to shooting yourself in the foot). Also, the ubiquitous autocomplete of modern java IDEs such as eclipse and netbeans, which lists all possible parameter choices/method names/whatever applies cannot, by definition, work nearly as well in a dynamic language. This is problematic in large projects, and doubly so if everyone mixes and matches camel case and underscores, like for example python.

Fixing the java code is mostly purely a language problem though. The underlying JVM, basics, or static nature aren’t to blame. Here’s my first attempt to fix the java fragment. Let’s call it wishful java:

Comparator<Integer> reverseNumeric =
   new Comparator(Integer a, Integer b) {
    return b - a;
};
List<Integer> a = [5, 1, 3, 2];
Collections.sort(a, reverseNumeric);  //still ugly
System.out.println(a);

It’s trivial to write a program that compiles this example to the previous fragment. yet this code is much, much cleaner. It’s still more wordy, but this time it’s not overly wordy. The extra words spent (mostly in naming the List as explicitly being both a List and of type Integer, and same story for the comparator) give us something in return which is usually worth the trouble: compiler-checkable certainty. Eclipse or netbeans users know why it’s so damn handy to make sure your compiler/IDE can know exactly what parameters you want.

We’ve added a native syntax for creating lists. We’ve also made it possible to implement single-method interfaces such as Comparator in one swoop. this ‘works’ because there is only one method defined by the java.util.Comparator interface, namely the compare method. If there were more, this kind of syntax couldn’t work, but then again, if there were more, a simple function object can’t get the job done anyway. Note that if the <Integer> markers bother you, you could allow an aggresive type inference strategy into the language, where lack of explicit generic types means they get inferred (from the parameter types in the first case, and from the contents of the list literal in the second case) if possible, but overriding is still possible.

This new example still has the power of java. For example, if we explicitly make our list, semantically, a list of objects:

List<Object> a = [5, 1, 3, 2];

We get a compile-time (read: development time, you get an instant red underline if you try, assuming you use a proper IDE) error, because our comparator is not set up to deal with comparing objects, only integers. We’d have to either make the list integer-only, or adjust our comparator to be capable of handling any objects. This is good – it allows you to catch bugs early, and it adds implicit documentation. In other words, for the same reason that naming your variable ‘userName’ instead of ‘a’ is generally a good idea, so is explicitly documenting that a list is restricted to certain types. This beats comments any day, because comments get out of date. Compile-time checked restrictions do not, because the moment they fall out of date (ie: because we allow our list to contain any object instead of just integers), we get compile-time errors.

While so far we’ve got a pretty nice language, our first draft of wishful java has two remaining problems. First, there’s still the ugly Collections.sort. We’ll try to solve that in a later post. The second problem is that, while usually all that explicit typing is very useful, sometimes it’s superfluous. Think, for example, of unit tests. Unit tests are never part of an API and never will be, the code tends to be very trivial, and it needs to be quickly written. Entering in all the type information may just be too much work. Also, when writing your unit tests before your actual code, as is suggested by most experts on the subject, it can be useful to not explicitly type stuff – you’ll figure out such details later.

We could solve this typing problem in our wishful java, actually. Here’s the second draft of wishful java, in case this code occurs in a place where we don’t want or need explicit typing:

reverseNumeric = function compare(a, b) {
    return b - a;
};
a = [5, 1, 3, 2];
Collections.sort(a, reverseNumeric);  //still ugly
System.out.println(a);

We could go even further and eliminate the ‘compare’ string, but as the java runtime library is still strongly and strictly typed, we could only use such unnamed function objects within our own code. Just adding the ‘compare’ text is no big deal, though, I’d say. The reason this can still be compiled to either ‘java’ or JVM bytecode is due to the reflections API.

Turns out all this ‘wishful’ java is very much like Groovy. Groovy does almost exactly what I suggested earlier: It compiles groovy to ordinary java bytecode, which can freely interact with plain old vanilla java5 code. Just like our two examples of wishful java, in groovy you can pick – you can type stuff, or you don’t.

Groovy also goes on with the nice syntactic sugar, beyond what ruby and python have to offer in some cases. For example, groovy supports native regexps. These generate compile-time errors if the regexps are wrong. Valid groovy:

Pattern p = /(foo)?[bar]/;

The nice thing about this is that in theory it’s faster in the extremely unlikely event speed matters, but mostly the syntax is more obvious (// always means a regular expression) and you get a compile-time/development-time (if your IDE supports it) error when there is a syntax error in your regexp. Again, the benefit of having the code document itself is very important.

Groovy is new and I expect they’ll be doing more and more nice stuff. Of course, unfortunately, in some ways they’ve gone too far. Groovy makes a big deal out of supporting straight closures, but in practice you can’t use these with any existing java code. It does not (yet?) support the ‘function interface’ syntax introduced in the first example, either. Will Groovy coalesce to the same level of robustness of java, combined with the comfortable programming style of python and ruby? We can only hope so!

---

Extra treatise – feel free to skip:

We’re still stuck with that ugly Collections.sort. The reason why it works this way is somewhat unclear, but let me try to explain anyway: First off, making some sort of feature that automatically calls the (static) Collections.sort method anytime:

a.sort(comparator);

shows up would take either mixins or macros, neither of which are supported by java. As far as macros goes – basically a good thing. If you allow macros like this you’ll quickly degenerate into either C preprocessor hell, or the compiler doesn’t know what’s going on and then you lose all the static typing advantages. Mixins would be much better but so far no one has come up with any serious candidate for syntax.

The basic java to do this would be to add sort() to the List interface, and then let each implementation figure out how to implement sort. Probably just by writing:

public void sort(Comparator comparator) {
    Collections.sort(this, comparator);}

A java1.6 feature is that interfaces can have static methods, which would mean this code could be cleaned up some to:

public void sort(Comparator comparator) {
    sort(this, comparator);}

by moving the sort static method into the List interface itself. Unfortunately, this tactic still has two problems: For one it’s sort of ugly that all implementations have to write explicit code just to implement default behaviour, and secondly, if you imagine for a moment that this sort thing is an upgrade from a previous version, then all third party list implementations just became broken, because they do not have this explicit sort method covered.

Still, sort has been added in java1.2 if not earlier, and by now if I were sun I would have just stuck a sort method into the List interface. Just because it’s a bit awkward to pull of syntactically doesn’t mean that you have to bother users with a godawful Collections.sort syntax.

Here’s a very nice solution: automated delegation. This particular trick is so powerful, it can actually allow an object that implements both Camera and Gun to have two different shoot() methods. The reason it can work is because all calls in java have a parent type.

Current situation:

public interface Camera {public void shoot();}
public interface Gun {public void shoot();}
public class CamGun implements Camera, Gun {
    public void shoot() {
        System.out.println("This can't be good!");
    }
}

In java it’s impossible to do this properly and fortunately at least this is trivially obvious when trying to program our CamGun. I submit that this is a better option compared to silently allowing it, but due to the way java calls are always typed it is in theory possible to figure out which method is intended in some cases. Example:

//Clearly, fire the gun:
Gun camgun = new CamGun(); camgun.shoot();


//Clearly, make a picture:
Camera camgun = new CamGun(); camgun.shoot();

//Don't know what to do:
CamGun camgun = new CamGun(); camgun.shoot();

//Fire the gun:
CamGun camgun = new CamGun(); ((Gun)camgun).shoot();

In all but the third scenario, the intention is obvious. Forcing a programmer to pick a supertype when the intention is unclear, such as in the fourth example, is also acceptable, especially because this situation will in practice not occur all that often anyway.

Any solution to this problem tends to solve the multiple inheritance problem as well – where two implementations of the same method are inherited, and the compiler has to prioritize one over the other, which in practice (see C++) is a recipe for disaster.

In python or ruby, where functions are first class objects, this problem can never be resolved properly. They do not have the notion of an interface or a supertype nearly as clearly as in java, and the notion of casts or types are foreign to the python/ruby class of programming languages regardless.

The only extra actual syntax required to support separate methods is this:

• Allow interface methods to have a method body, which counts as the default implementation
• Allow any method declaration to explicitly specify what it is overriding. There’s already an @Override annotation, which can simply be extended to take 1 optional class parameter

An example of this:

public interface Camera {
    public void shoot() {
        System.out.println("BANG!");
    }
}


public interface Gun {
    public void shoot() {
        System.out.println("CLICK!");
    }
}

public class FlashingCamGun implements Camera, Gun {
    public @Override(Camera) shoot() {
        System.out.println("FLASH!");
        Camera.shoot();  //just 'super' would be unclear.
    }
}

This code does everything we want, neatly sidesteps (in that it generates compiler errors, and fail-first is a good idea for this sort of thing, otherwise you have a hard to trace bug on your hands) any multiple inheritance or namespace overlap issues, yet would make implementing the Collections.sort thing an absolute breeze: Just add a default sort() implementation to java.util.List.

As a bonus we get a language which allows CameraGuns. No mainsteam languages I know of can handle cameraguns. So far cameraguns have just been just as an example where explicit type checking is good because it allows you to detect them. This cameragun is actually implementable.

---

I’ve listed a bunch of other possible java improvements a while ago here.