JDK regression

I wrote about this issue last week and spent a good chunk of this week better understanding the scope of the problem and brainstorming some possible mitigations. Probably the best explanation at this point can be found in this blog from Claes Redestad from Oracle. Ghadi Shayban has been working with Claes to better understand the problem and sync that up with our Clojure world. Big thanks to Claes, David Holmes, and Vladimir Ivanov for everything they’ve done on the ticket. I’d be happy to buy those guys a beer sometime.

In Clojure terms, we’ve verified that this does not affect AOT or genclass classes, and is really primarily isolated to doing a lot of work in user.clj. This is because user.clj is loaded during the static initializer of clojure.lang.RT (the Clojure runtime). Loading makes calls into the Compiler and other things that ultimately call back into the RT static methods (and that should look like the Bad or AlsoBad scenarios in Claes’s blog).

In terms of mediations, we’ve looked at two different approaches and have working solutions for both. The first is moving the load of user.clj out of RT static initializer and into an explicit initialization method. Then at any entry point, the explicit initializer needs to be called. This is a pretty simple change but has the tricky probleme of knowing where the “entry points” are. There are some obvious ones if you’re starting from clojure.main, the Java Clojure entry point, a genclasse or AOT class file and those are easy to hook. But because this happens sort of on-demand right now there are a ton of possible entry points people could be using in weird scenarios. We have not done any heroic effort at this point. Maybe for loading user.clj it’s not important to cover all those. Ghadi put this patch together.

Another approach is suggested by Claes in his blog - if you can move the actual static methods into a separate class and fully initialize that first, your initializer doesn’t encounter the problematic scenario. I put together a version of this. Basically all of the current methods in RT move into a new RT.Impls class which has no static initializer. That class is fast to load. The RT static initializer then just makes calls into the RT.Impls class instead. To remove the reentrant calls, everything in Clojure that calls RT calls into RT.Impl instead as well. All of the methods in RT do still need to be there though (or we’d break calls from already compiled Clojure classes from previous versions or from advanced Clojure code that called directly into RT). This works, but it’s a giant patch. We think maybe we wouldn’t need to patch all calls - could probably just be a strategic subset.

We’re still trying to decide the best course of action. I expect there will be some mitigations in Java itself, but hard to say their impact or when they might arrive.

clojure.main

I wrote a few weeks ago about clojure.main not making use of our new error printing and how that affects other tools like Leiningen. I spent some more time this week trying to assess exactly what we should do for the ticket with respect to printing, stack traces, etc. Still working on the decisions and a better patch for that.

spec

I was out with the flu early in the week and with the other stuff above, did not touch spec this week. Sorry! That’s how it goes sometimes.

Other stuff I enjoyed this week…

Two songs floated through my world this week that I’ll share with you. They are very different.

First, I enjoyed this progressive metal (metal core? who can keep the micro genres straight): Absolomb by Periphery. Does it djent? Yes it does.

Second, it’s that time of the year in St. Louis where my wife asks me every day if it’s time to open the pool yet. I was delighted to find this new jam today. Pool Party by Rudy Willingham is the funky tune you need to keep summer in your heart.