Trying something new here … writing down some visible things I worked on this week (not everything is visible of course :) and some conversations I was in around the Clojure sphere. This will be pretty rambly without much editing. Maybe I’ll do this more, maybe not, let me know if it’s useful.

IllegalAccessException

I spent some time earlier this week digging into an IllegalAccessException that was reported in REBL from people trying to use Java 11 + latest JavaFX + REBL. While I have spent some time working on illegal access warnings due to reflective accesses across modules with Java 9+, this one was a little different as it involved an actual exception. It also seemed like this particular error should have been caught by my changes in CLJ-2066.

Anyhow that involved trying to make something reproducible and examining the stack trace in more detail, which really told the story. There are two different paths in the Clojure reflector and CLJ-2066 addressed one of those, but this was in the second code path, trying to invoke a field or method reflectively.

The general problem the reflector faces is having a target object, a method name, and some parameters and the goal of either finding or invoking the most appropriate method. The reflector starts from the type of the target object to look for a matching method (same name, same arity, and compatible parameter types taking into account boxing, widening, etc).

In the new Java module system, code inside a module is, by default, inaccessible by code outside the module. Modules declare which parts are visible by “exporting” packages. Reflection has full rein to examine everything, but invocations are gated by accessibility according to the module definition.

It’s relatively common for a module to export a public interface, and a factory to create instances of the interface. Usually, the implementations are private and not exported. In this case, Clojure will have a target object of the private implementation class, but the methods on that private class are inaccessible. Invoking them will yield an illegal access warning or exception. Instead, the Clojure reflector needs to look up the class hierarchy and find a public, accessible variant of the method (usually the public interface). The reflector had some of that logic already, but the additionally accessibility check needed to be added, and this is all captured in CLJ-2454.

We are in the final burn down at this point to Clojure 1.10, having released RC4 last week. This access problem is squarely in the middle of compatibility with the latest Java releases, which is one of the key targets for 1.10, so we decided this was important enough to include before we release. That resulted in the release of 1.10.0-RC5.

Docs

I’ve also spent a lot of time in the last week updating docs on the web site. There are new bits and pieces of docs for 1.10 in several places and some migrations and updates from other places:

• Protocol metadata extension
• Error printing
• FAQ on illegal access warning
• tap
• Updated spec guide for 1.10 error printing
• Updated a bunch of exception messages across many pages to match 1.10
• FAQ on how to skip checking macro specs - missed in 1.9 updates

Also, I migrated Contrib workflow, Creating Tickets, and Developing Patches from the old community wiki. We plan to do a bit more clarifying some of the stuff in this area, so this is just a start. Sean Corfield requested some clarification around some of the contrib library stuff and I agree it’s probably a good time to do so as I see a lot of confusion and false assumptions about that.

I also burned through a bunch of site PRs - thanks to everyone on those. I won’t list them all but you can look at the recently closed PRs for more detail. I will highlight the addition of a new equality guide from Andy Fingerhut that is a newly updated version from the guide he has maintained for many years. Glad to finally add it!

We have a private CI system that auto-builds the clojure.org and clojurescript.org web sites. That’s been chugging along pretty much unattended for 3 years, but unfortunately the product where we have it hosted is shutting down tomorrow. So this week I’ve spent a bunch of time doing research to decide the best replacement option. Still working on that, but it will be a bit more manual till that’s up and running.

clojure.test

I had a long and interesting conversation on Slack this week about clojure.test, originally triggered I think by a tweet from Nikita Prokopov about the error reporting in clojure.test. I think there were several independent suggestions embodied in his changes there - focusing on root cause in stack trace reporting (good), filtering stack head to remove test infrastructure noise (good, and already being done to some degree), filtering other parts of the stack (I’m more dubious about), demunging Java to Clojure in stacks. I have mixed feelings about the last, but maybe reasonable in the context of clojure.test reporting.

Sean Corfield broadened the conversation to talk about clojure.test as a general area of improvement (and why it has not seen much love in years). Certainly many people have written their own take on test libs (even Stuart Sierra who was largely responsible for clojure.test, wrote a second version!) or test runners. So clearly, there is no shortage of ideas in this area.

But clojure.test has the privilege / onus of being “the one in the box”. I think Stuart S can garner a great deal of credit for it being good enough that many people are happy enough to get by using nothing more than that, even without substantial follow-on improvements. And certainly there is no reason you have to use it - all these external libs exist, do good stuff, and are used. But probably there are a set of changes for clojure.test that would be useful to people.

One thing that we have talked about is splitting out libraries like clojure.test into detached libs, like we did with spec. These could still be pulled in by default, but making them independent would let us rev them independently and more frequently than Clojure itself.

Both Sean and Rick Moynihan seemed interested in trying to come up with a list of improvement areas and that would be a helpful thing. There are only a relatively small number of clojure.test tickets in jira and I don’t think they really reflect the issues people have with it. Having a vetted and prioritized list of problems that could be addressed would be a huge help in deciding what is useful to do in the future. I’d say one thing we’re NOT interested in is breaking stuff. So any changes to clojure.test should be largely additive or in extensibility rather than radical changes to the api. Certainly I think there are some things in the areas of reporting, fixtures, and test running that could be useful.

Other tidbits

• Uploaded the Day of Datomic Cloud videos from Strange Loop
• Doing prep for clj and the Clojure api docs in relation to 1.10 release
• Updated many CLJ tickets, did some screening+prescreening for 1.11, cleaned up many tickets
• Starting to look at some tools.deps stuff again

Non Clojure stuff I enjoyed this week…

New Vulfpeck album, Hill Climber! I’m moderately obsessed with Vulfpeck and their unrelenting funkiness. Everyone should have more Vulf in their life. Sleeper track: Disco Ulysses.

Really good podcast with Ezra Klein interviewing Hasan Minhaj. Although actually, I think Hasan had some of the best questions for Ezra. Hasan’s new show is really good, also check that out.