clj exec

A new development version of clj is now available (1.10.1.600) and it includes the first public release of several strands of work that have been ongoing over the last few months (with more to come).

This version has not yet been promoted to “stable” release. You can either install it explicitly with that version though:

On Linux, use the instructions here but with 1.10.1.600.
On Mac, use a versioned release:
- brew uninstall clojure
- brew install clojure/tools/clojure@1.10.1.600
On Windows, use the install but with 1.10.1.600.

Execution

The primary feature of this release is support for a new kind of Clojure calling convention. Since the beginning, clj has been wrapping clojure.main for execution. clojure.main has its roots deep in the early years of Clojure and supports invoking the -main function of a Clojure namespace or running a Clojure script (reading and evaluating it).

We stepped back to think about how the command line for a Clojure entry point should look and came to several conclusions:

Execution should be tied to a fully-qualified function
The function’s namespace should be automatically loaded (thanks requiring-resolve!)
Arguments should be key/value, not positional
We want to work in edn, not strings

Thus you can now define programs as functions that take a map and execute them via clj:

clojure [clj-opt*] -X:an-alias [kpath v]*

-X is configured with a descriptor map with :fn and :args keys and stored under an alias in deps.edn:

{:aliases
 {:my-fn
  {:fn my.qualified/fn
   :args {:my {:data 123}
          :config 456}}}}

For more on using aliases to name edn data, see the “Alias data” section below.

To invoke, pass the name of the alias:

clj -X:my-fn

You can supply additional keys, or override values stored in the deps.edn file by passing pairs of key-path and value. The key-path should either be a single key or a vector of keys to refer to a nested key (as with assoc-in). Each key-path will be used to assoc-in to the original :args map, overriding the value there.

# Top key override
clj -X:my-fn :config 789

# Nested key override
clj -X:my-fn '[:my :data]' 789

Key/value pairs are read as edn strings. A general rule is that you can surround edn data with single quotes (like 'data') to get a literal shell string (which exec will read as edn data). In some cases (particularly numbers and keywords), you can omit the single quotes as bash does not have any conflicting interpretation. Note that strings must be passed in double quotes, also surrounded by single quotes to escape correctly with bash: clj -X:my-fn :config '"foo"'.

Local Maven install

One helper program included in this release is clojure.tools.deps.alpha.tools.install/install. You can execute this program with -X to install a jar into your local Maven cache.

The install argmap takes the following options:

:jar (required) - path to jar file
:pom (optional) - path to pom file
:lib (optional) - qualified symbol like my.org/lib
:version (optional) - string
:classifier (optional) - string
:local-repo (optional) - path to local repo (default = ~/.m2/repository)

Add the tool configuration to your deps.edn under an alias:

{:aliases
 {:install
  {:fn clojure.tools.deps.alpha.tools.install/install
  ;; :args map could be provided but can pass on command line instead
  }}}

To execute, use the built-in :deps alias to include tools.deps.alpha on the classpath, execute the install tool with -X and pass the args on the command line:

clj -A:deps -X:install :jar '"/path/to.jar"'

(As mentioned above, edn strings must be in double quotes, and then single-quoted for the shell.)

The install tool will find the pom inside the jar file (if it exists) and use that to determine the groupId, artifactId, and version coordinates to use when the jar is installed. Alternately, you can provide a pom file or specific coordinates via the other attributes.

Other existing “programs” provided with clj (like -Sresolve-tags) may be converted to this in the future.

Runtime basis

The primary functionality provided by tools.deps is to take a set of dependencies and compute the expansion of the transitive dependency tree and the subsequent classpath intended for a set of deps and paths. While we had means to give you parts of that result, it has been clear from working on other features and libraries (add-lib, tools.deps.graph, etc) that we needed something to tie all of this information together.

We call the output of the resolution process the “runtime basis” (or just “basis” for short). The runtime basis is a map that is a combination of the inputs to the process (merged deps, resolve-args, classpath-args) and the outputs (the lib-map and classpath-map). In combination this data can be used to understand the full classpath context of a subsequent execution.

The Clojure tools now caches the computed basis data, just as it previously cached the lib map data, and injects the basis file into the execution via the Java system property “clojure.basis”. Programs can read this data using the tools already in Clojure (it’s just an edn map):

(require '[clojure.java.io :as jio] '[clojure.edn :as edn])
(def basis (-> (System/getProperty "clojure.basis") 
               jio/file
               slurp
               edn/read-string))

Alias data

deps.edn aliases were always conceived of as an open mechanism to store edn data and give it a name for use in programs. For a long time we have only used it to provide such data to the internal processes of clj itself (via -A etc), but with this release we are expanding the use of alias data in several ways.

First, :paths can now (in addition to path strings), take alias names that refer to vectors of paths:

{:paths [:clj-paths :resource-paths]
 :aliases
 {:clj-paths ["src/clj" "src/cljc"]
  :resource-paths ["resources"]}}

Second, execution with clj -X:an-alias takes an alias that provides a map of inputs for a function.

Third, the runtime basis is injected into a program so anyone can write programs that take the alias for edn data stored in deps.edn and make use of it via the basis:

(def my-data (-> basis :aliases :an-alias)

Deprecated unqualified lib names

One other change that you may notice in this release is that we have deprecated support for unqualified lib names in :deps. Unqualified lib names (like cheshire or hiccup) are interpreted in Maven terms as having the same groupId and artifactId. Unqualified lib names will now warn on use - you can address by using the qualified name (cheshire/cheshire or hiccup/hiccup).

The groupId exists to disambiguate library names so our ecosystem is not just a race to grab unqualified names. The naming convention in Maven for groupId is to follow Java’s package naming rules and start with a reversed domain name or a trademark, something you control. Maven itself does not enforce this but the Maven Central repository does all new projects.

In cases where you have a lib with no domain name or trademark, you can use a third party source of identity (like github) in combination with an id you control on that site, so your lib id would be github-yourname/yourlib. Using a dashed name is preferred over a dotted name as that could imply a library owned by github.