Object Commando languages, development and design


Easy Java Interop with Clojure

This past week I started writing some code to work with Amazon EC2 Instances. I started using the JClouds library . It's a great library for spinning up public AMIs in a cloud neutral way, however didn't it do everything that I needed. Some of the things I needed were EC2 specific, so that's not so surprising. I fell back to the AWS SDK for Java, which basically just wraps calls to the Amazon web services. Using that library, I wrote some Clojure functions that wrapped the Java calls to do what I needed. Examples of what I needed would be to start up an existing EC2 EBS backed instance, stop an EBS instance and determine what state an EBS VM is in. This led to Clojure code that would build up some request objects and interpret some response POJOs. The API is a little awkward, even using Java. Starting, stopping and describing an instance via the API all require one thing, one or more instance ids. In the Amazon API, they have created a separate class for each (DescribeInstancesRequest, StopInstancesRequest and StartInstancesRequest) and have those classes include a method where you setInstanceIds rather than just calling a method and passing a list of Strings (or something similar). Working with this API helped me learn more about the Clojure Java Interop functions.

dot dot

The first feature that made my life easier was .. . This is a macro for chaining Java calls. It takes code that in Java would look like this:


And puts a similar feel in Clojure:

(.. instance getState getName)

Without this macro, you would have to:

(.getName (.getState instance))

It takes the return value of the first part (in the inner expression above) and passes it into the second. The code above works great for chained method calls, but doesn't help much with side affects.


I found doto useful when I needed to call setters in constructing POJOs. Calls to determine the status of a running EC2 instance returns an object graph of several nested objects and were particularly awkward to test, since there were a decent amount of objects to construct. Before I realized doto could help me, I had code that looked like below:

(defn single-instance-result-example []
    (let [reservation-list (ArrayList.)
           instance-list (ArrayList.)
           reservation (Reservation.)
           instance (Instance.)
           instance-result (DescribeInstancesResult.)]
       (.setInstanceId instance "testinstance")
       (.add instance-list instance)
       (.setInstances reservation instance-list)
       (.add reservation-list reservation)
       (.setReservations instance-result reservation-list)

I then transformed this into some code that used some nested dotos

(defn single-instance-result-example []
    (doto (DescribeInstancesResult.)
        (doto (ArrayList.)
                (doto (Reservation.)
	                (doto (ArrayList.)
                                (doto (Instance.)
                                    (.setInstanceId "testinstance")))))))))))

I must admit that the dotos here took some time to get comfortable with. Some interesting things to note is that the above does not include all of the intermediate references to objects. In the first example above I always passed the objects in, such as (.setInstanceId instance "testinstance"). This is no longer necessary with doto. The intermediate let-bound variables are also not necessary. I seeing the above code, I felt like I still had some room for improvement. In the above example and in other areas of my code I was seeing a common pattern:

(doto (ArrayList.)
    (.add (doto...))
    (.add (doto...))

So I created a quick macro that I called doto-list that would bundle that piece up:

(defmacro doto-list [& forms]
     `(doto (ArrayList.)
         ~@(map (fn [item] `(.add ~item)) forms)))

Which then made the function look like:

(defn single-instance-result-example []
    (doto (DescribeInstancesResult.)
            (doto (Reservation.)
	                (doto (Instance.)
	                    (.setInstanceId "test")))))))))

Which I think is a nice improvement when I'm creating a decent amount of ArrayLists.


The next piece I used integration with the AWS APIs was memfn. The call to describeInstances returns a List of Instance objects which have a few fields I'm interested in. There are quite a few fields on the Instance object (20+) and I was only interested in a few. Furthermore, I also did not want the callers of the functions to have to know they were dealing with a Java objects. One was bean, which transforms an object into a map of it's bean properties. It seemed like it would work for me, but would pull over fields I cared about and a lot of ones that I didn't. It would also require knowledge of the Java object and the generated map structure. I thought a better way to do this would be memfn. The memfn macro takes an method name (and optionally arguments) and returns a function that takes an object as a parameter. The function then invokes the method on that object when called. It basically translates the above call into:

((memfn getPublicIpAddress) some-object)

This seemed closer to what I wanted, but what I really wanted was a function called "public-ip" that you could pass an instance to. So I ended up attaching a name to the memfn function that was returned by creating a little macro:

(defmacro defmemfn
     [name method-name & args]
     `(def ~name (memfn ~method-name ~@args)))

A call to defmemfn looks like:

(defmemfn public-ip getPublicIpAddress)

And asking for a public ip looks like:

(public-ip instance)

Up Next - Testing

In conclusion, I think that working with the AWS APIs was easy thanks to Clojure's great Java integration. Testing this code was also much easier than I thought which I'll post next.

Comments (4) Trackbacks (0)
  1. Hey, Ryan.

    I’m not sure by this code where jclouds left you hanging, maybe you can help us figure it out!

    According to the above, you are using the provider-specific apis, which are probably not clearly documented in our clojure api. However, they do exist! For example, have a look at ebs.clj, elastic_ip.clj

    Sorry it seems our documentation failed you and caused you to go through trouble!

    Maybe next time!

  2. Hello,

    Please note:

    First on dot dot:
    dot dot is not the idiomatic way of chaining calls those days. It’s -> which works both for java interop and regular function calls:
    (.. instance getState getName) becomes (-> instance .getState .getName)
    dot dot is still useful as a more basic building block, e.g. when writing macros.

    Please also note the -?> operator which prevents the code to throw a NullPointerException if at some point in the call chain a method/function returns null:

    (-> instance .getState .getName) will throw NPE if .getState returns null (because invoking .getName on null will throw NPE)
    (-?> instance .getState .getname) will return null if .getState returns null (will not try at all to invoke .getName)

    -?> (and there’s also a .?. equivalent version) is in clojure contrib: clojure.contrib.core

    The name -?> is derived form similar operators in OO languages, e.g. in groovy to achieve the same NullPointerException-preserving behaviour, there’s the ?. operator : instance?.getState()?.getName()

    Second on memfn:
    memfn has been called “deprecated” several times by Rich Hickey himself several times on google group threads or #clojure (IRC).
    The more idiomatic way of encapsulating is via anonymous functions, e.g. :

    (def public-ip #(.getPublicIpAddress %))

    In particular this will allow you to place type hints to avoid reflection performance costs at runtime:
    (def public-ip #(.getPublicIpAdress #^com.some.package/ExpectedTypeOrInterface %))



  3. Thanks for the info Laurent! I use -> quite a bit, but didn’t know it worked for Java Interop too. Can you link to some information on-?>. I can’t seem to find info on it and it’s probably the best fit for what I am doing.

    I’ll switch my code from memfn to the above anonymous function stuff. Reading through the docs on memfn, I didn’t see anything about it being deprecated, so I figured it was safe to use.

    Thanks for the feedback!


  4. Man, this looks like my lisp from the college days

Leave a comment


No trackbacks yet.