Clojure in the cloud. Part 1: Google App Engine

25.04.2014 | Tero | Comments

This is the first post in a group of tutorials on deploying Clojure applications on various cloud platforms.

All posts in the series:

Google App Engine (GAE) was one of the first successful PaaS solutions. It is a battle tested platfom for building and deploying applications to the cloud.

Currently App Engine provides officials SDKs for Python, PHP, Go and Java. Clojure does not have it’s own SDK but worry not, since Clojure is hosted on the JVM, the Java version will work just fine.

The prerequisites for this tutorial are:

Google App Engine
image: Google App Engine

Google App Engine

There are multiple Leiningen plugins, that make developing Clojure apps for the App Engine easier. However, since the App Engine SDK and interfaces evolve over time, the plugins tend to get outdated pretty quickly.

Developing a Clojure app for GAE is actually pretty simple even without the help of extra tools that hide what’s really happening. Basically what you need is an exploded war file with a proper deployment descriptor.

The source code for this tutorial are in Github.

Create a hello worldish Compojure app

If you are already familiar with creating web apps with Ring and Compojure, you may wish to jump straight to packaging the app for deployment.

For this example, we are creating a Ring/Compojure app using a Leiningen template compojure-template. It is a “batteries included” template for developing Compojure applications. For GAE deployment lein-ring would suffice since it contains the functionality for creating a GAE compatible package.

lein new compojure-app gae-app-demo

This sets up a simple project structure for a Compojure app that will show a “Hello World!” when deployed. We will take it a tiny bit further by creating a form and adding a form handler that displays the submitted data.

The compojure-app template creates the following file structure:

gae-app-demo
 ├── project.clj
 ├── README.md
 ├── resources
 │   └── public
 │       ├── css
 │       │   └── screen.css
 │       ├── img
 │       └── js
 ├── src
 │   └── gae_app_demo
 │       ├── handler.clj
 │       ├── models
 │       ├── repl.clj
 │       ├── routes
 │       │   └── home.clj
 │       └── views
 │           └── layout.clj
 └── test
     └── gae_app_demo
         └── test
             └── handler.clj

Project configuration

project.clj, the Leiningen configuration file looks like this:

(defproject gae-app-demo "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [compojure "1.1.6"]
                 [hiccup "1.0.5"]
                 [ring-server "0.3.1"]]
  :plugins [[lein-ring "0.8.10"]]
  :ring {:handler gae-app-demo.handler/app
         :init gae-app-demo.handler/init
         :destroy gae-app-demo.handler/destroy}
  :aot :all
  :profiles
  {:production
   {:ring
    {:open-browser? false, :stacktraces? false, :auto-reload? false}}
   :dev
   {:dependencies [[ring-mock "0.1.5"] [ring/ring-devel "1.2.1"]]}})

If you are not familiar with projects built with Leiningen, we will quickly go through the settings. As the name suggests, project.clj is just a normal Clojure file. All it contains is the macro defproject. The first argument to defproject is the project name and the second is the current version of your app. Rest of the data are key-value pairs. Keys are keywords and values are Clojure data types like strings, numbers, maps or vectors.

  • :description - Evident. Eg. searchable in the Clojars should you choose to publish your project as a component.
  • :URL - Whatever site you wish your project to refer to. This is just metadata like :description
  • :dependencies - Runtime dependencies. Here we have:
    • Clojure itself. Note that you don’t need to have Clojure installed. Leiningen handles it dynamically.
    • Compojure, the web framework we are using. An extension to the Ring library.
    • Hiccup, an HTML templating library
    • ring-server, a library for (programmatically) running your web app. Starts a webserver and deploys the app.
  • :plugins - Leiningen plugins used with this project. A set of hooks and tasks you can run from the command line.
    • lein-ring, Very useful tasks. Eg. run the app locally by starting a web server on demand. Also contains tasks for packaging your application. We will use that later in the tutorial.
  • :ring, configuration for the lein-ring plugin
    • :handler, A Ring handler that creates the response for each request. Ring handlers are normal Clojure functions, which take a request map and return a result map.
    • :init, A function that is called before the handler starts.
    • :destroy, A function that is called before the handler exists.
  • :aot, specifies if the project should be ahead-of-time compiled to JVM bytecode. Otherwise Clojure compiles on-the-fly from the source. AOT compiled code doesn’t need the source code. Basically AOT compiled Clojure code is like any other compiled Java code. You need AOT If you wish to package your application as a war. That’s what we will be doing.
    • :all, compile all namespaces. You could specify just the namespaces you want AOT compiled.
  • :profiles, Leiningen supports extra configuration sets for different contexts.
    • production, settings for production releases.
    • dev, support for debugging and mocking requests for unit tests.

Routes and handlers (= the code)

handler.clj
(ns gae-app-demo.handler
  (:require [compojure.core :refer [defroutes routes]]
            [compojure.handler :as handler]
            [compojure.route :as route]
            [gae-app-demo.routes.home :refer [home-routes]]))
 
(defn init []
  (println "gae-app-demo is starting"))
 
(defn destroy []
  (println "gae-app-demo is shutting down"))
 
(defroutes app-routes
  (route/resources "/")
  (route/not-found "Not Found"))
 
(def app
  (-> (routes home-routes app-routes)
      (handler/site)))

This is where it all starts. The var app, which we specified in project.clj, is evaluated to a Ring handler. A Ring handler is a function that takes a request and returns a result. Both are maps. The request is decorated with various middleware functions before reaching the final handler function. The request typically contains eg. the query string and form params and the result typically contains eg. the HTTP status code and the actual HTML-response.

First a set of routes is specified and “merged” into a single handler function. Routes are a mechanism of Compojure for dispatching a request to a correct piece of code. Routes are defined using a macro defroutes. Defroutes returns an ordinary Ring handler. The function compojure.core/routes combines several handlers.

Here we have two sets of routes, home-routes and app-routes. home-routes contain our main logic. They are defined in src/gae-app-demo/routes/home.clj. app-routes specify the path to static resources and a response when none of the specified routes match the request (= 404).

The -> aka. thread-first macro feeds the routes (= the handler function) to the function compojure.handler/site, which is a collection of Ring middleware functions. Ring middleware are functions that take a handler and return a handler. They are an elegant and functional way for decorating handlers. A middleware function can manipulate handler input (request), output (result) or both.

The functions init and destroy were also specified in project.clj. Here they don’t have any purpose other than logging the events.

home.clj

(ns gae-app-demo.routes.home
  (:require [compojure.core :refer [defroutes GET POST]]
            [gae-app-demo.views.layout :as layout]))
 
(defn form-page []
  (layout/common (layout/registeration-form)))
 
(defn result-page [{:keys [form-params]}]
  (layout/common (layout/result form-params)))
 
(defroutes home-routes
  (GET "/" [] (form-page))
  (POST "/" request (result-page request)))

The route definitions in home.clj are very simple. GET “/” serves the form and POST “/” handles the submitted form data and creates a result page. Both route definitions delegate the HTML creation to layout.clj

The function result-page uses destructuring for extracting the value of the key :form-params from the request map. If you are not familiar with destructuring, check out this excellent blog post by Jay Fields.

layout.clj

(ns gae-app-demo.views.layout
  (:require [hiccup.page :refer [html5 include-css]])
  (:require [hiccup.form :refer [form-to submit-button]]))
 
(defn common [& body]
  (html5
    [:head
     [:title "GAE demo"]]
     (include-css "/css/screen.css")
    [:body body]))
 
(defn custom-input [type name label placeholder]
  [:div
   [:label label
    [:input {:type type :name name :placeholder placeholder}]]])
 
(defn registeration-form []
  [:div
   [:h1 "A form for the ages"]
   (form-to [:post "/"]
            (custom-input "text" "first-name" "First name" "First name")
            (custom-input "text" "last-name" "Last name" "Last name")
            (custom-input "text" "email" "Email" "Your email")
            (submit-button "Submit"))])
 
(defn result [form-params]
  [:div
   [:h1 "Form data sent to the server"]
   (for [[field-name field-value] form-params]
     [:div (str field-name ": " field-value)])])

Here we use the Hiccup library for rendering the HTML. One of the coolest things about Hiccup is that it uses plain and simple Clojure data structures. There is no magical special syntax. This has two major benefits:

  1. Easy to read and write
  2. Everything is programmable. You can iterate or modify the data structures like any other data. See the line 29 for example. We use for - Clojure’s list comprehension - for generating markup for each form parameter and its value.

Vectors that start with a keyword represent a tag. Tag attributes are defined using a map.

For example

[:input {:type "text" :name "email" :placeholder "your email"}]

generates

<input name="email" placeholder="Your email" type="text">

Creating parameterized templates with Hiccup is also trivial. Just create a function that takes arguments and use those arguments when building the HTML like we have done in the function custom-input.

You can create the whole document from scratch if you like, but Hiccup does provide some handy utility functions for easing the task. We have used include-css, form-to and submit-button - all of them rather self-explanatory.

Finally, in the function common all of the Hiccup data structures we have created are wrapped by the macro html5, which creates the actual HTML document.

Packaging the app for deployment

lein ring uberwar

This packages the application to a war file with required dependencies. The war file will be found from the folder target.

There are 2 manual steps before the app is ready to be deployed.

  1. Extract the war file. GAE requires the exploded (=unzipped) version.
    • Since war files are simple zip files, you can just: unzip -d <where-to-extract> target/<war-file>
  2. Add the deployment descriptor appengine-web.xml to the directory <exploded-war>/WEB-INF

appengine-web.xml

<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
  <application>your-gae-application-id</application>
  <version>1</version>
  <threadsafe>false</threadsafe>
</appengine-web-app>

The only thing you need to modify is the application id. It is basically the name of your application you gave when creating it. You can check your application id at https://appengine.google.com/

The tag threadsafe specifies whether GAE will send requests serially or in parallel. You can leave it to false for now.

The structure of an exploded war ready to be deployed to App Engine

META-INF/
  manifest.mf

WEB-INF/
  classes/
  lib/
  appengine-web.xml
  web.xml

If you are previously familiar with war files, there is nothing out-of-ordinary here except appengine-web.xml. That is all there is to it, the app is ready for deployment.

Deploying GAE apps

GAE apps are deployed using the scripts provided by the SDK. The scripts are found in the folder <SDK-root>/bin.

Make sure the bin folder is in your PATH. We will be using 2 scripts: dev_appserver and appcfg. The bin folder contains versions for both Unix and Windows environments.

Deploy locally to test your app

Before deploying your app to the public server, you can test it locally.

dev_appserver.sh <path-to-exploded-war>

This will start a Jetty server and deploy the app. The default port is 8080 so you can point your browser to http://localhost:8080. It should greet you with the form we just created.

Deploy to App Engine

When you are confident enough that your app works, you can deploy it to App Engine using the script appcfg.

appcfg.sh update <path-to-exploded-war>

Uploading the war to the server takes a while. When done, you can admire your new and shiny cloud app at:

http://<your-application-id>.appspot.com.

Footnotes

  1. More info about uploading and managing a Java App using appcfg

Tero Kadenius is a software developer, ScrumMaster and a self proclaimed change agent with years of experience in the industry. Tero is the chairman of Agile Jyväskylä Ry, the non-profit behind the AgileJkl conference. Tero has a background in Java based technologies. Currently Tero finds functional programming a great tool for solving hard problems.

comments powered by Disqus