Develop local, deploy (cloud) global - Java and CouchDB

Leaving the cosy world of Domino Designer behind, venturing into IBM Bluemix, Java and Cloudant, I'm challenged with a new set of task to master. Spoiled by Notes where Ctrl+O gives you instant access to any application, regardless of being stored locally or on a server I struggled a little with my usual practise of

develop local, deploy (Bluemix) global

The task at hand is to develop a Java Liberty based application, that uses CouchDB/Cloudant as its NoSQL data store. I want to be able to develop/test the application while being completely offline and deploy it to Bluemix. I don't want any code to have conditions offline/online, but rather use configuration of the runtimes for it.
Luckily I have access to really smart developers (thx Sai), so I succeeded.
This is what I found out, I needed to do. The list serves as reference for myself and others living in a latency/bandwidth challenged environment.

1. Read: There are a number of articles around, that contain bits and pieces of the information required. In no specific order:
   • Refresh on Java Naming References and Binding Information
   • Sai Vennam introduces Cloudant on Bluemix
   • Setting up the liberty profile
   • Cloudant in the liberty profile
2. Install: This is a big jump forward. No more looking for older versions, but rather bleeding edge. Tools of the trade:
   • GIT. When you are on Windows or Mac, try the nice GUI of SourceTree, and don't forget to learn git-flow (best explained here)
   • A current version of the Eclipse IDE (Luna at the time of writing, the Java edition suffices)
   • The liberty profile beta. The Beta is necessary, since it contains some of the features, notably couchdb, which are available in Bluemix by default. Use the option to drag the link onto your running Eclipse client
   • Maven - the Java way to resolve dependencies (guess where bower and npm got their ideas from)
   • CURL (that's my little command line ninja stuff, you can get away without it)
   • Apache CouchDB
3. Configure: Java loves indirection. So there are a few moving parts as well (details below)
   • The Cloudant service in Bluemix
   • The JNDI name in the web.xml. Bluemix will discover the Cloudant service and create the matching entries in the server.xml automagically
   • A local profile for a server running the Liberty 9.0 profile
   • The configuration for the local CouchDB in the local server.xml
   • Replication between your local CouchDB instance and the Cloudant server database (if you want to keep the data in sync)

The flow of the data access looks like this

Code

I consider configuration setting code too. So we have web.xml, server.xml (only the local one, the Bluemix server.xml is automagic), your Java code and the replication from/to local and couchDB

Local server.xml

In Eclipse you configure a new liberty server, and then edit the server.xml. In the directory where the server.xml can be found, add a directory lib that will contain the jars to make everything work. I will list the jars, that are current at the time of writing this, but at any time you can peek at the Bluemix server.xml to see what is there (Sai mentioned, his team was contemplating local magic as well).

<server description="stw local test server">
  <include location="local-variables.xml" />
  <!-- Enable features -->
  <featureManager>
    <feature>jsp-2.2</feature>
    <feature>localConnector-1.0</feature>
    <feature>couchdb-1.0</feature>
    <feature>jndi-1.0</feature>
  </featureManager>
  <!-- To access this server from a remote client add a host attribute to
       the following element, e.g. host="*" -->
  <httpEndpoint httpPort="9080" httpsPort="9443" id="defaultHttpEndpoint" />
  <applicationMonitor updateTrigger="mbean" />
  <library id="couchdb-lib">
    <fileset dir='${server.config.dir}/lib'
          includes='commons-codec-1.6.jar
     commons-io-2.0.1.jar
     commons-logging-1.1.3.jar
     httpclient-4.3.6.jar
     httpclient-cache-4.3.6.jar
     httpcore-4.3.3.jar
     jackson-annotations-2.2.2.jar
     jackson-core-2.2.2.jar
     jackson-databind-2.2.2.jar
     jcl-over-slf4j-1.6.6.jar
     org.ektorp-1.4.2.jar
     slf4j-api-1.6.6.jar
     slf4j-jdk14-1.6.6.jar' />
  </library>
  <couchdb id="cloudantNoSQLDB-smartpollscouch"
       jndiName="couchdb/smartpollscouch"
       libraryRef="couchdb-lib"
       password="${cloud.services.smartpollscouch.connection.password}"
       url="${cloud.services.smartpollscouch.connection.url}"
       username="${cloud.services.smartpollscouch.connection.username}" />
  <webApplication id="SmartPolls" location="SmartPolls.war" name="SmartPolls">
    <classloader commonLibraryRef="couchdb-lib" />
  </webApplication>
</server>

You want to make sure that you have the exact versions of the Java classes in the lib directory.

web.xml

The web.xml is easy, it needs to map to the jndiName you created in the server.xml, which needs to be the name you gave your Cloudant Blumix instance for the automagic to work.

<resource-env-ref>
 <resource-env-ref-name>couchdb/smartpollscouch</resource-env-ref-name>
 <resource-env-ref-type>org.ektorp.CouchDbInstance</resource-env-ref-type>
</resource-env-ref>

local-variables.xml

<server>
  <variable name='cloud.services.smartpollscouch.connection.username' value='localuser'/>
  <variable name='cloud.services.smartpollscouch.connection.password' value='localpassword'/>
  <variable name='cloud.services.smartpollscouch.connection.url' value='http://localhost:5984'/>
</server>

Java Class

Of course you could read the web.xml and look for the name there, but the simplest way is just to specify the name:

public class Couch() {
 public final static String COUCH_JAVANAME = "java:comp/env/couchdb/smartpollscouch";
    public final static String DBNAME         = "smartpolls";

    public static CouchDbConnector getDB() throws MalformedURLException, NamingException {
        CouchDbInstance dbi = (CouchDbInstance) new InitialContext().lookup(COUCH_JAVANAME);
        CouchDbConnector db = dbi.createConnector(DBNAME, true);
        return db;
    }

}

You then simply call Couch.getDB(); to access the database. Following these steps you get a valid connection both in Bluemix as well as in your local instance. The final step is replication.

Replication

Replication (or in newspeak: sync) in Cloudant/CouchDB is always one way. To get a full sync you issue 2 commands, one each for each direction. The easiest one is in the Fauxton UI of your local CouchDB, but you can POST to localhost too. The format would be:
curl http://localuser:localpassword@localhost:5984/_replicator -X POST -k -H 'Content-type: application/json' --data @replicateToServer.json
The JSON file would look like this:

{
 "source": "smartpolls",
 "target": "https://cloudantuser:cloudantpassword@cloudanturl_from_VACAP/smartpolls",
 "connection_timeout": 60000
}

To pull from the server, you just swap source and target.
As usual YMMV