Usability - Productivity - Business - The web - Singapore & Twins

The quick and dirty Domino Cloudant export

Moving data out of Domino never has been hard with all the APIs available. The challenge always has been: move them where? Ignoring for a second all security considerations, the challenge is to find a target structure that matches the Domino model. Neither flat table storage nor RDBMS fit that very well.
A close contender is MongoDB which is used in one compelling Notes retirement offering. However the closest match in concept and structure is Apache CouchDB, not surprisingly due to its heritage and origin.
It is maintained by a team led by the highly skilled Jan Lehnardt and of course there are differences to Notes.
But the fit is good enough. Using the lightweight Java library Ektorp exporting a set of documents from Notes to CouchDB is a breeze. The core class is a simple mapping of a Notes document to a JSON structure:
package com.notessensei.export;

import java.util.HashMap;
import java.util.Map;
import java.util.Vector;

import lotus.domino.Document;
import lotus.domino.Item;
import lotus.domino.NotesException;

public class NotesJsonDoc {
	public static String ID_FIELD = "_id";
	public static String REV_FIELD = "_rev";
	private Map<String, String> content = new HashMap<String, String>();
	public NotesJsonDoc(Document source) throws NotesException {
		Vector allItems = source.getItems();
		for (Object itemObject : allItems) {
			Item item = (Item) itemObject;
			this.content.put(item.getName(), item.getText());
		this.content.put(ID_FIELD, source.getUniversalID());
	public Map<String, String> getContent() {
		return this.content;
	public NotesJsonDoc setRevision(String revision) {
		this.content.put(REV_FIELD, revision);
		return this;
	public String getId() {
		return this.content.get(ID_FIELD);

Read more

Posted by on 2016-01-21 01:45 | Comments (1) | categories: Bluemix

The 3C of leader - leader: Clarity, Competence, Control

An organisation failing their objectives has a reflexive reaction: implement more controls. Rumor has it, some sales organisations perform daily cadence calls towards the end of a period to ensure deals close (as if time away from the customer helps).
L. David Marquet, in his book "Turn the Ship Around!" paints a different story. Like a plane needs 3 points to be defined, functional organisations need 3C to operate at peak performance: Clarity, Competence & Control.
The Leader Leader principles
I wrote a short review on Amazon, but prefer the comfort of my blog to unfold a few thoughts.
In his story David is in an interesting position. In the pecking order of the Navy he is middle management, having at least 3-4 layers of management above him (until you reach POTUS), however on his ship he is in ultimate command. On high sea that is obvious. I wonder how many division heads, country leaders, section heads. etc. can related with that and see themselves in the same location and opportunity.
The killer question he asks is about processes (Page 159): " Have your processes become the master rather than the servant?". This is a brilliant question to irritate process champions - but essential to become a high performance operation.
The stance on "guiding principles" sum it up nicely: "The guiding principles needed to do just that: provide guidance on decisions". If they lack clarity, they can't do that. Some people use role models (contemporary, historical or spiritual): "What would [insert-name-here] do", but a clearly formulated set of principles (like "We the people ....") help to stay focused.
To that extend I like IBM's core values (and the mission: "Become the most relevant company on the planet"): Dedication to every client's success, Innovation that matters - for our company and for the world, Trust and personal responsibility in all relationships. Using them to decide if in doubt helps to preserve personal integrity.
Of course you could listen to Immanuel Kant and follow the categorical imperative:" Act only according to that maxim whereby you can, at the same time, will that it should become a universal law." Or you take the shorter version as promoted by Lama Marut :" Be nice".

Enjoy your read. Makes a good present to manager too,

Posted by on 2016-01-09 04:00 | Comments (0) | categories: Business Learning

Mess with the Bluemix Colors

The Bluemix designers consider their color scheme: robust, decent and unobstrusive. However not everybody likes the dark colors ( some do). Stylish to the rescue. It comes in Firefox and Chrome flavours. It requires a custom style sheet and it might take you a while to figure things out. So use this for starters:
 .dashboardArtifactCreationSection .tile,
 .tile-container .tile,
 .bluemix-global-header .bluemix-nav-list,
 .cloudOEAppDetails .cloudOEActionBarDockedNavArea,
 .cloudOEAppDetails .cloudOEActionBarNavigationTreeNode,
 .d-category-section .category-header,
 .cloudOEActionBarContentArea .cloudOEFilterBar .cloudOESearchBox .dijitInputField,
 .cloudOEActionBarContentArea .cloudOEFilterBar .cloudOESearchBox input,
 .cloudOEActionBarContentArea .cloudOEFilterBar .cloudOESearchBox .dijitTextBox,
 .d-docked-nav-area .d-nav-container,
 .appDetailsOverview_Health .tile-segmented,
 .appDetailsOverview_EstimateCost {
    background-color: #3366ff;
    color: #fff;
  body {
  background-color: white;
  header {
    background-color: #6677CC;
  .catalog-container {
    background-color:  #65C4FF;
    color: black;
   .cloudOEDockedOpenNav {
    background-color:  #4dc4ff;
     color: black;
Now go and pick nice colors.
As usual YMMV

Posted by on 2016-01-08 11:57 | Comments (0) | categories: Bluemix

Mail archive with Apache CouchDB / IBM Cloudant - Part 1

Like it or not, your eMail turned into the archive of your (working) past. One of the challenges with this archive is the tendency to switch eMail systems from time to time. IBM Notes won't open your Outlook PSD file, nor would Outlook open your Notes NSF database.
So a vendor and format neutral solution is required. The obvious choice here is MIME, which is for one, the format any message crossing the internet is encoded in, secondly all eMail applications support MIME - to some extend. Just storing each message into a directory structure isn't a good solution either, since navigation and search leave much to be desired, so some more work is needed.
Of course open standards tend to be ambiguous enough to allow different interpretation or the implementation of propriety extension. MIME is no exception. You can send any type of attachments, including malicious payloads, which are encoded and outside the MIME standard.
So looking at an archival solution here is my list of requirements:
  • Needs to be able to store MIME messages
  • Mime headers an other id fields need to be captured in database fields
  • Need to be able to sync on different locations for backup/availability
  • Need to be able to provide navigation access by sorted, filtered lists
  • Interface to do some analytics
  • Full text search
  • HTML and text content should be displayed directly, all other types should be listed as attachments
  • Inline images (href / src in the html content pointing to other mime parts) need to be dealt with
  • Import capabilities
  • Source code for inspection available, OpenSource if possible
Looking at the requirement, I concluded: I got a clear idea what I want to have, but I haven't found it. The logical next step: Let's build it.

Read more

Posted by on 2015-12-24 12:42 | Comments (0) | categories: Bluemix IBM Notes

Automated Tests in Bluemix Build and Deploy

Bluemix is a beautiful environment for agile software development. Its build and deploy capability ensure continious delivery, so you can focus on code. A well run project requires automatic testing (starting with unit tests up to integration testing).
You can configure this in the Build and Deploy pipeline, so your project looks like this:
A fully working pipeline
While you could argue: " I run my tests local", you might encounter the situation where you use the online editor and you then depend on the tests running in Bluemix. Setting up build and deploy is well documented and straight forward.

Tip: For Java projects you should use Maven or Gradle build, so your library dependencies are properly resolved. For Node.js projects a "simple" build would suffice, however using npm install would make the stage fail if your package.json has an issue, so you don't run later stages and fail there

However the documentation for the test stage simply states: " If you want to require that certain conditions are met, include test jobs before or after your build and deploy jobs. Test jobs are highly customizable. For example, you might run tests on your project code and a deployed instance of your app. ". That's a little " thin".
Inspecting the test job screen itself, we can learn that we have different testing options
Choices for testing
I specifically like the Sauce labs integration and the ability to run a code, security and vulnerability scan (So a real pipeline might have up to 4 distinct test stages). However the screen for "simple" tests, where unit tests go, isn't particularily helpful:
The helpful test screen
So lets shed some light on the inner workings.

Read more

Posted by on 2015-11-26 10:43 | Comments (0) | categories: Bluemix

It's just HTML, CSS and JavaScript isn't it?

Web development is easy isn't it? After all it is just the open standards of HTML, CSS and JavaScript.
That's roughly the same fallacy as to say: It's easy to write a great novel, since it is just words and grammar.
The simple reason: It is hard.
I'll outline my very opinonate version of what skills you need for mastery of front-end programming. The list doesn't include the skills to figure out what to program, just how. So here it goes:


Jeffry Veen, who'm I had the pleasure to meet in Hong Kong once, summarized it nicely in his book: The Art and Science of Web Design already in 2000: " HTML provides structure, CSS layout and JavaScript behavior". Like English, development needs a style guide (and the mastery of it), web development has several guides, besides the plain definitions mentioned in the beginning.


  • The standard today is HTML5. So ignore any screams "But it must run in IE6" and see what HTML5 can do. There's a handy list of compatible elements you can check for reference
  • The predominant guideline for structure is Twitter bootstrap. It defines a layout and structure language, so your page structures become understandable by many developers. Twitter bootstrap contains CSS and JavaScript too, since they are intertwined
  • If you don't like Bootstrap, there are other options, but you have been warned
  • The other HTML structure to look at is the Ionic framework. Again it has more than HTML only


Most frameworks include CSS, so when you picked one above, you are pretty much covered. However understanding it well takes a while, so you can modify the template you started with. My still favourite place to get a feel for the power of CSS is CSS Zengarden. The W3C provides a collection of links to tutorials to deepen your knowledge. My strong advice to the developers: don't touch it initially. Use a template. Once your application is functional, then revisit the CSS (or let a designer do that).


Probably the most controversial part. IBM uses Dojo big time, ReactJS is gaining traction and there is Aurelia up and coming. So there's lot to watch out for. But that is not where you get started.
  • Start learning JavaScript including ES6. Some like CoffeeScript too, but not necessarily for starters
  • The most popular core library is jQuery, so get to know it. The $ operator is quite convenient and powerful
  • To build MVC style single page applications I recommend to use AngularJS. It has a huge body of knowledge, a nice form module and an active community. Version 2.0 will have a huge boost in performance too. Make sure you know John Papa's style guide and follow it
  • And again: have a look at Ionic and Cordova (used in Ionic) for mobile development. It uses AngularJS under the hood
There are tons of additional libraries around for all sorts of requirements, which probably warrants another article.


With all the complexity around, you don't have to start from scratch. There are plenty of templates around, that, free or a little fee, give you a head start. Be very clear: they are an incredible resource when you know how all is working together, but they don't relieve you from learning the skill. Here are my favourites

Tools and resources

Notepad is not a development tool. There are quite some you need if you want to be productive
  • You need a capable editor, luckily you have plenty of choices or you opt for one or the other IDE
  • Node.js: a JavaScript runtime based on Google's V8 engine. It provides all runtime for all the other tools. If you don't have a node.js installation on your developer workstation, where were you hiding the last 2 years?
  • Bower: a dependency manager for browser files like CSS, JS with their various frameworks. You add a depencency (e.g. Bootstrap) to the bower.json file and Bower will find the right version for you
  • Grunt: a task runner application, used for assembly of web sites/applications (or any other stuff). Configured using a Gruntfile.js: it can do all sorts of steps like: copy files, combine and minify, check code quality, run tests etc.
  • Yeoman: A application scaffolding tool. It puts all the tools and needed configurations together. It uses generators to blueprint different applications from classic web, to reveal application to mobile hybrid apps. The site lists hundreds of generators and you are invited to modify them or roll your own. I like generator-angularand mcfly
  • GIT: the version control system. Use it from the command line, your IDE or a version control client
  • Watch NPM for new development

Those are the tools to mastery, when you just want HTML, CSS and JavaScript

Posted by on 2015-11-13 09:48 | Comments (2) | categories: Software

Multitenancy - a blast from the past?

Wikipedia defines multitenancy as: " Software Multitenancy refers to a software architecture in which a single instance of a software runs on a server and serves multiple tenants ... Multitenancy contrasts with multi-instance architectures, where separate software instances operate on behalf of different tenants "
Looking at contemporary architectures like Docker, OpenStack, Bluemix, AWS or Azure, I can't see any actual payloads being multi-tenant. Applications by large run single tenant. The prevalent (and IMHO only current valid use case) is the administrative components that manages the rollout of the individual services. So instead of building that one huge system, you "only" need to have a management layer (and enough runtime).
Multi-Tenant architecture
The typical argument for a multitenancy deployment is rooted in the experience that platforms are expensive and hard to build. So when one platform is ready, it is better everybody uses it. With Virtual Machines, Containers and PaaS this argument becomes weaker and weaker. The added complexity in code outweighs the cost for spinning up another instance (which can be done in minutes).
A multi-instance architecture mitigates these efforts:
Multi-Instance architecture
The burden to manage the tenants lies with the Identity and Entitlement management. Quite often (see Bluemix) those are only accessed by admin and development personnel, so their load is lighter. So next time someone asks for multitenancy, send them thinking.

Posted by on 2015-11-04 12:36 | Comments (1) | categories: Software

The lowdown on Notes application web and mobile enablement

There are millions (if not billions) of lines of code written for the Notes client in small, large, simple, complex, epidermal and business critical applications. With the rise of browsers, tablets and mobile devices there is a need to web and mobile enable these applications, not at least since IBM won't let the C code of the Notes client run on iOS (which it could).
Whether we like it or not, that body of code can be considered technical debt.
  The strategy I see is mostly the same: classify applications by size, business criticality and perceived complexity. Then retire as much as possible and start a pilot for one (either insignificant or overly complex - mostly both) application using XPages. From there it goes nowhere.
Guess what. There's a better way.
Keep the part where you decide if an application can be retired. If you need to retain read access you could check out LDC Via, who remove the need to maintain a Domino infrastructure for those applications.
Then go and collect evidence (I know evidence based management isn't in fashion right now). There are elements in Notes that translate easily into web or mobile, parts that are hard and some stuff browsers can't do.
Collect how much you use of those, it makes decisions easier.

Design element and difficulty

  • View navigation:
    generally easy, its just data and Domino makes an excellent server for JSON data (I would render my own style, not the ?ReadViewEntries&Outputformat=JSON, but that's another story)
  • Categorised views:
    medium. The twisties we got so used to in Notes are not a native construct on the web (show me one that was web native). But with a little effort you can devise interaction patterns, go peek here.
  • View actions:
    HARD. They can be anything. Worst when code for front-end and back-end classes get mixed
  • Forms (read mode):
    Easy to medium. You could simply open a form using a browser and see something - or craft some XPage to show
  • Forms (edit mode):
    medium to hard (see below): You need to get your validation right and all the nice Hide-when conditions
  • RichText:
    HARD to impossible. RichText != HTML (and that's not even !==). For rendering you want to have Ben's enhancements as a start. RichText can contain attachments, wild formatting, OLE objects, Sections, Hidden stuff, tabbed or timed tables etc. All that doesn't really translate to HTML. To really know how RichText was used in your application and organisation you need to scan the actual data and analyse how RichText was used in each application
  • Code in forms:
    HARD. There is no LotusScript running in your browser, so you need to rewrite that logic. So better to know how many functions and lines of code you are looking at. Do your CoCoMo analysis. Code in fields (on enter/exit) needs to be re-though (nice word for retired)
  • Attachments:
    TEDIOUS. HTML doesn't know attachments, so you have to bridge between user expectations: can put an attachment anywhere in the text and the storage reality of HTML: simulate this by placing an image and a link. You might consider adding webDAV for a round trip editing experience
  • Reader & Author fields:
    Easy. Works
  • Encryption and Signature:
    (popular in approval applications) wait for Domino 9.0.2 - it supposedly will work there
  • Business Logic in hide-when formula:
    hard to find in Domino designer, DXL to the rescue
This list is probably not complete, but you get the gist

The approach

I would advocate (after the retirement of old applications):
  • Look at the whole application portfolio, not just one pilot. The best analogy: remember how long it took to drive your first 100m in a car (and I'm not talking about the 18 years wait to come of age). You had to take driving lessons (at least in civilized countries you have to), learn for the test, get a car (bargin with a parent?). So it took you around 20-40h, lets say: 20h. So you conclude to get to the supermarket, 3 km away it will take you 600h - so driving is out, you rather walk. Same applied to the "pilot". The first application is the hardest, so your experience wih it will screw the estimations.
    Furhermore looking at all applications gives you new synery potential and efficiency of scale
  • Create one module that handles view navigation for all applications (that probably would be a OSGi plug-in)
    When initially opening a document, it would point to a notes:// URL (you could have a little icon indicating that). Within a few weeks you have a working application giving some access to your Notes data (I would use JSON, AngularJS and Ionic for that app)
  • Next generate the forms using angular-formly or some {{mustache}} templates, so you have read-only access (DXL and XSLT skills come in handy)
  • Add a "request" functionality that lists the functions (Action buttons) an application once had, let users propose "I want this", so you get some priorities sorted out
  • Transit to contemporary development with a Domino backend
  • Do not fall into the trap: must look and work like a Notes client (we had that with the Java applets in R5 - last century). If someone is happy with the look of the Notes client - give them the IBM Client access plugin (PC only)

Why no magic button?

You could ask: why doesn't IBM provide this as an automatic module? After all they wrote LotusScript, the formula language and the Notes client. The short answer: IBM tried, we even had a prototype of LotusScript running on the JVM. The Notes client is such an incredible forgiving environment, any quality and mix of code "simply runs"™. I've seen code happily running (yes I'm talking about you 8000-lines-function) that was a melange of unspeakables.
Browsers on the other hand are fragile and unforgiving (not talking about quirks mode here). Any attempt to automate that never reached the stage of "point-and-shoot" - so it will not become an offering.

Why not just default and rewrite?

Good idea. Until you look closer. Domino's declarative security (ACL, Reader and Author fields anyone) is hard to replicate. The schema free Domino NoSQL store does neither map to Sharepoint nor an RDBMS (CouchDB could work), so you suddenly deal with data migration - a nightmare that can eat 80% of your total project cost. Lastly, the incremental approach outlined here allows to continue to use the applications (and eventually enhance them along the way), so disruption is minimised. Also the working Notes client provides a fallback for missing or malfunctioning features, this substantially lowers application risks.

Business partners to the rescue

You are not alone on this journey. IBM business partners and IBM Software Services for Collaboration are here to help. If you like the approach "I see Notes as one huge pool", you might want to talk to Redpill, they perfected this approach.

As usual YMMV.

Posted by on 2015-11-03 11:00 | Comments (5) | categories: IBM Notes

Domino, Extlib, GRUNT, JSON and Yeoman


With a few tweaks and clever setup, you can have web developers deliver front-ends for Domino without ever touching it.
Contemporary web development workflows separate front-end and back-end through a JSON API and HTTP (that's 21st century 3270 for you). The approach in these workflows is to treat the webserver as source of static files (HTML, CSS, JS) and JSON payload data being shuffled back and forth. This article describes how my development setup makes all this work with Domino and Domino designer.

The full monty

The front-end tools I use are:
  • Node.js: a JavaScript runtime based on Google's V8 engine. It provides all runtime for all the other tools. If you don't have a node.js installation on your developer workstation, where were you hiding the last 2 years?
  • Bower: a dependency manager for browser files like CSS, JS with their various frameworks. You add a depencency (e.g. Bootstrap) to the bower.json file and Bower will find the right version for you
  • Grunt: a task runner application, used for assembly of web sites/applications (or any other stuff). Configured using a Gruntfile.js: it can do all sorts of steps like: copy files, combine and minify, check code quality, run tests etc.
  • Yeoman: A application scaffolding tool. It puts all the tools and needed configurations together. It uses generators to blueprint different applications from classic web, to reveal application to mobile hybrid apps. The site lists hundreds of generators and you are invited to modifiy them or roll your own
  • GIT: the version control system
The list doesn't include one or the other editor, one or the other IDE or version control client, since theses choices don't influence the workflow steps or setup. Pick what you like.
My general directory layout starts with a project folder that has entries for code, documentation, nsf and a misc. folder. I only put the code folder into the GIT version control.
Seperation of concerns for frontend and backend development
The front-end developer doesn't need any of the Domino components available to create user interface and interaction models. The back-end developer will use exclusively the REST controls from the Domino XPages Extension library (which is part of a default current Domino server install). The two directory structures are linked in a GIT project, but that isn't mandatory. One step that made my live much easier: take the dist directory and link it to WebContent. This saves me the step to copy things around. The fine-tuning of the setup is where the fun starts.

Read more

Posted by on 2015-10-30 09:08 | Comments (3) | categories: XPages

There's a plug-in for that! Getting started with Cloud Foundry plug-ins

Since IBM Bluemix is build on top of Cloud Foundry, all the knowhow and tooling available for the later can be used in Bluemix too.
One of the tools I'm fond of is the Cloud Foundry command line cf. The tool is a thin veneer over the Cloud Foundry REST API and is written in GO. "Thin veneer" is a slight understatement, since the cf command line is powerful, convenient and - icing on the cake - extensible.
A list of current plug-ins can be found in the CF Plug-in directory. Now the installation instructions haven't kept up with the cf releases, so here are the steps you need:
  1. Head to the CF Command Line release page and make sure you have the latest release installed.
    At time of this writing that would be 6.12.2
  2. Add the community repository to your installation using this command:
    cf add-plugin-repo CF-Community http://plugins.cloudfoundry.org/
    This command is only available in cf versions > 6.10. A lot of blog entries or even the github documentation suggest downloads or even golang installs. With the availability of the repository these steps are no longer necessary (but you are free to use them)
  3. Now listing all available plug-ins is as simple as cf repo-plugins
  4. To install a specific plug-in you issue the command:
    cf install-plugin '[name of plugin as listed]' -r CF-Community
    If the name doesn't contain white space, the quotes can be omitted
  5. After installation cf help provides the short instructions on how to use the modules
The interesting question now is, what are the plug-ins worthwhile looking at.

Read more

Posted by on 2015-08-25 12:03 | Comments (0) | categories: Bluemix