Workflow and Organisational Directory

An interesting question was raised in a recent development discussion: " What is the best way to keep the organisation data of our workflow application(s)?" My first answer was easy: "Definitely not inside the application itself". But then the choices start (too many of them are not good for you): Keep them in the Domino directory, in a custom Notes application, in a RDBMS, pull from HR, pull from LDAP? Each of these approaches do have their merit and their drawbacks. I'll elaborate.
Of course the first question to ask: Should I build my own workflow engine? There are a number of choices readily available. You can start with Ironworkflows on OpenNTF which even provides a graphical workflow editor running in the browser (and full source code included). You could have a look at XFlow, that will gain with some help based on a a great library a graphical editor in due time. It also can work with different engines. If you are not into "complete source provided" type of applications, you can checkout PAVONE Espresso Workflow which runs both on Domino and Websphere and got a nice graphical editor too. The graphical editors seem to be a must in these days, but might only determine a fraction of the usefulness of a workflow engine (I know that statement is a candidate for flame proved underwear).
But back to the initial question: Where to store the organisation data? If you build your own engine I would strongly suggest, regardless of where you store, to abstract workflow logic from storage. Define the API your application will call (like getWFDefinition(...), getApprovers(...), resolveRole(...) etc. What calls would make sense in an engine warrants a post on its own. Then code against the API (this is called contract first programming). This way it doesn't matter if your storage backend changes and you can keep the considerations for storage separate from the considerations for your applications. In Domino such an API would be best implemented using the Extensibility API (that's what XFlow is doing). Now to the storage options:

• Domino directory (my preference)
  If there wasn't THE ADMIN the Domino directory is an excellent choice. It has ample fields to capture organisational information like department, managers etc. You even have well defined extension points (yes: places where you legally can mess with the directory).
  Big advantage: easy to sync (we give you TDI), available for other applications via LDAP and when a user goes the roles go with him. That is the general idea of a directory service.
  Biggest challenge: the admins jealously guard the directory and won't let you get anywhere near it to customise or update data. Cited reason: "If you mess with the directory, IBM won't support us" and "We can't grant the access rights without risking other parts of the data being messed up". There *is* a slight difference between altering the directory design (=fun with support) and updating the schema. Dealing with the schema (and LDAP) requires to leave the protected yellow bubble for a moment and learn about the finer details of LDAP (e.g. I didn't know that you can't use # for a group name, but & is allowed).
  Open to discussion: how well does Domino Directory serve to capture all the different roles? I've seen very few engines using the directory alone. What I have seen is the Domino directory in use for lookups, but a custom front application for data entry, so the workflow owners can't mess with other data in the directory. Also popular: sync the Domino directory using TDI with an external backend like the HR system
   
• Custom Notes application
  That's the most prevalent form of corporate storage for organisation data. The original Prozessware used that approach as do the tools mentioned above. Clear advantage is the flexibility of a solution, since you can tailor it to whatever you fancy without the risks perceived in adding to the Domino directory. Of course you loose the exposure via LDAP, which makes it a solution confined to Domino (unless you add some web services, which would be a good idea) and you need to keep the Domino directory in sync with the org database. What I've seen very often are org directories that synchronise with all sorts of backends: RDBMS, HR Systems, web services etc. This of course opens the eternal question of sync vs. direct access
   
• RDBMS
  As long as you look for a Domino solution there is little reason to entertain a RDBMS (unless of course your architect was hiding under a rock and thinks NoSQL isn't a database). So typically this approach can be found where e.g. HR systems run on different platforms. I'm not a big fan of direct RDBMS access since it ties you down to a rigid structure and, more important, adds another breaking point: if your RDBMS is down your workflow stops, so you need to queue your apps for processing. You can sync the data or at least use LEI/DECS to avoid language rojak
   
• Pull from HR
  A good HR system would have a higher level API (probably a web service) that allows to pull org information by your workflow engine. While more business like than the RDBMS approach these APIs are rarer. If your HR system has one - good for you, use it. Again you need to decide sync vs. direct access
   
• LDAP
  If your LDAP server isn't plagued with a fragile engine it is a good place to store org data in it. LDAP can be queried by a lot of platforms and is language neutral. In Domino LDAP goes well with XPages (I've seen a LDAP data source somewhere) and Java, but not so much with LotusScript. Of course if you use Domino Directory as (one of your) LDAP engine(s) you get the best of both. A good idea when working LDAP (Domino or others) is to have a sound LDAP front-end to view and enter data. Apache's Directory Studio is a good choice here

I'll cover more thoughts on what the org directory should provide in a later post. As usual YMMV.