Flow, Rules, Complexity and Simplicity in Workflow

When I make the claim " Most workflows are simple", in return I'm hit with bewildered looks and the assertion: " No, ours are quite complex". My little provocation is quite deliberate, since it serves as an opening gambit to discuss the relation between flow, rules and lookups. All workflows begin rather simple. I'll take a travel approval workflow as a sample (resemblance of workflows of existing companies would be pure coincidence).

The explanation is simple: " You request approval from management, finance and compliance. When all say OK, you are good to go". Translating them into a flow diagram is quite easy:

With a simple straight line, one might wonder what is the fuzz about workflow systems and all the big diagrams are all about. So when looking closer, each box "hides" quite some complexity. Depending on different factors some of the approval are automatic. " When we travel to a customer to sign a sizable deal and we are inside our budget allocation, we don't need to ask finance". So the workflow looks a little more complex actually:

The above diagram shows the flow in greater detail, with a series of decision points, both inside a functional unit (management, finance and compliance) as well as between them. The question still stays quite high level. "required Y/N", "1st line manager approves". Since the drawing tools make it easy - and it looks impressive - the temptation to draw further lures strongly. You could easily model the quest for the manager inside a workflow designer. The result might look a little like this:

Of course that looks a little silly, after all a manager can be looked up from a (LDAP) directory, so (s)he doesn't need to be hard-coded. Typically the roles of people are handled by a role resolver. When it comes to the rules, the flow editors often ends up containing hardcoded rules. Hardcoding (even in a shiny picture) creates technical debt, you will, one day, pay for. Looking at the sample flow below, you find the tip of the iceberg how complex Rules expressed as flows can become:

It boils down to a series of design questions, that can greatly impact the performance and maintainability of your workflow system. Whenever "lets-draw-it-all-in-a-big-picture knocks", so send him to play scratch or PicoCricket. Here is my take:

• Flows should kept lean. That makes them easy to grasp and simple to visualize. Have a high level process that resembles the general flow, which each module being self contained. I wouldn't model "Is that module required Y/N" into such a main flow, but have the "skip me" logic left to the module
• Never put a name into a flow. This is what roles are for. A good role resolver that is aware of hierarchies (corporate, functional and project hierarchies) and can handle delegation well (per period, per application, per decision type). If a call var approver=resolveRole(curApp,curStep,metaData) can do the trick, I don't need to draw a huge graphic with decision points for: is delegation active, is person available etc.
• Loops are your friend. In a lot of flow situations the number of approval levels depends on data conditions. So you start drawing flows that look absolut identical after each other, only the amount/duration changes. A loop with values fetched can be easily reconfigured without redrawing (and redeploying) a flow definition
• Don't model the HOW of a decision into a flow, but the what. "Do I need finance approval" to be asked to the finance sub module, rather than "is this customer travel and if yes below 5k and if no, has it been budgeted"
• The natural companion of a workflow engine is a rules engine. Moving the rules that get evaluated from a workflow diagram into a rule engine allows for leaner flows and better management. Rules generally are independet from the processes, at least when you follow the Rules manifesto
• There is no hard and fast rule where your modeling in the flow designer ends and rules/discrete code should start. It is a function of experience and IT capabilities. Having an easy way to link modules, like an enterprise service bus makes complex systems easier to handle
• There is a huge difference between self contained workflows (stuff you would build in IBM Notes or MS-Sharepoint), handling state withing one application, and workflow orchestrating systems (like the IBM Business Process Manager) that coordinate the collaboration of many distinct systems

As usual YMMV