I spend a lot of my time developing workflows and forms (in InfoPath) to replace existing paper-based processes. As such I have been through a lot of learning experiences, some of them rather painful, when it comes to successful implementation. There are plenty of technical articles out there about how to develop a workflow in the SharePoint environment, SharePoint is an extremely well 'socialised' development environment with people giving away superb 'free' advice and code all over the place. Google, of course, is your friend here (or whatever search engine you've sworn undying allegiance to in the battle to win the war of all wars: which free text box we type our search terms into.)
For me, the hallmark of good workflow design allows for
traceability, meaning that you can quickly and independently of the SharePoint environment deliver real-time status and progress information on workflow progress;
modularity (being able to 'chunk out' workflow sections where you need similar functionality in a range of areas) and
sustainability throughout the (re)development process. This last one is critical in the initial stages of workflow release, because it is in the nature of process mapping that intricacies are missed and certain specific variations unaccounted for.
Sustainability.
Lets face it, nothing is more annoying (and yet nothing should be less surprising) than crafting a nice neat workflow, having it tested (supposedly) to user satisfaction and then hearing "ah, we forgot about this case, because they hardly ever come up" or "we weren't at this stage of the cycle when we were thinking about this, so we forgot that between January and June we deal with them differently." No matter how good your up front business analysis is, common sense (and human nature) suggests that you
will run into these situations and if you haven't developed your workflow with the capacity for it to restart and pick up where it left off, then you will immediately break or damage all existing workflows when you deploy the new version that takes into account the latest remembered foible. Obviously you want to make sure that your process mapping and review methodologies are as robust as possible, but not planning for potential missed cases is asking for trouble. Sure, you could deploy the new version under a different solution package but this then means it must have a different name (you can't have two workflows on a single list/site with the same friendly name), different workflow guid and all sorts of other things that mess with your ability to provide good tracing and performance indication data. I resolve this by ensuring all workflows I develop are 'self-aware' enough that they can tell, from the state of the workflow form, where they need to begin from in the event of being restarted. They also know how to maintain their relationship with the tracing data in the event of a restart such that you don't get 'data gaps' where a restart has taken place. This is one of the key relationships in workflow implementation that is often forgotten: that between form and workflow logic. Form design is seen as important from a UI perspective but it is not necessarily seen as a way of persisting 'dehydrated' wf data sufficient to pick up where we left off if, for whatever reason, a workflow needs to be restarted.
Modularity is hard to deliver in SharePoint workflow development (although you can cut and paste from one designer window to another in VS, I wouldn't recommend it, the order of parts and how various conditional logic works seems to get a bit screwed up and patching back in all of your declarative conditions takes a while). However whilst the actual design surface can be a pain to modularise, there's no excuse for not developing a good 'helper' class for common functionality and ensuring that your business objects are developed to support liaison between them and the workflow layer. If you're keen on delivering high performance KPI-style data to a usable front end, then that means developing a class for handling interactions between your workflow and tracing environments.
Additionally, if you're going to be using the functionality a lot, then
developing a custom workflow action, might be the way to go.
One key factor in delivering both restartability and modularity is getting to grips with state machine worklows (
2007,
2010). They allow far better flexibility in dealing with 'odd' restart situations and allow you to design a workflow, have it be 'corrected' and not have to completely re-do your process flow logic. Additionally, you can only really do repeating tasks and achieve useful functionality such as workflow suspension (there's a requirement local to me that workflows can be hibernated for a dynamically set period of time from any stage of the workflow) and 'sending back' (users love being able to send something back a stage, or two stages and let someone know they haven't done their 'bit' right) when you delve into state machine workflows. They make each part of a workflow a modular component that can be run through an indefinite number of times and suddenly give you an immense amount of flexibility when it comes to developing truly 'business process like' workflow solutions.
Traceability. So, why send SharePoint hosted workflow data outside of SharePoint for the purpose of delivering information on status and progress of workflows? Surely, I hear you all not asking, that's mad and I should just let people check the status of workflows using native SharePoint data? Well, if you are developing workflows where all of your business data is similarly hosted in SP lists or tied just to SP documents then that's not the worst idea in the world.
There are still some hurdles to overcome when it comes to making sure users can find that data quickly (drilling down to task-level information for a SP workflow takes way too many mouse-clicks and knowledge of where the 'workflows' link resides) and there's some help on delivering that already available out there on
various blogs (this blog uses jQuery to start a workflow 'on page', the same approach can grab more detailed information on workflows and display it 'in line' with other data).
When it comes to delivering what I'd call 'real world' workflow solutions, you often want to combine SharePoint workflow data with LOB data and then surface it not only in your SharePoint environment, but probably elsewhere as well. In my case a lot of the data needs to be combined and presented within SharePoint as part of a LOB application, but also within SharePoint as part of our Reporting Services infrastructure. Now, I know it's technically possible to use SP web services with SSRS to produce reports that draw from both SQL LOB data and SP wf data, but I'll tell you right now that from both a performance and support standpoint it certainly isn't desirable. Additionally (and this is a gotcha that appears to have been cleared up in SP2010, though not really to satisfaction) you have the issue of longevity of SP workflow related data. SP likes to
clean up after itself and even where data does persist, it is not in a form that any normal end user would comprehend. SP lists (to a lesser extent in v2010) also have certain performance issues relating to large numbers of items being returned in a single query and, over time, all workflows would likely run into them (unless they're exceptionally low volume).
As such I persist a set of workflow related status data in SQL and log workflow progress as it goes along and then use this to tie together workflow progress and other LOB data to produce usable reports, KPIs, and tracing information as part of the LOB applications I deliver entirely within SharePoint. I can then also use this same data to expose information relating to SP-based workflows and processes to non-SP environments more straightforwardly than it might be attempting to get those other application environments to "play nice" with SP.
So, what does this mean?
How this pans out in practical implementation and development terms is something I'll cover in a future post. The key messages I'm trying to get across here is that sometimes you need to think beyond SharePoint to deliver truly useful SP-based applications and that, especially with workflow development, you need to approach your implementation in a way that recognises some of the shortcomings of the SharePoint workflow deployment model, such that you don't have to reinvent the wheel just because an outlying case rears its head three months after full release.
- rob
