Category Archives: Functional

Divide and conquer – processing high volume records within Dynamics CRM

Dynamics CRM is a great product for enterprises that want to efficiently track and optimise all interactions with their customers. Often there is a need to perform a particular operation on a batch of records. For example, in a customer-loyalty scenario, one might want to automatically re-evaluate a customer’s gold-star status on a nightly interval. In another scenario, one might want to recalculated an account balance based on the transactions of the previous day. In this post I will outline a robust design that lets a workflow be executed for an unlimited amount of records.

When working in CRM Online, we as CRM developers, vendors and partners have no control over the maximum allowed time that server side custom code components are allowed to execute within CRM. One of these limitations is that any custom workflow activity can run no longer than 2 minutes.

Next to the time limitation, there is also a concept of a depth limitation to prevent the system from containing malfunctioning code and ending up in an infinite loop. This limitation means that custom code calling itself or other custom code components get to do so at most 7 times every hour (for a maximum depth of 8).

These limitation makes the processing of large volumes of records from within custom code in CRM quite challenging.

The Microsoft Dynamics CRM 2015 Asynchronous Batch Process Solution is a good attempt at a framework to perform scheduled operations on a collection of records defined by a Fetch XML query. However, due to the aforementioned limitations of CRM, it is not a robust framework. We had implemented this solution at one of my client’s projects in CRM Online. It worked fine for low volumes of records, but we were only able to process 3000 records before CRM timed out and the batch job failed.

To overcome this problem, the client is usually presented with an external solution that processes these high volumes of records outside of CRM. In this article, I will present a design that will allow CRM to process a virtually unbounded amount of records based on a Fetch XML query.

Note, however, that CRM is not meant to act as batch processing system and thus it is still the responsibility of the system’s architect to determine if CRM is to be made responsible for the operation, or that an external system would be the best solution. In this article, we will assume that no more than 50.000 records will be scheduled in any given batch. This conveniently coincides with CRM’s maximum for the count aggregation. At the end of the article I will outline how to increase this number.

The basics

In this article, I’m extending the concepts used in the aforementioned Microsoft Dynamics CRM 2015 Asynchronous Batch Process Solution. This solution generally lets users schedule Workflow Processes in the following manner:

The user configures a batch of processes to be executed (i.e., configures a record of entity Batch). To achieve this the following fields are configured (some details are not relevant and are omitted):

  • A target workflow. This is the workflow that will be executed for every record that is scheduled in the batch.
  • An periodic interval at which the target workflow is to be executed on all records in the batch.
  • A valid Fetch XML query. This query is executed at the periodic interval and the result set defines all the target records for which the target workflow is executed.
  • Other details, like whether the batch is currently suspended or active, previous result sets, etc.

Upon scheduling a batch, a background process is automatically started. This process in turn starts the batch at the user configured time and interval. This triggers a custom workflow activity. (Further details of the implementation of this process are not relevant.)

The custom workflow activity reads the batch record configuration and uses the Fetch XML query to retrieve a list of target records for which to start. It then continues to fire an ExecuteWorkflowRequest message for the target workflow on all the target records.

In the environment that we used to test this solution, we had managed to start between 2000 and 3000 workflows before the workflow activity was timed out by the sandbox. Depending on your environment, this could potentially be a larger volume (or smaller!).

Regardless, this is not a trustworthy scheduler unless you are absolutely sure that the size of your scheduled batch never exceeds a low volume of target records.

Counting the scheduled records in the batch

To prevent the sandbox from timing out our scheduler, we will have to build a pipeline of sub-batch records that will perform the batch activation in an isolated manner.

Based on the resources available in your CRM organization you will have to chose an artificial limit of number of process activations that can safely be started from a custom workflow activity before hitting the 2 minute time-out limit. We had found that in our organization a limit of 250 process activations was a very conservative sub-batch size and we decided to go with that.

At the time of the scheduled start, rather than immediately starting the activation of target workflow processes, the scheduling framework first transforms the user inputted Fetch XML query to an aggregate count query. This can easily be achieved by adding the correct attributes to the Fetch XML document in c# code in combination with entity metadata. (I will discuss CRM’s 50k aggregate count limitation in a later section.)

After the amount of target records has been retrieved, the amount of sub-batches that we will create to split up the work can now be calculated as

[Total record count] / [sub-batch limit], rounded up to the nearest integer.

Say that we have a scenario where there are 13.433 records to be scheduled in an overnight batch. This would yield

Ceil (13433 / 250) = 54 sub-batches.

The initial workflow activity, rather than activating workflows for all of these 13422 records, will create 54 sequentially numbered sub-batches that are linked to each other in a pipeline manner.

The sub-batch entity

The sub-batches will contain the following details:

  • Lookup to the parent batch
  • Lookup to next sub-batch in the pipeline, or null when current record is the tail
  • Page number (‘order’), starting at number 1 for records 1-250, number 2 for records 251-500, etc.
  • Paging cookie (text field). Must be null upon creation of the sub-batch record.
  • A status reason with set of statuses, including ‘Waiting’, ‘Processing’, ‘Completed’ and ‘No More Results’
  • Lookup to a batch status record. This record can contain details about the progress of the entire batch, successes/failures etc. Details omitted for this article.

The worker process

A new worker workflow process is introduced that performs the workflow activations for the sub-batch. This worker process calls a custom workflow activity. We will discuss trigger conditions for this workflow in the next section.

When the current sub-batch has page number 1, the fetch query will be retrieved from the related batch configuration. Using c# code, inject the correct paging attributes to this Fetch XML definition.

For example, for batch number 1 and sub-batch size 250, this will be: count=250 page=1.

After the workflow activity has retrieved the entire page and subsequently activated the target process for all target records, the workflow activity will output 2 important arguments:

  • the returned paging cookie resulting from the execution of the Fetch XML
  • the value indicating if there are more records to be retrieved

The worker process checks the value of the outputted arguments. If there are no more records, and if the value of next sub-batch is not null, then the next sub-batch will be updated with a status of ‘No More Results’  (and this will in turn propagate throughout the pipeline).

If there are more records to be returned, and the next sub-batch does not equal null, then the worker process copies the outputted paging cookie to the designated field of the sub-batch that is next in the pipeline.

Overcoming the depth level restriction – the trigger for the worker workflow process

It is of utter importance that the trigger for the worker workflow process is the creation of the sub-batch record. This will make sure that the workflow processes are not killed by the platform due to iterative call depth.

The consequence of this trigger is that the worker process will need a wait condition, since none of the sub-batches have the correct paging cookie yet.

The waiting condition for the worker process is (OR):

  • Page Number == 1,  OR
  • Paging Cookie is not null, OR
  • Status is updated with ‘No More Results’

Since we are working with asynchronous processes and we need to guarantee the next sub-batch record is created before the worker process finishes its batch (to propagate the paging cookie to the next sub-batch and start its worker process), it is required that we create the batches in reverse order.

Sub-batch 54 is created, then sub-batch 53, 52, …, 2, 1.

All sub-batch worker processes that are now started due to the creation of these sub-batches go into a waiting state, except the worker process for the first sub-batch. This is due to the value of Page Number being 1.

This first sub-batch will run the fetch query for the first page without a paging cookie. Its paging cookie is propagated to the second sub-batch, triggering its worker process to start. Then sub-batch 3, 4, etc.

After the last sub-batch has finished execution, the entire batch can be updated as completed.

Extending the maximum size of scheduled records in a batch beyond 50k

It is easy to extend the maximum amount of records included within any scheduled batch to more than CRM’s 50k aggregate count limit. Simply execute the fetch aggregate, and when the result returns the dreaded 50.000 maximum count, this will need to be overridden in code to the new limit, say 100.000.

Rather than creating a pipeline of sub-batches based on the actual aggregate record count result, the amount sub-batches that will be created is based on the configured maximum of 100.000. In a scenario where the sub-batch size is equal to 250, a pipeline with a total of 400 sub-batch records will be created when the aggregate count result equals 50.000.

To minimise wasted resources, a new status reason of No More Results will need to be added on the sub-batch entity. When the paged fetch query is returned with a MoreRecords status of false, the next sub-batch in the pipeline will need to be updated with this status. The initial waiting condition of the sub-batch workflow can then be triggered to continue operation when a sub-batch is updated with a status reason of No More Results. If the workflow has bee restarted due to this new status, it will propagate this status to the next batch in the row and terminate immediately afterwards.

If the result set had somehow been modified during the execution of the batch, for example, when page 53 of the Fetch XML query is retrieved with More Record = False, then the next sub-batch will also be updated with a status of No More Results.

Error handling

Similar to propagating a No More Record status throughout the entire pipeline of waiting sub-batches, it is recommended to propagate any error states with an error status. This will require the custom workflow activity to output any unrecoverable error states to the sub-batch workflow.

Propagating this error state to all subsequent sub-batches in the pipeline will prevent the buildup over time of a large amount of workflows sitting idle in a waiting condition that will never be fulfilled.

Conclusion

With a proper design adhering to the fundamentals described in this article, it is very well possible to overcome the limitations imposed on custom workflows and code in Dynamics CRM Online and still manage to execute workflow processes on a large volume of records. However, every Dynamics CRM solution is unique and careful thoughts have to be given what system or solution best suits the task at hand.

 

This article is written by Tomas Schulkes. The design concepts can be freely used in any and all commercial and non-commercial software. The entire text and individual parts of the text is copyright by Tomas Schulkes and cannot be reproduced without my consent.

38 total views, no views today

Designing Maintainable Workflows in Microsoft Dynamics CRM

Below is an article I’ve written with Sander Bockting, a CRM Service Line colleague at Avanade, about designing maintainable workflows in Microsoft Dynamics CRM.

Designing Maintainable Workflows in Microsoft Dynamics CRM

The powerful and user-friendly workflow solution in Microsoft Dynamics CRM can be leveraged to easily build relatively straightforward business processes. However, when business processes become more complex, the effort required to translate them into a maintainable workflow increases disproportionately. This two-part article provides practical guidelines for translating complex business processes into maintainable, easily testable workflows.

Helping the business get their complex workflows

While working on a large enterprise Dynamics CRM solution, we were handed seven PowerPoint slides containing nearly one hundred human tasks. The tasks were part of two key business processes. Many of the tasks were to be executed in parallel, some were to be executed conditionally, and others functioned as approval tasks – only to be executed once the preceding tasks were canceled or completed. The processes even contained multiple recursions. Of course, each task had it is unique details, such as the owner to whom it should be assigned and due dates that were dependent on fields in CRM.

It quickly became apparent that simply building a large workflow would not result in a maintainable solution, should it even be possible at all. A more structured approach was required that would be resilient to change and where the workflow execution state could be understood at a glance.

In an attempt to make effective use of our computer science background, we decided to take a more academic approach to formalize complex business processes – written by domain experts – into a model decomposition where each model could be implemented using a CRM workflow.  Here we share our methodology and experiences.

Reduced maintainability of larger workflows

Dynamics CRM workflows provide great flexibility and extensibility for automation of system-driven business processes. Conditional checks, waits and branches provide for enough expressiveness to implement most business processes. Custom workflow activities can be leveraged when things get really complicated. These tools are especially applicable to smaller workflows; larger workflows tend to come with challenges in maintainability. Workflows can be considered maintainable when:

  • It is possible to quickly grasp its functional intent
    This becomes increasingly difficult when depth of the workflow steps increase or when there is recursion involved in the workflow steps. Recursion means steps may need to be looped until a certain condition is met.
  • Grasping the current execution state of a workflow (set) does not take much effort
    Dynamics CRM provides out-of-the-box functionality to see the current execution state for one workflow. But this does not help the system administrator in great detail when multiple workflows depend on each other.
  • Small changes do not impact a large portion of the workflow
    Many developers that have built workflows with CRM are familiar with the frustration where you have to delete and rebuild a workflow branch, because the “Insert Before step” did not do the trick in accommodating the required change.
  • Long running and complex (multi-staged) workflows can be changed without disrupting operation
    Without proper planning ahead, it may be necessary to let the old workflow instances finish while the new instances run in parallel. This is not always acceptable for the business.

Approach

To overcome the maintainability issues of larger workflows and to help in the process of implementing complex business processes, we have adopted a six-staged structured methodology:

  1. Capture the desired functionality of the workflows in a model using a functional language that can be easily understood by the customer and the developers.
  2. The functional model is used to break up the monolithic large workflow into smaller and more maintainable workflows.
  3. Create a status record that tracks completion details of the individual workflows. The record shows the total progress of the combined workflow.
  4. Each of those workflows is described in a technical model.
  5. Each technical model allows for translation into an actual CRM workflow.
  6. Create the Starter Workflow to initialize the composite workflow.

A more detailed explanation of each stage is given below.

Stage 1: Design Functional Model

The idea for a workflow frequently arises at non-technical users. Our experience is that these users are perfectly capable of creating a schematic view of their desired business processes. Their schematic views can be used to discuss the flow and act as a starting point for the technical workflow. Among many different customers, we have found that they all came up with comparable notations. We collected them into a uniform set of functional process symbols.

Functional Process Symbols

Functional process symbols describe the business process from a functional point of view. The representation is used as an input for the translation from a business process into a workflow. The symbols should be seen as an accessible notation for the customer to define their processes. Although not obligatory, getting a design based on these symbols may speed up your work in translating it into a technical design.

To increase diagram readability, we chose to let the customer describe details of actions or tasks (for example, start date, priority, subject, etc.) in a separate document. Letters and digits were used as a reference between the two design artifacts.

crm-workflow-functional-symbols

Figure 1: Legend of functional process symbols

Symbol Definition
Start of diagram Indicates start of the business process.
State A state indicates a task or action to perform or finish to be able to take the next transition.
Branch A branch models the paths and their preconditions that could be traversed. Only 1 path can be traversed simultaneously. A branch does not represent an action or task.
Transition A transition models a possible one-way path from one state to another or to a branch. The transition could hold a guard that models the precondition required to traverse the path. For example, the completion of a task, the passing of a certain amount of time, etc.
Parallel activity streams start This one-to-many fork models multiple activity paths that take place in parallel.
Parallel activity streams convergence This many-to-one fork models the convergence of multiple parallel paths. The outgoing path can only be traversed when all parallel paths are finished.

Using Functional Process Symbols to create schematic views

The customer uses the functional process symbols to create schematic views of their business processes. The most common situations and their compositions are provided below.

Situation description Schematic view
Sequential execution of workflow items.  Sequential execution of workflow itemsSequential execution of workflow itemsSequential execution of workflow itemseersteSequential execution of workflow itemsParallel execution of workflow items
Parallel execution of workflow items.  Parallel execution of workflow items
Repeat a workflow item until a condition is met (e.g. ensure that an end-user entered a valid value in a record).  Repeat a workflow item until a condition is met
The first state is a task or an action to be performed by a person. The second tasks is assigned to a supervising entity, and is meant to make a judgment about the outcome of the first state. If undesirable, the state or action is to be performed again (e.g. useful in an approval flow).  Repeat a set of workflow items until a workflow item was completed successfully
Repeat a set of workflow items from the beginning each time the outcome of a task or action was not desirable (e.g. a complex approval flow with multiple approvers).  Repeat a set of workflow items from the beginning each time a task was not completed successfully
Wait until a specific point in time is reached.  Wait until a specific point in time is reached

Stage 2: Box Identification – Identifying child workflows

When a schematic view of the desired business process is complete, the next step is to break up the view into smaller, workable components. The task is to identify the areas that must be modeled as child workflows within the main workflow to encapsulate the entire business process. These areas are called boxes. The procedure of finding them in a business process flow is called box identification. Box identification is characterized by finding the right start and end of a box.  Let’s look at the mechanics of this process in more detail, as well as an example.

Finding the start of a box

A box should have one incoming and one outgoing transition. Breaking up the business process into smaller boxes is done by traversing the business process path from the start. Three situations have been identified as good fits for a new box. While traversing the path:

  • A branch element is encountered.
  • A state is found that has an incoming transition from a state that has not been traversed yet; the transition is a recursive call from a child element.

Finding the end of a box

The end of the box depends on the situation that started the creation of a box.

  1. For a box that was created after encountering a branch element, the end of it is marked by the first transition that is traversed by all possible paths that started after the branch element.
  2. For a box identified by a recursive call, its end is identified recursively:
    1. Include all possible states between the start state of the box and the state making the recursive call to the start state, including the start state and the state making the recursive call.
    2. If this set of states has any state making a recursive call to a state that is not yet in the box, repeat step A for these states, also including them in the box.
    3. Repeat this process until there are no more recursive calls found.

The resulting set of states together form the box.

Box identification example

Box identification exampleFigure 3: An example of box identification that breaks down one large functional process design into smaller boxes. Each box eventually results in a CRM workflow.

Figure 3 shows a large functional process design. The black box covers the entire business process or Complete Workflow; it is the main workflow and will be the first box (1) (see Figure 4). Within the black box, traverse the path until the branch element is found, indicating the need for a new box (2): Subflow A. The end of Subflow A is found by the first single transition going out of the box.

Breakdown into three boxesFigure 4: Breakdown into three boxes

Within Subflow A, observe there is yet another branch element in the business process. The blue box indicates a child workflow (Subflow A’) of Subflow A: box 3. The end of Subflow A’ is also found by identifying the single arrow going out of the box.

Stage 3: Design and Build Status Record Entity

Due to the lack of a native CRM mechanism to have a workflow pause while a child workflow runs, we have devised our own method to track the execution state of a child workflow. We have done this by means of defining a custom entity for the composite workflow, called the status record entity. The idea is to have this status record entity contain a number of datetime fields equal to the number of workflows identified in stage 2 that are to be waited upon. The names of these fields on the status record entity should preferably be descriptive for the child workflow that is to be waited upon for better readability.

After initiating a child workflow, the parent workflow will pause until the status record field corresponding to the child workflow is filled with a datetime.

After execution, the child workflow will set the value of its status record field to the current date and time and then end. This signals the parent workflow to resume duty.

The primary entity of the workflow is linked to the status record by means of a new field preferably hidden from the entities forms. The status record is initiated at the start of the overhead workflow.

It can frequently occur that a parent workflow resume after waiting (because the child workflow has set the date and time in the appropriate field), and see that the result of the child workflow is not satisfactory. When the child workflow is to be run again, the value in the status record field should be emptied before initiating the child workflow.

Note that when a child workflow is a ‘fire and forget’ workflow, it will not need representation in the status record entity.

Stage 4: Create a technical model for each identified box

The previous stages described the identification of boxes that function as separate (child) workflows and the creation of a status record entity that maintains the connection between each of the (child) workflows. Next, each box is translated into a technical model. Just like the Functional Process Symbols are used to create functional models, the Technical Process Symbols are used to create technical blueprints of the workflows.

Technical Process Symbols

Besides functional process symbols – to capture business processes in diagrams interpretable by your customer – a set of technical process symbols is given below. The symbols serve as intermediate language and allow you to translate the functional diagrams into technical diagrams in a more formal way. They provide just the right amount of expressivity to create diagrams for implementable workflows.

Technical Process SymbolsFigure 5: Legend of technical process symbols

Symbol Definition
Workflow start Indicates start of workflow.
Recursive Call Recursive call to current workflow.
Workflow End Indicates end of workflow.
Record Update / Create Operation Models a create operation of a data model record (for example, a Task) or an update operation on a data model record.
Status Record Update Models an update operation on the status record related to the primary entity.
Composite State The Composite state models a child workflow. It is a call to a child workflow.
Branch A branch models the paths and their preconditions that could to be traversed. Only 1 path can be traversed simultaneously. A branch does not represent an action or task.
Guard on workflow records The guard describes the condition that must have been met in order to traverse the path it guards. A guard can consist of multiple conditions joined by logical operators. A guard can also be a mix of workflow record guards and status record guards, however the labels will then be written in blue to show the guard’s dependency on the status record.
Guard on status record field(s) The guard describes the condition on one or multiple fields of the status record that must have been met in order to traverse the path it guards. A guard can consist of multiple conditions joined by logical operators. A guard can also be a mix of workflow record guards and status record guards.
Parallel paths Parallel paths show actions and states that take place in parallel.

Creating technical models using the symbols

For each identified box, a technical model is created using the Technical Process Symbols. An example is given that shows how a parent workflow (Workflow X) is connected to a child workflow (Workflow Y). With the use of a status record, Workflow X will only continue processing once Workflow Y filled the appropriate status record field with a date.

Workflow XFigure 6: Workflow X – it depends on a subworkflow Y that should be finished before it can continue. It waits until the appropriate status record field contains data.

Workflow YFigure 7: Workflow Y stores the current date and time in a status record field when it has successfully reached the end. Workflow X waits until the appropriate status record field contains data.

Stage 5: Translate technical models into CRM workflows using the editor

Translating the technical models created in stage 4 into CRM workflows should be pretty straightforward if a sufficient breakdown of boxes has been made. It is assumed the reader has a good understanding of creating processes in Dynamics CRM 2011. If not, the internet has various tutorials on how to build a workflow.

All child workflows in the composite workflow need to be defined for the same primary entity. When creating the workflows in Dynamics CRM, be sure to uncheck the ‘Start when’ options. Also, make the workflows ‘available to run’ only as ‘child workflow’, unless you are testing or debugging a specific child workflow. In this case, do not forget to manually attach a new Status record. Otherwise you run the risk of either not having a Status record attached yet, or that some datetime fields have already been filled with data.

Translation of the technical symbols is done with the following components.

  1. The ‘recursive call’ symbol is implemented by the Start Child Workflow step, calling itself as a child workflow.
  2. The ‘status record update’ symbol is implemented by the Update Record step.
  3. The ‘record update / create operation’ symbol is implemented by either the Create Record step or the Update Record step.
    1. If the create operation is used and the outgoing transition of this state has a guard, the ‘record create operation’ is followed by a wait step on the guard.
    2. The wait step is omitted in case the outgoing transition of the composite state is connected directly to the end point of a ‘parallel paths’ step.
  4. The ‘composite state’ symbol represents a child workflow and is therefore implemented by a Start Child Workflow step, starting the appropriate child workflow.
    1. If the outgoing transition of this composite state has a guard, the Start Child Workflow step is followed by a wait step on the guard.
    2. This wait step is omitted in case the outgoing transition of the composite state is connected directly to the end point of a ‘parallel paths’ step.
  5. The ‘branch’ symbol is implemented by a Check Condition step.
  6. The ‘guard on workflow records’ and ‘guard on status record field(s)’ symbols are implemented by a Wait Condition step, waiting on the designated field to fulfill the guard defined on the step.
  7. The ‘parallel paths’ symbol is implemented by creating all of the steps for the individual paths and then defined a wait step that guards all of the waiting conditions in each path. In case an individual path contains more than one symbol, it should be boxed into a new workflow as described previously.
  8. The ‘workflow start’ symbol needs no translation.
  9. The ‘workflow end’ symbol is implemented by the Stop Workflow step. If this is the last step of the workflow, it does not need to be created.

Stage 6: Build Workflow Starter

The sixth and final stage is to build the workflow that starts the composite workflow. Now that all of the workflows have been designed and defined, a starter workflow must be constructed. This starter workflow is the workflow that will be defined to run as the initial workflow for the primary entity.

When run, the starter workflow constructs a new instance of the Status Record entity. After creating the new Status record, the Workflow Starter starts the workflow that is defined for the outer most box of the business model. This, in essence, starts the desired workflow.

In our solution, the starter workflow is also used as a prerequisites test. It tests if the conditions are met that allow the workflow to run. Although not required, it could abstract the actual workflow from the logic that checks whether or not the workflow should be triggered to run. One could even imagine defining one starter workflow for starting one or more workflows from a set of possible workflows.

Discussion

The introduced workflow design methodology introduces several major advantages. It allows for special constructs that are not possible with large monolithic workflows such as loops. It also allows for easier maintenance when workflows should be changed. Changes are limited to shorter-running workflows, which allows the support team to replace these workflows by new versions without disrupting the long running process. When making changes to child workflows, it is required to have a thorough understanding of the dependencies between the different child workflows, and this is where the technical designs come in play. If the workflows would not have been split up into smaller components, it would have been a lot more difficult to modify the large workflow and several changes may require the removal of entire branches (e.g. when an extra branch condition should be added before a tree of workflow items). The smaller child workflows also improve testing possibilities because smaller sets of workflows can be tested individually and with less associated test conditions.

Each set of workflows requires a new entity in the form of a workflow status record. Although this implies additional work, the workflow status record allows for direct insight workflow execution progress, removing the need for tooling such as BAM (Business Activity Monitoring) when there are no complex requirements in this respect. And when requirements become more complex, the status record can be leveraged to measure workflow execution durations.

The design methodology lowers the threshold to create complex workflows in Dynamics CRM. However, workflows are still complex so it is wise to be cautious to prevent certain problems:

  • It is a Microsoft best practice to create one single long workflow instead of child workflows called by a parent workflow when it comes to maximum throughput[1]. If workflow definitions do not change frequently and the workflows are not that complex that they should be split up into separate workflows, it is worth considering one single workflow.
  • An important step in the design methodology is decomposing the large functional model into smaller child workflows through the process of box identification. When mistakes are made during this boxing process, it is possible that you end up finding yourself unable to create the actual Dynamics CRM workflows because a recursive call was required at a level where it cannot be implemented through standard Dynamics CRM workflow steps. In that case, all stages starting from stage 2 will have to be redone for the affected workflows.
  • To reduce complexity, this methodology ‘forces’ you to create many child workflows. This could mean that you run into the Maximum Workflow Depth (default set at 8), meaning that your workflows will stop when a chain of child workflows is called. When this occurs, increase the Maximum Workflow Depth[2]. This can be done using PowerShell.

Summary

This article described a structured approach to translate complex business processes into maintainable workflows. Two modelling notations were introduced that can be used to create a functional model and a technical model. A methodology was given to translate the functional model into a technical model. Finally, this technical model is the basis for the actual Dynamics CRM workflow set implementation. The child workflows have dependencies with each other that are managed through a status record. The status record makes it easy to grasp the current execution state of the workflow by looking which child workflow completed last. This setup ensures more flexibility; a business process change likely affects a (small number of) child workflow(s), minimizing the need to modify a long running workflow with the corresponding maintenance support issues.


[1] http://msdn.microsoft.com/en-us/library/gg509027.aspx

[2] http://msdn.microsoft.com/en-us/library/microsoft.xrm.sdk.remoteexecutioncontext.depth.aspx