Optimizing SharePoint 2010 Workflow execution

I need help optimizing my workflows to execute in less than 5 minutes.

Without getting into too much detail.  I have three lists; Master, POC and Checklist.
Each list has a workflow associated with it.  The POC list iniciates a looping process
that copies list items from Master into the Checklist list.  I had to introduce a delay
of 1 minute in the workflow to allow related list items to be copied successfully into the
new list items. This is really background information. The workflows do work and items are
copied into the Checklist.  The problem is that it only copies one item every 5 minutes.
Despite my changing the following items on the WFE:


stsadm -o setproperty -pn workflow-eventdelivery-throttle -pv "25"
This should allow up to 25 workflows to be executed at a time. (Originally 15)

Batch size:

stsadm -o setproperty -pn workitem-eventdelivery-batchsize -pv "125"
Increased my workflow batch size to 125 workflows. (Originally 100)

Time out:
I kept the same (5 minutes)

Workflow Timer Interval:

stsadm -o setproperty -pn job-workflow -pv 3 -url https://MyApplication_URL/ 
This should allow me to execute workflows in as little as 3 minutes. (Originally 5 minutes)

I verfied that each of these values were set.  
Before changing these values I was getting one item copied every 5 minutes.
After changing these value I get:

1. 3 min
2. 5 min
3. 4 min
4. 5 min
5-500. 5 min each

There appears to be another setting that moves my timer jobs to execute on 5 minute intervals that coincide with the clock (ie start at 3:33, next 3:36, next 3:41, next 3:45, next 3:50, 3:55, 4:00, 4:05, etc.). I need help getting my workflows
to execute in less than 5 minutes for each item copied.
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Try changing the workflow interval to 1 minute:
stsadm -o setproperty -pn job-workflow -pv “Every 1 minutes between 0 and 30″ -url http://<servername:host>/applicationname
macarrillo1Author Commented:
“Every 1 minutes between 0 and 30″

Won't that setting mean that my workflows will only work between 0-30 minutes of each hour?
The other half of the hour the workflows will not work.
macarrillo1Author Commented:
Ok, Made the change and have tested for a couple of hours.
Looks like I get a record (item) copied every 5 minutes.  There is no difference between the first half (0-30) of the hour and the second half (31-59).  The workflow copies a item every 5 minutes.

I did verify that the setting was changed on all WFE by using the following command:

stsadm -o getproperty -pn job-workflow -url https://MyApplication_URL/ 

<Property Exist="Yes" Value="Every 1 minutes between 0 and 30" />PS C:\Windows\system32>
Cloud Class® Course: Amazon Web Services - Basic

Are you thinking about creating an Amazon Web Services account for your business? Not sure where to start? In this course you’ll get an overview of the history of AWS and take a tour of their user interface.

Sorry cut it from my notes and didn't check it.  Think it should be this:

stsadm -o setproperty -pn job-workflow -pv “Every 1 minute between 0 and 59″ -url http://<servername:host>/applicationname  

However, re-reading your problem I can't see this making any difference if your problem is that only 1 workflow runs every 5 minutes.  You should get <WorkflowBatchSize> number of workflows running every every iteration.  

Can you run the WF manually and run two or more at once?
macarrillo1Author Commented:
On Tuesday, I can try again. I should be able to run workflows manually.  As you can see I have adjusted the WorkflowBatchSize but it only ever runs one at a time. Sigh! Although that is another problem.

For this problem.  When I initiate the workflow a record(list item) is immediately created. Five minutes latter another is created.  So the actual workflow can execute almost immediately.  So it is not that the workflow actually takes 5 minutes to execute. There is a pause in the workflow that is 1 minute to allow a related list to pull in data.  So I should be getting a record created every minute instead of every 5 minutes. There has to be another setting that controls how often a timer job fires.
Ah I think I see.  So your workflow iteratively processes the list, one item at a time, rather than one workflow item being created per item ?

In this scenario you will struggle to process more than one item every few minutes simply becuase the WF timer processes all working in units of minutes.
macarrillo1Author Commented:
That is correct.  And I am processing one record at a time because I am pulling data from a related list and for some reason it takes time (1-2 minutes) to make the relationship and pull the data from the related list.  Thus, the pause of 1 minute in the workflow.  So at best, I should be able to process one record per minute.  That is why I adjusted the Workflow Timer interval for the Web application to 3 minutes.  So I really should be processing records at 1 every 3 minutes.  But Instead, I am processing at 1 every 5 minutes.
In this case you would be better off using a batch process or powershell.

If you have developed your WF in Visual Studio add a clode block and process the other list in that.

AS a WF using delays you will never get it wokring much faster that it currently is.  This is becuase of the 1 minute unit in time delays.  You can try the seting I gave above to change the WF timer job to 1 minute but it will still be between 1 & 3 minutes between iterations.
macarrillo1Author Commented:
Unfortunatley, my environment and facility do not allow for Visual Studio Workflow development.  So I am limited to what I can do in SharePoint or SharePoint Designer 2010. And, yes, I realize that this workflow is less than optimal with an expected 1-3 minute/record creation. But that is the point; I should be getting records created at 1-3 minutes / records, but instead I am getting records created at 5 minute intervals consistently.

This is so frustrating, logically, I should be getting records at 1-3 minute intervals, not 5 minute intervals.  It is acting as if there is another overriding parameter that limits it to 5 minutes.

Sigh!  So colly92002, it is your opinion that this is as good as I can get it within the limits I have described?
As already suggested, the command:
stsadm -o setproperty -propertyname "job-workflow" -propertyvalue "every 1 minutes between 0 and 59" -url http://WebApplicationName

should set your SPTimer job to run every minute.  

This is likely to be the best you can do.  Note that throttling/WF delays/other time jobs will mean that the best you will ever get is more likely to be 3 minutes anyway.  The WF architecture is designed to work on batches of items at a time, in a non-time critical scenario (e.g sending and email and awaiting a response, or staritng an automation job on an item and wating for that to complete).

Could you change to your WF to work on a single item, but to be attached to an event reciever (i.e. on creation) ?  If it simply needs to do lots of processing in a loop on another list, then the only way is to do this in code, or wait for the SPTimer job to come back around as you are doing.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
macarrillo1Author Commented:
Thank You this is very helpful!
I had originally not had a pause and it would process 5 records before it started to fail.  Then I tried pausing every third record which worked better but was not consistent.  The problem comes about because I am using a related list and it takes time for the key to retrieve the data from the related list.  Your suggestion of using an 'on Creation' gives me the idea that I could initially copy the key into the new list item.  Then another workflow would be triggered with a 1 minute pause to retrieve the values from the related list. So the records could be created as fast as possible (no pauses) followed by a separate update with a pause.

In any event you provided me with several possible paths.

Thanks Again!
No problem.  As a final suggestion, could you incorprate a lookup to List2 as part of List1 or is it something that needs to be done automatically by the system?

As an aside, if you have access to Visual Studio and have any experience in building features, then building an event reciever to lookup values on creation is fairly trivial (much easier than WF).
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Server Apps

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.