Solved

Problem with Domino 8.5.2 Agent running very slowly

Posted on 2010-09-20
12
956 Views
Last Modified: 2013-12-18
We have an IBM Lotus Domino agent that updates 6400 documents running on a schedule.

When you run this agent on Domino 7.02 it takes under 10 minutes, on Domino 8.5.2 it times out after 35 minutes and has only updated 4800 documents.

We have now increased the run time to 45 minutes to see if it will get through, however I would really like to get to the root of the problem. We have already recompiled this agent under Domino 8.5.2 and the problem is still there.

The server is Windows 2008 standard SP2.

Thanks
0
Comment
Question by:johnstockbridge
  • 4
  • 3
  • 3
  • +2
12 Comments
 
LVL 22

Expert Comment

by:mbonaci
ID: 33723137
I'm afraid that, without the agent's code, we (myself at least) are not able to give you suggestions on how to make it faster.
That's too much time for updating 6k documents on any version, so I assume that the code can be optimized.

Are you, by any chance, using NotesDocument's ComputeWithForm method?
0
 

Author Comment

by:johnstockbridge
ID: 33723154
Thanks for your assistance. Here is a copy of the agent - there is nothing complicated about it. make it ru on a 7.02 server and it works fine. Bring it back to 8.5.2 and it times out.

Sub Initialize
       
    Dim s As New notessession
    Dim db As notesdatabase
    Set db = s.currentdatabase
   
    Dim docSchedule As notesdocument    
    Dim item As notesitem
    Dim strFullDate As String, strSelectedDate As String, strToday As String    
    Dim intDateDiff As Integer, cnt As Integer, intCompletedCnt As Integer, intUpdatedCnt As Integer
       
    Print "Starting Update Schedule Offsets agent"
   
    'Get a view of current schedule docs
    Dim viewSchedule As notesview
    Set viewSchedule = db.GetView("(All Staff Schedules Update)")    
    Call viewSchedule.Refresh
    viewSchedule.AutoUpdate = False
       
    'Get the current date    
    strToday = Cstr(Today())
    Dim dateToday As New NotesDateTime( strToday )
   
    'Initialize some counters
    intCompletedCnt = 0
    intUpdatedCnt = 0
    cnt = 0
   
    'get the first schedule doc
    Set docSchedule = viewSchedule.GetFirstDocument
   
    'update the date offsets on each document in the view of current schedule documents
    While Not (docSchedule Is Nothing)
        'update the document count
        cnt = cnt + 1        
        'calculate the date offset for the date on the schedule document eg today=0, tomorrow=1, nextday=2, etc
        strSelectedDate = docSchedule.StaffScheduleDate(0)        
        intDateDiff = Cdat(strSelectedDate) - Cdat(strToday)
        If intDateDiff < -14 Then
            'schedule is older than two weeks so drop it out of the schedules view
            Set item = docSchedule.ReplaceItemValue("scheduleCompleted", "Y")
            Set item = docSchedule.ReplaceItemValue("DateOffset", "-100")
            intCompletedCnt = intCompletedCnt + 1
        Else
            'update the offset
            Set item = docSchedule.ReplaceItemValue("DateOffset",Cstr(intDateDiff) )
            intUpdatedCnt =  intUpdatedCnt + 1
        End If        

        'save the schedule doc
        Call docSchedule.Save(True, False, True)
               
        'get the next schedule doc
        Set docSchedule = viewSchedule.GetNextDocument(docSchedule)
       
    Wend
           
    'call a manual view refresh
    Call viewSchedule.Refresh
   
    Print "Finished Update Schedule Offsets agent. Processed " + Cstr(cnt) + " documents. Updated " + Cstr(intUpdatedCnt) + ". Completed " + Cstr(intCompletedCnt)
   
End Sub
0
 
LVL 10

Expert Comment

by:larsberntrop
ID: 33723562
first:, whenever changing the doc, I do: see example 1

second, I have a hunch that the selection formula for the view (All Staff Schedules Update) uses scheduleCompleted or DateOffset to exclude documents

anyway, I would recode to collect all the updates into a list of NotesDocumentCollection objects, and use the StampAll method to batch update the documents needing change.
See Example 2

Let us know how it performs

any difference between the 7.0.2 and 8.5.2 server other than de Domino version? as in OS, antivrus, specs? Or did you upgrade the 7.0.2 server with 8.5.2?


' Example 1 - document loop
Do Until doc Is Nothing
    Set docNext = view.GetNextDocument(doc)

    ' ... process & save doc

    Set doc = docNext
Loop

' Example 2 - NotesDocumentCollection
Dim l_dc List As notesDocumentCollection

Do Until doc Is Nothing
    Set docNext = view.GetNextDocument(doc)
    ' insert intDiff calculation
    If intDateDiff < -14 Then
        addDoc l_dc, -15, doc
    Else
        addDoc l_dc, intDiff, doc
    End If
    Set doc = docNext
Loop
Forall dc in l_dc
If ListTag(dc) = "-15" Then
    intCompletedCnt = dc.Count
    dc.StampAll "scheduleCompleted", "Y"
    dc.StampAll "DateOffset", "-100"
Else
    intUpdatedCnt =  intUpdatedCnt + dc.Count
    dc.StampAll "DateOffset", ListTag(dc)
End If

End Sub
Sub addDoc(l_dc List As NotesDocumentCollection, index As Integer, doc As NotesDocument)
    If Not IsElement(l_dc(index)) Then
        Set l_dc(index) = db.CreateDocumentCollection
    End If
    l_dc(index).AddDocument doc
End Sub

Open in new window

0
 
LVL 10

Expert Comment

by:doninja
ID: 33723760
I would look at above examples to streamline.

From my own experience I did notice some differences in versions when trying to use .autoupdate = false in code.
It seems to wait for the view to update when makign a change even tho autoupdate is false. it does not update the contents in the stored view but seems to just take longer to get the next document.
As a simplification I found using code similar to above example 1 and not using .autoupdate=false sped things up.

also if you have upgraded the database versions look if the database has any properites changes such as compressing documents etc. This could be increasing CPU load above 7.0.2 and causing the bottleneck rather than disk IO, although that would be the reverse of normal :)
0
 
LVL 22

Expert Comment

by:mbonaci
ID: 33724035
I don't see a problem in the code. It's very clean.

@larsberntrop
I always use "Until", but as far as performance is concerned

    While Not Something

is the same thing as

    Do Until Something

I like the idea of stamping the collection, but it would only work for the first part of If statement (with Else part you wont be able to do this because intDateDiff is different for every document - see bellow).

@johnstockbridge
Do you know how to create empty NotesDocumentCollection?
If not follow this link (or see the example in the code section):
http://www.e-e.com/Q_21622223.html
        If intDateDiff < -14 Then

            'schedule is older than two weeks so drop it out of the schedules view

            'add doc to collection

            intCompletedCnt = intCompletedCnt + 1

        Else

            'update the offset

            Set item = docSchedule.ReplaceItemValue("DateOffset",Cstr(intDateDiff) )

            intUpdatedCnt =  intUpdatedCnt + 1

        End If





Utility method that creates empty NotesDocumentCollection object:

Function initCollection() As NotesDocumentCollection

	Dim s As New NotesSession

	Dim db As NotesDatabase

	Dim c As NotesDocumentCollection

	Dim dummyDoc As NotesDocument

     

	Set db = s.CurrentDatabase

	Set dummyDoc = db.GetProfileDocument( "dummyProfile" )

	Set c = dummyDoc.Responses

	Set initCollection = c

End Function

Open in new window

0
 
LVL 10

Expert Comment

by:larsberntrop
ID: 33724185
Since a 8.5.2 server is available, I use the simple way of creating an empty NotesDocumentCollection: db.CreateDocumentCollection

I like using until, cause I like to use do Loop instead of whil wend.  Also save a Not, i like positive thinking!

I gather from your reaction a misunderstanding.  I first loop through all the documents, building the different document collections.  As many collections are created for the different intDiff values that are encountered, with exception of values of intDiff < -14, which are replaced with -15. Then I simply process all elemnts of the list, stamping the documents.  -15 is treated differently, as in the code given in the question.  I do see I left out an 'End ForAll',
but hey, it's an example...


' Example 1 - document loop
Do Until doc Is Nothing
    Set docNext = view.GetNextDocument(doc)

    ' ... process & save doc

    Set doc = docNext
Loop

' Example 2 - NotesDocumentCollection
Dim l_dc List As notesDocumentCollection

Do Until doc Is Nothing
    Set docNext = view.GetNextDocument(doc)
    ' insert intDiff calculation
    If intDateDiff < -14 Then
        addDoc l_dc, -15, doc
    Else
        addDoc l_dc, intDiff, doc
    End If
    Set doc = docNext
Loop
Forall dc in l_dc
    If ListTag(dc) = "-15" Then
        intCompletedCnt = dc.Count
        dc.StampAll "scheduleCompleted", "Y"
        dc.StampAll "DateOffset", "-100"
    Else
        intUpdatedCnt =  intUpdatedCnt + dc.Count
        dc.StampAll "DateOffset", ListTag(dc)
    End If
End ForAll

End Sub
Sub addDoc(l_dc List As NotesDocumentCollection, index As Integer, doc As NotesDocument)
    If Not IsElement(l_dc(index)) Then
        Set l_dc(index) = db.CreateDocumentCollection
    End If
    l_dc(index).AddDocument doc
End Sub

Open in new window

0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 10

Expert Comment

by:larsberntrop
ID: 33724189
db.CreateDocumentCollection is used inth r8 mail template, just waiting to be documented...
0
 
LVL 22

Expert Comment

by:mbonaci
ID: 33724321
Create the collection for every unique intDateDiff?
Depends on the document data, of course, but I have a feeling that would take longer than the current code.
0
 
LVL 10

Expert Comment

by:larsberntrop
ID: 33724750
why do you have that feeling? First we read all the necessary data in memory, and then do updates which are batched.  Should be loads faster.  

I suggest we let the requestor implement it, and he reports the time. Then we will know.
0
 

Author Comment

by:johnstockbridge
ID: 33724885
Thank you all for your assistance, we will try out the various solutions suggested and let you know.

It is night-time here in Australia so it will be a few hours yet :-)
0
 

Accepted Solution

by:
johnstockbridge earned 0 total points
ID: 33805916
Thanks to all who responded to this question, however the answer was to upgrade the databases to ODS51 if you have upgraded to Domino 8.5.2 otherwise some agents will not run properly.

Just for information, the old agent on 7.02 took 25 minutes to run, the same agent running under 8.5.2 with databases upgraded to ODS51 now takes 3 minutes.
0
 
LVL 46

Expert Comment

by:Sjef Bosman
ID: 33813460
Just some remarks...

1. Why the agent? Why do you need the DateOffset field in the document? Is it used elsewhere in the database's code? There's no way you can ditch that field?

2. Is the database full-text search enabled? If so, you can do a simple FTSearch to find the documents that need to be modified, and use NotesDocumentCollection.StampAll

3. To speed things up even more, try also setting
    viewSchedule.AutoUpdate= False
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

For users on the Lotus Notes 8 Standard client, this article provides information on checking the Java Heap size and adjusting it to half of your system RAM in attempt to get the Lotus Notes 8.x Standard client to run faster.  I've had to exercise t…
Article by: Rob
Notes 8.5 Archiving Steps and Tips This article covers setting up a Notes archive, and helps understand some of the menu choices making setting up and maintaining a Notes archive file easier.
This tutorial gives a high-level tour of the interface of Marketo (a marketing automation tool to help businesses track and engage prospective customers and drive them to purchase). You will see the main areas including Marketing Activities, Design …
Windows 10 is mostly good. However the one thing that annoys me is how many clicks you have to do to dial a VPN connection. You have to go to settings from the start menu, (2 clicks), Network and Internet (1 click), Click VPN (another click) then fi…

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now