Solved

Problem with Domino 8.5.2 Agent running very slowly

Posted on 2010-09-20
12
951 Views
Last Modified: 2013-12-18
We have an IBM Lotus Domino agent that updates 6400 documents running on a schedule.

When you run this agent on Domino 7.02 it takes under 10 minutes, on Domino 8.5.2 it times out after 35 minutes and has only updated 4800 documents.

We have now increased the run time to 45 minutes to see if it will get through, however I would really like to get to the root of the problem. We have already recompiled this agent under Domino 8.5.2 and the problem is still there.

The server is Windows 2008 standard SP2.

Thanks
0
Comment
Question by:johnstockbridge
  • 4
  • 3
  • 3
  • +2
12 Comments
 
LVL 22

Expert Comment

by:mbonaci
Comment Utility
I'm afraid that, without the agent's code, we (myself at least) are not able to give you suggestions on how to make it faster.
That's too much time for updating 6k documents on any version, so I assume that the code can be optimized.

Are you, by any chance, using NotesDocument's ComputeWithForm method?
0
 

Author Comment

by:johnstockbridge
Comment Utility
Thanks for your assistance. Here is a copy of the agent - there is nothing complicated about it. make it ru on a 7.02 server and it works fine. Bring it back to 8.5.2 and it times out.

Sub Initialize
       
    Dim s As New notessession
    Dim db As notesdatabase
    Set db = s.currentdatabase
   
    Dim docSchedule As notesdocument    
    Dim item As notesitem
    Dim strFullDate As String, strSelectedDate As String, strToday As String    
    Dim intDateDiff As Integer, cnt As Integer, intCompletedCnt As Integer, intUpdatedCnt As Integer
       
    Print "Starting Update Schedule Offsets agent"
   
    'Get a view of current schedule docs
    Dim viewSchedule As notesview
    Set viewSchedule = db.GetView("(All Staff Schedules Update)")    
    Call viewSchedule.Refresh
    viewSchedule.AutoUpdate = False
       
    'Get the current date    
    strToday = Cstr(Today())
    Dim dateToday As New NotesDateTime( strToday )
   
    'Initialize some counters
    intCompletedCnt = 0
    intUpdatedCnt = 0
    cnt = 0
   
    'get the first schedule doc
    Set docSchedule = viewSchedule.GetFirstDocument
   
    'update the date offsets on each document in the view of current schedule documents
    While Not (docSchedule Is Nothing)
        'update the document count
        cnt = cnt + 1        
        'calculate the date offset for the date on the schedule document eg today=0, tomorrow=1, nextday=2, etc
        strSelectedDate = docSchedule.StaffScheduleDate(0)        
        intDateDiff = Cdat(strSelectedDate) - Cdat(strToday)
        If intDateDiff < -14 Then
            'schedule is older than two weeks so drop it out of the schedules view
            Set item = docSchedule.ReplaceItemValue("scheduleCompleted", "Y")
            Set item = docSchedule.ReplaceItemValue("DateOffset", "-100")
            intCompletedCnt = intCompletedCnt + 1
        Else
            'update the offset
            Set item = docSchedule.ReplaceItemValue("DateOffset",Cstr(intDateDiff) )
            intUpdatedCnt =  intUpdatedCnt + 1
        End If        

        'save the schedule doc
        Call docSchedule.Save(True, False, True)
               
        'get the next schedule doc
        Set docSchedule = viewSchedule.GetNextDocument(docSchedule)
       
    Wend
           
    'call a manual view refresh
    Call viewSchedule.Refresh
   
    Print "Finished Update Schedule Offsets agent. Processed " + Cstr(cnt) + " documents. Updated " + Cstr(intUpdatedCnt) + ". Completed " + Cstr(intCompletedCnt)
   
End Sub
0
 
LVL 10

Expert Comment

by:larsberntrop
Comment Utility
first:, whenever changing the doc, I do: see example 1

second, I have a hunch that the selection formula for the view (All Staff Schedules Update) uses scheduleCompleted or DateOffset to exclude documents

anyway, I would recode to collect all the updates into a list of NotesDocumentCollection objects, and use the StampAll method to batch update the documents needing change.
See Example 2

Let us know how it performs

any difference between the 7.0.2 and 8.5.2 server other than de Domino version? as in OS, antivrus, specs? Or did you upgrade the 7.0.2 server with 8.5.2?


' Example 1 - document loop
Do Until doc Is Nothing
    Set docNext = view.GetNextDocument(doc)

    ' ... process & save doc

    Set doc = docNext
Loop

' Example 2 - NotesDocumentCollection
Dim l_dc List As notesDocumentCollection

Do Until doc Is Nothing
    Set docNext = view.GetNextDocument(doc)
    ' insert intDiff calculation
    If intDateDiff < -14 Then
        addDoc l_dc, -15, doc
    Else
        addDoc l_dc, intDiff, doc
    End If
    Set doc = docNext
Loop
Forall dc in l_dc
If ListTag(dc) = "-15" Then
    intCompletedCnt = dc.Count
    dc.StampAll "scheduleCompleted", "Y"
    dc.StampAll "DateOffset", "-100"
Else
    intUpdatedCnt =  intUpdatedCnt + dc.Count
    dc.StampAll "DateOffset", ListTag(dc)
End If

End Sub
Sub addDoc(l_dc List As NotesDocumentCollection, index As Integer, doc As NotesDocument)
    If Not IsElement(l_dc(index)) Then
        Set l_dc(index) = db.CreateDocumentCollection
    End If
    l_dc(index).AddDocument doc
End Sub

Open in new window

0
 
LVL 10

Expert Comment

by:doninja
Comment Utility
I would look at above examples to streamline.

From my own experience I did notice some differences in versions when trying to use .autoupdate = false in code.
It seems to wait for the view to update when makign a change even tho autoupdate is false. it does not update the contents in the stored view but seems to just take longer to get the next document.
As a simplification I found using code similar to above example 1 and not using .autoupdate=false sped things up.

also if you have upgraded the database versions look if the database has any properites changes such as compressing documents etc. This could be increasing CPU load above 7.0.2 and causing the bottleneck rather than disk IO, although that would be the reverse of normal :)
0
 
LVL 22

Expert Comment

by:mbonaci
Comment Utility
I don't see a problem in the code. It's very clean.

@larsberntrop
I always use "Until", but as far as performance is concerned

    While Not Something

is the same thing as

    Do Until Something

I like the idea of stamping the collection, but it would only work for the first part of If statement (with Else part you wont be able to do this because intDateDiff is different for every document - see bellow).

@johnstockbridge
Do you know how to create empty NotesDocumentCollection?
If not follow this link (or see the example in the code section):
http://www.e-e.com/Q_21622223.html
        If intDateDiff < -14 Then

            'schedule is older than two weeks so drop it out of the schedules view

            'add doc to collection

            intCompletedCnt = intCompletedCnt + 1

        Else

            'update the offset

            Set item = docSchedule.ReplaceItemValue("DateOffset",Cstr(intDateDiff) )

            intUpdatedCnt =  intUpdatedCnt + 1

        End If





Utility method that creates empty NotesDocumentCollection object:

Function initCollection() As NotesDocumentCollection

	Dim s As New NotesSession

	Dim db As NotesDatabase

	Dim c As NotesDocumentCollection

	Dim dummyDoc As NotesDocument

     

	Set db = s.CurrentDatabase

	Set dummyDoc = db.GetProfileDocument( "dummyProfile" )

	Set c = dummyDoc.Responses

	Set initCollection = c

End Function

Open in new window

0
 
LVL 10

Expert Comment

by:larsberntrop
Comment Utility
Since a 8.5.2 server is available, I use the simple way of creating an empty NotesDocumentCollection: db.CreateDocumentCollection

I like using until, cause I like to use do Loop instead of whil wend.  Also save a Not, i like positive thinking!

I gather from your reaction a misunderstanding.  I first loop through all the documents, building the different document collections.  As many collections are created for the different intDiff values that are encountered, with exception of values of intDiff < -14, which are replaced with -15. Then I simply process all elemnts of the list, stamping the documents.  -15 is treated differently, as in the code given in the question.  I do see I left out an 'End ForAll',
but hey, it's an example...


' Example 1 - document loop
Do Until doc Is Nothing
    Set docNext = view.GetNextDocument(doc)

    ' ... process & save doc

    Set doc = docNext
Loop

' Example 2 - NotesDocumentCollection
Dim l_dc List As notesDocumentCollection

Do Until doc Is Nothing
    Set docNext = view.GetNextDocument(doc)
    ' insert intDiff calculation
    If intDateDiff < -14 Then
        addDoc l_dc, -15, doc
    Else
        addDoc l_dc, intDiff, doc
    End If
    Set doc = docNext
Loop
Forall dc in l_dc
    If ListTag(dc) = "-15" Then
        intCompletedCnt = dc.Count
        dc.StampAll "scheduleCompleted", "Y"
        dc.StampAll "DateOffset", "-100"
    Else
        intUpdatedCnt =  intUpdatedCnt + dc.Count
        dc.StampAll "DateOffset", ListTag(dc)
    End If
End ForAll

End Sub
Sub addDoc(l_dc List As NotesDocumentCollection, index As Integer, doc As NotesDocument)
    If Not IsElement(l_dc(index)) Then
        Set l_dc(index) = db.CreateDocumentCollection
    End If
    l_dc(index).AddDocument doc
End Sub

Open in new window

0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 10

Expert Comment

by:larsberntrop
Comment Utility
db.CreateDocumentCollection is used inth r8 mail template, just waiting to be documented...
0
 
LVL 22

Expert Comment

by:mbonaci
Comment Utility
Create the collection for every unique intDateDiff?
Depends on the document data, of course, but I have a feeling that would take longer than the current code.
0
 
LVL 10

Expert Comment

by:larsberntrop
Comment Utility
why do you have that feeling? First we read all the necessary data in memory, and then do updates which are batched.  Should be loads faster.  

I suggest we let the requestor implement it, and he reports the time. Then we will know.
0
 

Author Comment

by:johnstockbridge
Comment Utility
Thank you all for your assistance, we will try out the various solutions suggested and let you know.

It is night-time here in Australia so it will be a few hours yet :-)
0
 

Accepted Solution

by:
johnstockbridge earned 0 total points
Comment Utility
Thanks to all who responded to this question, however the answer was to upgrade the databases to ODS51 if you have upgraded to Domino 8.5.2 otherwise some agents will not run properly.

Just for information, the old agent on 7.02 took 25 minutes to run, the same agent running under 8.5.2 with databases upgraded to ODS51 now takes 3 minutes.
0
 
LVL 46

Expert Comment

by:Sjef Bosman
Comment Utility
Just some remarks...

1. Why the agent? Why do you need the DateOffset field in the document? Is it used elsewhere in the database's code? There's no way you can ditch that field?

2. Is the database full-text search enabled? If so, you can do a simple FTSearch to find the documents that need to be modified, and use NotesDocumentCollection.StampAll

3. To speed things up even more, try also setting
    viewSchedule.AutoUpdate= False
0

Featured Post

Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

Join & Write a Comment

For users on the Lotus Notes 8 Standard client, this article provides information on checking the Java Heap size and adjusting it to half of your system RAM in attempt to get the Lotus Notes 8.x Standard client to run faster.  I've had to exercise t…
  In today’s Arena we can’t imagine our lives without Internet as we are highly used to of it. If we consider our life style just for only 2 min we found that face to face communication is swapped by e-communication.  Every Where from Works place to…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
This video shows how to remove a single email address from the Outlook 2010 Auto Suggestion memory. NOTE: For Outlook 2016 and 2013 perform the exact same steps. Open a new email: Click the New email button in Outlook. Start typing the address: …

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

6 Experts available now in Live!

Get 1:1 Help Now