Need basic information about Lotus Notes

Posted on 2007-10-18
Last Modified: 2013-12-18
I am having performance issues in Lotus Notes/Domino.  The users are complaining about slow operation of both Notes and a custom CRM they purchased.  I know Exchange.. Domino is very new to me.

Can you please tell me:
1. How to determine the number of email messages in each inbox ( all I see is the number of unread messages).
2. How to compress the email files to improve performance.

I know there are obviously tons of other possible issues, but I need to take one step at a time.  I can't find anywhaere for training that doesn't cost an arm and a leg.
Question by:AdminAssociates
    LVL 63

    Expert Comment

    Some of the info you need may be in th catalog.nsf on the server.

    TO compress emails use the compact command..
    see the Notes Admin client Help


    I would set it up to run nightly/ weekly depending on server usage.
    I hope this helps !
    LVL 31

    Expert Comment

    Open the server's log file (log.nsf), and go to the Database\Sizes view. This gives you the file size in kilobytes as well as the whitespace size of each file.

    You can drill down into each entry.  When you open up an entry, you can see how many folders there are, and how much storage space each foler occupies.  If your users replicate (like cached mode in Outlook), however, the folder sizes will all be zero, because folder cointents are only built out when used, and a replicatd user only uses the local copy of the folder, not the servfer copy of the folder.

    Catalog.nsf provides similar data., but the information on the mail files is buried. Catalog includes the number of documents (messages, calendaring items, system tracking records) in the file.

    You can run the following simple script to get actual inbox message counts, assuming you have access to all mailboxes. Caution: it may buold out the indexes of the Inbox folders for those replicated users, which might take a long time and spike your server CPU and cause an expansion of allocated disk space.

    Const Server = "name of the server"
    open "C:\inboxes.txt" for output as #1
    print #1 , "filename" , "inbox" , "all" , "title"
    Dim databases as new notesDbDirectory(server)
    dim db as notesdatabase, folder as notesView, count as long
    set db = databases.getFirstDatabase
    do until db is nothing
      count = count #1
      print count , db.fileName
      if not db.isOpen then"","")
      if db.isOpen then
        set folder = db.getView( {($Inbox)} )
        if folder is nothing then
          print #1 , db.filePath , -1 , db.allDocuments.Count , db.Title , "No inbox"
          print #1 , db.filePath , folder.entryCount , db.allDocuments.Count , db.Title , "No inbox"
        end if
        print #1 , db.filePath , -1 , -1, "Unable to open file"
      end if
      set db = databases.getNextDatabase
    close #1
    LVL 4

    Expert Comment

    Could you give us some server specs. (cpu, memory and probably most important - disk configuration (raid, san, mirror ?), free diskspace and total storage capacity).

    How many users are we talkin about ? And all on one server ?

    Mailfiles should be extremely large to influence on performance to a degree where users complaint about them...

    Performance should not be a problem - so i'm quite sure we can figure out the reason for your problems.


    Author Comment

    Yes Brian... I suppose some Specs would help... Sorry for being dense :)

    The server is dedicated to Lotus Notes/Domino and is running MS 2003 Server R2 with SP2

    CPU: Quad-Core Xenon x5355 processor running at 2.66 GHz
    RAM: 4 Gig Kingston DDR2
    Drives:   5 SATA II drives in the following configurations:
      (2) 150 GB drives on the On-boiard Intel 631xEBS/632xESB Raid Controller configured as RAID 1
            These are the OS drives and the Domino Program.  There is 141 GB free of the 150 GB Capacity.

     (3)  250 GB drives on a 3WARE 9550SX RAID controller configured as RAID 5
           This is where the databases reside -  There is 434 GB free of the  465 GB (after format) Capacity

    The NIC is an Intel Pro/1000 EB with I/O Acceleration. However, they only have a 10/100 MB switch.

    The cabling is all certified CAT5. N.E.X.T. is low and a sniffer reports hardly any packet collisions.
    All of the workstations are fairly new with 2.X GHz processors and at least 512 MB RAM.

    There are only ten users on this system.

    Mail file sizes are:
    User 1     -    17,549  using  2.015 GB
    User 2    -     32,635           1.807 GB
    User 3  -        9,569              793 MB
    User 4    -    28,775           1.885 GB
    User 5        -  1,433              406 MB
    User 6     -     1,978              179 MB
    User 7 -         1,760              166 MB
    User 8    -      1,224              245 MB
    Users 9 and 10 are my account and my Nptesadministrator account, which have no mail in the Inboxes and <150 MB file sizes each.

    From what I understand, IBM suggests keeping the Inboxes to less than 1000 emails each, so that may possibly be part of the trouble.

    There is a custom built CRM program/database that uses about  3.5 GB and two other client databases of about 1 GB each.  That's it... the machine is so underutilized it almost echo's.

    Network speeds for the other parts of the network including file DC/File Server access ( with several SQL databases on it) is quite acceptable.  The only lag is in Notes/Domino.

    However as I stated in my original post, I know Exchange, but hardly anything about Lotus Notes.  What I do know is, if I had an Exchange server with this hardware and only 10 users, I would expect nose-bleed speeds which is what the client was expecing for his dollar.  Instead they feel they are constantly waiting for the system to keep up/catch up.

    Any suggestions would be appreciated.   Thanx !
    LVL 31

    Expert Comment

    First place I always look is how Domino is utilizing memory and cache.  That's the most frequent problem. I use that alongside Windows indicators (system memory usage, CPU, perfmon write queues).

    And by the way, yes, smaller inboxes give you better performance, but not at all in this range.  It affects three things:
    1) How fast router can dleiver messages, since each delivery operation is also an "add to folder" operation on the inbox
    2) Speed of inbox builds on open -- when a folder is opened, if items have been added since its display list was last refreshed, it has to update that display list (the visible part of the index), and since the display part is an inherent part of the sorted index structure, this also takes indexing time
    3) Speed of display of existing inbox (i.e., even if unchanged isnce last opened, or the time it takes after #2 completes if there is something new); there is more I/O and processing to figure out what data to give back to the client when the inbox is larger

    Note that #1 is invisible to the user, except as it affects overall system performance, while #2 and #3 are directly visible to the user.  AFter a few seconds, if the client has not received and processed everything it needs to display teh inbox, it presents that "ugly white updating/why don't you do other stuff while I navel gaze" screen.

    I would guess that the CRM is a bigger culprit here, because they tend to have lots of indexes that get updated en masse.

    I would also consider "hiding" the problem.  It sounds as if users work directly off the server, instead of rpelicating.  That's like turning off cached mode in OL.  It doesn;t really solve the problem,  but it is something I would do even if you were nothaving problems.

    However, I would only expect that to make a difference on an already loaded system.  You have few users, and up to date specs.  Each of the factors above is relatively small, and tends to only be affected cumulatively across a large number of users.  And you can easily prove this, by creating a new folder, "Old Inbox," for users 1 and 4, and moving everything but the last couple hundred messages from the real inbox to the new folder.  I don't think you'll see a blip of difference.
    LVL 63

    Expert Comment

    Certainly seems like a nice fast machine.

    Have you run perfMon in log mode to get a baseline, and see if it is reasonable. ?

    What do the Lotus Domino - Monitoring results DB show.

    Make sure you have event monitoring turned on.

    Certainly should not be slow...

    LVL 4

    Accepted Solution

    I agree with qwaltee and SysExpert... The hardware is fast enough - for comparison (one of our servers is a dual Pentium III 800 MHz, 4 gb of RAM and 400 gb of disks [28 disks in a RAID 5]). It supports close to 1000 users without a problem, so 10 users on your hardware :-)

    If you check your Task Manager and then add the column "CPU time", could you give us a top 5 of the most cpu intensive processes ? This might lead to what area of the server is causing problems...

    My best guess is that the problem is caused by a performance bottleneck in your disk-subsystem, most likely caused by a improperly designed CRM application (or one of the other 2 applications). Domino is a "disk-intensive" application, and your RAID does only have 3 drive spindles - it might not matter (and apparently I would say that it doesn't).

    To identify disk-performance problems, look at the server statistics concerning disk queue length... If I remember correctly does a value of more than 1 indicate a performance problem with the disk.

    Also - make sure that your Antivirus software (the filesystem one) does not scan the Domino databases... NO filesystem scanner will be able to scan a Domino database for viruses - so wasting cpu etc. on scanning will only make performance worse.

    Finally (but one thing that doesn't matter in your installation) - RAID 5 is not an optimal diskconfiguration when considering performance on a database system - RAID 1 gives you way better peformance (but is also more expensive).

    Author Comment

    Well..   The top 5 contestants according to CPU Time are:

    Obviously, we ignore "system Idle Time" which is by far the highest number.

    Then we have:
    1) nupdate.exe
    2) rtvscan.exe ( I am not scanning .nsf or .ft files... just the mail...should I change this?)
    3) svchost.exe
    4) nrouter.exe
    5) nadminp.exe

    PS:  I had never heard that a mirror set was more efficient than a striped set with parity,  Are you sure that is true?

    Thanx !

    Author Comment

    Checking the Disk Queue Length I find:
    average:  0.020
    Maximum :  0.397
    LVL 4

    Expert Comment

    OK... nupdate.exe is responsible for rebuilding view indexes in databases on the Domino server - how much CPU time has this process used compared to the servers uptime ?

    I'm not sure what you mean with 2)... what mail are you scanning ? If you are referring to mail*.box then just exclude them... They are databases as well, in which the filesystem scanner cannot find viruses.

    Disk is apparently not your problem - but RAID1 is faster than RAID5 (and I should have said RAID1+0 instead of just RAID1 - a mirrored stripeset) - depending on the storage capacity required. Of course RAID 0 is the fastest, but also provides no security what so ever.

    RAID1 used both drives for reading and only have a slight overhead on disk writes... RAID5 has slower write rates because of the parity calculations needed.

    All in all, in your system the problem is theoretical :-) So never mind, but if you start using a SAN or have many local disks - the configuration matters.

    Just occured to me now as well.... Make sure to disable scanning of Notes on your clients as well ... That is extremely important. I believe the rtvscan.exe is part of Symantec Antivirus ? This product integrates with Notes via an extension manager (that is the notes term :-)), and scans all communication between the client and the server - it performs very poor (not just Symantec, but every implementation).

    To make sure that it is disabled - try to edit notes.ini on a client and find these lines:


    make a note of them, and delete the lines from the file... Restart the Notes client and see how it performs.
    LVL 31

    Expert Comment

    RAID 5 is fine for your size installation.  And the queue length should never be more than 1+# of usable spindles.  In a 3-drive RAID 5 config, that would mean that you would prefer queue length under 3 and would definitely have perofrmance issues on queue length greater than 4, but you have less than 1 (no surprise under this load), so that's not the problem.

    I agree about the virus scanning.  For a test, why don't you just turn it off altoegther for a few hours.  It is not really protecting Domino, it is protecting the OS, and if you have that fairly locked down and AV running elsewhere throughout your org, the risk should be very low.

    I would look at memory pools again.

    Featured Post

    How your wiki can always stay up-to-date

    Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
    - Increase transparency
    - Onboard new hires faster
    - Access from mobile/offline

    Join & Write a Comment

    This is an old article, please see an updated version of this article, located here:
    I thought it will be a good idea to make a post as it will help in case someone else faces these issues. I trust this gives an idea how each entry in Notes.ini can mean a lot for the Domino Server to be functioning properly. This article discusses t…
    Hi everyone! This is Experts Exchange customer support.  This quick video will show you how to change your primary email address.  If you have any questions, then please Write a Comment below!
    This video discusses moving either the default database or any database to a new volume.

    732 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    17 Experts available now in Live!

    Get 1:1 Help Now