asked on

Lotus Domino Slow Access to View - HDD shows peak access

Hi All,
I have a custom Lotus Domino Application. Changing to particular views is REALLLLY slow. As in 1min 4 seconds. The views that take a long time have children documents. If I select all the documents in that view there is ~16,000 docs ( inc. children )

I thought Lotus Notes opened just enough documents for the viewing pane, and collected more from the server as you paged down? So I thought this wouldn't be a problem. If I remove the children docs from the view it performs well.

If I put the database local on my workstation accessing ( with the children documents ) it takes 7 seconds; That's what I want but from the server - but currently I'm at 1min 4 secs!

OK then: Why is it so slow?

a. Network
Came in at 10pm when there is next to no activity on our network. Same problem -- virtually the same access speed for the database.... Viewing the network activity graph in Windows 2003 Small Business Server confirms this low level of activity --- no bottle neck. Viewing the performance of each network CARD also shows relatively little activity.

b. Processor
Processor graph shows very little.. 20% / 30% usage. It's a pretty good HP server 6 cpu's or something like that.

c. HDD
Bingo.... HDD activity is shown peaking for 5 seconds at a time when accessing the view(s) in question. ( ie. 100% usage ). The actual label on the graph for HDD activity is 'Disk Queue Length'' It get's absolutely hammered!

My workstation doesn't have this problem though?? The server is raid 5 - as a trial I put an IDE drive in the HP server ( as my workstation was in that config ) and run Lotus Domino off it's own HDD... Still the same problem ( actually a few seconds slower - still a max-out on the HDD access on the performance graph ).

What am I missing :)

Many thanks for your assistance,
Adel

320adel

ASKER

A little more info... Another view that we use takes 39 seconds to display... When I expand the categories there are just 109 documents... Here is the server HDD activity while I access this view. Maxing out the HDD!!

What's going on I have spent days on this now. Would love some input / ideas.

hddaccess.png

CRAK

Could be the indexer. Does the view contain any system date/time related evaluations (selection/query)?

320adel

ASKER

View 1 in Question:
SELECT ((Form = "Contact" & Relationship = "Client") | @AllChildren) & DeleteFlag = ""

View 2:
SELECT ((Form = "Contact" & Relationship = "Prospect" & ProspectCheck != "Yes" & VisualCheck != "Yes" & IsDocBeingImported & IHCStatus = "Awaiting IHC") | @IsResponseDoc) & DeleteFlag = ""

ASKER CERTIFIED SOLUTION

CRAK

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

320adel

ASKER

Example of View 2:
****************** Column 1 *******************
@If(
      Form = "Contact";
            @If(
                  DealStatus = "Not Lodged";
                        @If(
                              !@IsTime(DateLodgementExpected);
                                    (@Today - SaleDate) / 86400;
                              (@Today - DateLodgementExpected) / 86400
                        );
                        DealStatus = "Lodged";
                  (@Today - DateConditionalExpected) / 86400;
                        DealStatus = "Conditional";
                  (@Today - DateUnConditionalExpected) / 86400;
                        DealStatus = "Unconditional";
                  (@Today - DateUnconditionalDocsExpected) / 86400;
                        DealStatus = "Unconditional Docs Received";
                  (@Today - DateSettlementExpected) / 86400;
                        DealStatus = "Settled";
                  (DateSettled - DateSettlementExpected) / 86400;
                        DealStatus = "CRASHED";
                  (DateCrashed - SaleDate) / 86400;
                        ""
            )
;
      ""
)

****************** Column 2 *******************
t_Purple:=102:102:153;
t_White:=255:255:255;
t_Black:=1:1:1;
@If(
Form="Contact";t_White:t_Black;
Form="Loan";214:216:226:t_Black;
Form="Task"&TaskStatus="In Progress";255:129:0:t_Black;
Form="Task"&TaskStatus="Completed";252:191:118:t_Black;
239:239:239:t_Black)

****************** Column 3 *******************
@If(
      Form = "Contact";
            IHCDate;
            ""
)

****************** Column 4 *******************
LoanDaysCol

****************** Column 5 *******************
t_White:=255:255:255;
t_Red:=255:0:0;
@If(Form="Contact" & LoanDaysCol >= -3; t_White:t_Red; RowColorCol)

****************** Column 6 *******************
RowColorCol

****************** Column 7 *******************
t_LoanDays :=
@If(
      Form = "Loan";
            @If(
                  LoanStatus = "Not Lodged";
                        (@Today - DateLodgementExpected) / 86400;
                  LoanStatus = "Lodged";
                        (@Today - DateConditionalExpected) / 86400;
                  LoanStatus = "Conditional";
                        (@Today - DateUnConditionalExpected) / 86400;
                  LoanStatus = "Unconditional";
                        (@Today - DateUnconditionalDocsExpected) / 86400;
                  LoanStatus = "Unconditional Docs Received";
                        (@Today - DateSettlementExpected) / 86400;
                  LoanStatus = "Settled";
                        (DateSettled - DateSettlementExpected) / 86400;
                  LoanStatus = "CRASHED";
                        (DateCrashed - SaleDate) / 86400;
                  ""
            )
;
      ""
);

@If(
      Form = "Loan";
            LoanDisplay +
            @If(
                  @IsNumber(t_LoanDays);
                        " [" +
                        @If(
                              t_LoanDays > 0;
                                    "+";
                              ""
                        )
                        + @Text(t_LoanDays) + " DAYS]";
                  ""
            );
      Form="Task";
            TaskDisplay;
      Form="Event";
            EventDisplay;
      Form="TravelBookingEmail":"ClientBookingEmail":"SMS";
            BookingDisplay;
      ""
)

****************** Column 8 *******************
ContactName+RelatedContactDocID

****************** Column 9 thru 14 *******************
Standard Plain Text Fields ( surnames / etc )

Thanks CRAK!

320adel

ASKER

There's "a couple" of 'em in there huh!

Why is it ok locally?

Sjef Bosman

Because y uo use @Today in your calculations, the view is recalculated EVERY time you open or refresh it.

Sjef Bosman

Ah, that's been said already...

CRAK

Most people would recommend a redesign. E.g. a scheduled agents evaluating those date-computations and labelling documents. The label (just a field value) would then affect the columns display.
Downside: a lot of mutations each night!

Alternatively, you could replace each of the @Todays:

@TextToTime("today")

(benefit from "today" as a constant)

I'd assing a variable to do that computation only once:

e.g. view 2, col 1:

Dt:=@TextToTime("today");
@If(
Form = "Contact";
@If(
DealStatus = "Not Lodged";
@If(
!@IsTime(DateLodgementExpected);
(Dt - SaleDate) / 86400;
(Dt - DateLodgementExpected) / 86400
);
DealStatus = "Lodged";
(Dt - DateConditionalExpected) / 86400;
DealStatus = "Conditional";
(Dt - DateUnConditionalExpected) / 86400;
DealStatus = "Unconditional";
....

320adel

ASKER

Hey Sjef -- why would it be OK locally though??? Why is it the HDD of the server getting hammered - not the CPU?

Further explanation on how it works would be great if possible

Kind Regards,
Adel

CRAK

Hi Sjef, good to have you around:
I've never really understood why this doesn't cause a similar drag locally. Do you?

320adel

ASKER

Thanks Crak, Thanks Sjef will give this a go!

Adel

Sjef Bosman

Why the HDD activity: the whole view's index has to be rewritten to the database.

You could try to use the @TextToTime("today") method and set the indexing to "Auto, at most every 4 hours" or so. Do a thorough check on this! If I were Notes, and I noticed that nothing in the document has changed, and there's no @Today in the formula, I wouldn't even bother to recalculate the column entry! So even when the server goes to the next day, many column values aren't recalculated.

It would have been nice if there were an option to throw away the view's index every night, of one specific view, so the first one to open it would have a problem. Sadly, there is no such option. You'd have to use a scheduled agent, to run at some time early in the morning, that opens the view and issues a view.Refresh. I'm not quite certain that this will really rebuild the view though.

Less delay on local: maybe because the index blocks are rewritten to disk later?

By the way, if the same database is used in many timezones in the world, calculations may differ for those places where your "today" is not their "today".

CRAK

That could be done by admin: launch a scheduled updall -V

Sjef Bosman

But that removes the indexes of ALL views in the database. What a waste...

CRAK

According to admin help that should update all built views. But I'm no Admin expert. A suitable updall, scheduled past midnight in combination with view indexes not updated in daytime at all, may reduce disk I/O caused by date/time evaliations in views even more! The downside would be that new docs may not appear until the next day!

320adel

ASKER

Thanks heaps CRAK! View load time was 1min 4 sec. Now it is something like 4 seconds. Gotta be happy with that eh!!

We have removed these calcs as in all honesty not need for day to day operations. Will create a 2nd set of views that people can get them from when required.

Thanks for your help!

320adel

ASKER

Thanks heaps CRAK! Thanks Sjef for your expansion too..

View load time was 1min 4 sec. Now it is something like 4 seconds. Gotta be happy with that eh!!

We have removed these calcs as in all honesty not need for day to day operations. Will create a 2nd set of views that people can get them from when required.

Thanks for your help!

Adel