Solved

Abnormal increase of database size

Posted on 2004-09-10
16
325 Views
Last Modified: 2013-11-16
A couple of weeks i replaced the design of one of our Notes applications.
One of the databases using this termplate was approx 9.6 Gb at that time.
Since the design update the database has been growing very fast , already 28 Gb at this time.
When i examine the db, i notice 10 views all having a huge sizes ( 850 Mb )
Next week i will start from scratch through a replica move and see what happens with the size.
In the new design template i did not change any view definition , so i really have no clue what is goiing on.
The huge database size results in very slow view rebuilding and delays of several minutes changing views.

Anyone had already an issue like this ? Solution ? Tips ?
 
0
Comment
Question by:Steamer
  • 4
  • 3
  • 3
  • +3
16 Comments
 
LVL 24

Expert Comment

by:HemanthaKumar
ID: 12029033
How many documents ?? And any Attachments ? They cause to build up substantial size

run copy style compact.. equivalent to creating a new copy

syntax : load compact dbname -C

Also run updall -R nightly on this db

~Hemanth
0
 
LVL 46

Expert Comment

by:Sjef Bosman
ID: 12029047
Notes version? Full-text search?

Run a simple compact on the database: in the client, Database Properties, 2nd tab, Compact. You may compact the design template as well.
0
 

Author Comment

by:Steamer
ID: 12030835

There are approx 120000 docs and a rather small part have attachments.
Notes version 5 on server.

Compacting doesn't do much , the size of the views is there and not shrinking.
The number of docs hasn't increased significantly in the last few weeks.
The attachments were there before too and did't result in any problems.
I will try the compacting.
At the rate the database is increasing now , i am reaching the 64 Gb in 2 weeks.
I am not a big expert , but this seems like a loop or bug or something causing the views to grow out of proportion.
On monday i will compare the view sizes to the original size they had in the db before the design update
0
Gigs: Get Your Project Delivered by an Expert

Select from freelancers specializing in everything from database administration to programming, who have proven themselves as experts in their field. Hire the best, collaborate easily, pay securely and get projects done right.

 
LVL 13

Expert Comment

by:CRAK
ID: 12031179
Have you checked the existance of replication/save errors?
Agents running on 2 servers once per day, could have that effect....
0
 
LVL 46

Expert Comment

by:Sjef Bosman
ID: 12033755
120000 docs and 9.6 Gb? That's a lot, an average of 80Kb per document. I know of a database with 42000 docs in ... 0.6 Gb, i.e 14Kb per document.

Look first in the Notes.log database of your server, under Usage, by Size, and open the document of your database. It shows the sizes of the views, so you can see what view is the largest. There is an option in the Compact function of the administrator to throw away one or all view indexes.

Can you see that the number of documents grows unexplainably? Say over the weekend, when nobody add anything?
0
 
LVL 24

Expert Comment

by:HemanthaKumar
ID: 12055223

> There are approx 120000 docs and a rather small part have attachments.
> Notes version 5 on server.

That is far sure a large number of docs..

If you feel the docs doesn't add upto the size of the db.. then check the design to see if there are any duplicates.

Few questions..

Is this db in a cluster ?
And are any replication setup for this db ?

Sometimes cluster replication or normal replication can be the cause to add up space unnecessarily.. there by creating large amount of white space. Use Compact -B option to deal this

If that doesn't help try creating a brand new copy out of this db and check the size.


0
 

Author Comment

by:Steamer
ID: 12057515

>120000 docs and 9.6 Gb? That's a lot, an average of 80Kb per document. I know of a database >with 42000 docs in ... 0.6 Gb, i.e 14Kb per document.

That's the strange part : the 125000 docs represent 3,9 Gb of data ( size of the db with no views). The rest of the size is being taken by design elements ( mainly views ) : pe there are 10 views with no resticting selection ( all docs ) each of them 850 Mb in size.
In an earlier copy of the database these were only 200 Mb.( which i guess is reasonable )
All my views did grow 400 % in 2 weeks time.
At this time the size is being stable 29.50 Gb , but the application is verys slow since it takes very long to update views.

>Have you checked the existance of replication/save errors?
>Agents running on 2 servers once per day, could have that effect....

No more replication conflicts than a couple of weeks ago.

>Look first in the Notes.log database of your server, under Usage, by Size, and open the >document of your database. It shows the sizes of the views, so you can see what view is the l>argest. There is an option in the Compact function of the administrator to throw away one or all
 view indexes.

I threw them all away , but when i access the view again , it is still growing to the 850 Mb mentioned above.

> Is this db in a cluster ?

It was in a cluster , but they took it out of it because at first we thought that was being the reason for all this to happen.

What i will do next is to recreate a view from scratch and see if this one also is growing out of proportion. I still think something is corrupt in the database.
On the other hand the same has happened to other databases using the same template but on a different day. This would mean a problem with the templae.
On the other hand : also the shared views in each particular database has grown approx. 400 %
Also we have contacted IBM with this issue , but apparently nothing is documented in their knowledge base concerning this problem.
This is driving me crazy.
0
 
LVL 24

Expert Comment

by:HemanthaKumar
ID: 12057666
Did you try compact - B option which can reduce the size of the db ?

Also are there any advanced flags set for the db ? Like Don't overwrite free space ?? or any other . If so disable it and see . Most of the time these options create much trouble than relieving it...
0
 
LVL 46

Expert Comment

by:Sjef Bosman
ID: 12057978
Last resort strategy: mail me, or anyone asking for it, a part of the template, e.g. some views that grow beyond the horizon. My mail adderss is in my profile.
0
 

Expert Comment

by:buoroj
ID: 12284394
I had the same problem where the views were too large. That was with a 5.0.11 server.

Try looking at the design of the views:
1) DO NOT use @Now or complex formulas in the view selection.
2) remove as many of the unwanted columns as possible.
3) do not use column sorting
4) do not use complex formulas in columns

In my case I managed to reduce the size of most of the views except two. In their case the view tries to display multiple forms in a particular order. The ordering was done using a formula (if form = a then 1 else if b then 2...)

IBM could not help. And I had to schedule outages to compact the database.
After upgrading to 6.0.3 server and performing a compact -c -D and then an updall the database settled down.
0
 

Author Comment

by:Steamer
ID: 12509872
Kinda forgot about this

Actually we finally found the causing for this abnormal increase in size.
In these databases there was an * ( asteriks )  in a reader/author field of numerous documents.
NOtes does handle these documents very bad, because apparently it attaches the complete addressbook to the views showing these documents.
The more documents had the asteriks , the more the database was growing.
We had people of IBM looking on this and they didn't find it was a problem.
 Anyway , i removed all * and ran an UPDAL and compact and everything is back to normal.
0
 
LVL 46

Expert Comment

by:Sjef Bosman
ID: 12512492
That's good news! I think this information is very valuable, so may I suggest you ask for the question to be PAQ'ed and for the points to be refunded, since you found the answer yourself? Thanks for letting us know.

Sjef
0
 
LVL 13

Expert Comment

by:CRAK
ID: 12514771
Ok with me.

Thanks for sharing that Steamer. Like Sjef mentioned: very valuable info! I am working on this application where I intended to translate empty reader fields into "*". I guess I'll better adjust a few ACL's ald use e.g. an [All] role instead.
0
 

Accepted Solution

by:
ee_ai_construct earned 0 total points
ID: 12541531
Question answered by asker or dialog valuable.
Closed, 300 points refunded.
ee_ai_construct (replacement part #xm34)
Community Support Admin
0

Featured Post

Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

For users on the Lotus Notes 8 Standard client, this article provides information on checking the Java Heap size and adjusting it to half of your system RAM in attempt to get the Lotus Notes 8.x Standard client to run faster.  I've had to exercise t…
  In today’s Arena we can’t imagine our lives without Internet as we are highly used to of it. If we consider our life style just for only 2 min we found that face to face communication is swapped by e-communication.  Every Where from Works place to…
This tutorial gives a high-level tour of the interface of Marketo (a marketing automation tool to help businesses track and engage prospective customers and drive them to purchase). You will see the main areas including Marketing Activities, Design …
Email security requires an ever evolving service that stays up to date with counter-evolving threats. The Email Laundry perform Research and Development to ensure their email security service evolves faster than cyber criminals. We apply our Threat…

776 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question