Solved

Abnormal increase of database size

Posted on 2004-09-10
16
313 Views
Last Modified: 2013-11-16
A couple of weeks i replaced the design of one of our Notes applications.
One of the databases using this termplate was approx 9.6 Gb at that time.
Since the design update the database has been growing very fast , already 28 Gb at this time.
When i examine the db, i notice 10 views all having a huge sizes ( 850 Mb )
Next week i will start from scratch through a replica move and see what happens with the size.
In the new design template i did not change any view definition , so i really have no clue what is goiing on.
The huge database size results in very slow view rebuilding and delays of several minutes changing views.

Anyone had already an issue like this ? Solution ? Tips ?
 
0
Comment
Question by:Steamer
  • 4
  • 3
  • 3
  • +3
16 Comments
 
LVL 24

Expert Comment

by:HemanthaKumar
Comment Utility
How many documents ?? And any Attachments ? They cause to build up substantial size

run copy style compact.. equivalent to creating a new copy

syntax : load compact dbname -C

Also run updall -R nightly on this db

~Hemanth
0
 
LVL 46

Expert Comment

by:Sjef Bosman
Comment Utility
Notes version? Full-text search?

Run a simple compact on the database: in the client, Database Properties, 2nd tab, Compact. You may compact the design template as well.
0
 

Author Comment

by:Steamer
Comment Utility

There are approx 120000 docs and a rather small part have attachments.
Notes version 5 on server.

Compacting doesn't do much , the size of the views is there and not shrinking.
The number of docs hasn't increased significantly in the last few weeks.
The attachments were there before too and did't result in any problems.
I will try the compacting.
At the rate the database is increasing now , i am reaching the 64 Gb in 2 weeks.
I am not a big expert , but this seems like a loop or bug or something causing the views to grow out of proportion.
On monday i will compare the view sizes to the original size they had in the db before the design update
0
 
LVL 13

Expert Comment

by:CRAK
Comment Utility
Have you checked the existance of replication/save errors?
Agents running on 2 servers once per day, could have that effect....
0
 
LVL 46

Expert Comment

by:Sjef Bosman
Comment Utility
120000 docs and 9.6 Gb? That's a lot, an average of 80Kb per document. I know of a database with 42000 docs in ... 0.6 Gb, i.e 14Kb per document.

Look first in the Notes.log database of your server, under Usage, by Size, and open the document of your database. It shows the sizes of the views, so you can see what view is the largest. There is an option in the Compact function of the administrator to throw away one or all view indexes.

Can you see that the number of documents grows unexplainably? Say over the weekend, when nobody add anything?
0
 
LVL 24

Expert Comment

by:HemanthaKumar
Comment Utility

> There are approx 120000 docs and a rather small part have attachments.
> Notes version 5 on server.

That is far sure a large number of docs..

If you feel the docs doesn't add upto the size of the db.. then check the design to see if there are any duplicates.

Few questions..

Is this db in a cluster ?
And are any replication setup for this db ?

Sometimes cluster replication or normal replication can be the cause to add up space unnecessarily.. there by creating large amount of white space. Use Compact -B option to deal this

If that doesn't help try creating a brand new copy out of this db and check the size.


0
 

Author Comment

by:Steamer
Comment Utility

>120000 docs and 9.6 Gb? That's a lot, an average of 80Kb per document. I know of a database >with 42000 docs in ... 0.6 Gb, i.e 14Kb per document.

That's the strange part : the 125000 docs represent 3,9 Gb of data ( size of the db with no views). The rest of the size is being taken by design elements ( mainly views ) : pe there are 10 views with no resticting selection ( all docs ) each of them 850 Mb in size.
In an earlier copy of the database these were only 200 Mb.( which i guess is reasonable )
All my views did grow 400 % in 2 weeks time.
At this time the size is being stable 29.50 Gb , but the application is verys slow since it takes very long to update views.

>Have you checked the existance of replication/save errors?
>Agents running on 2 servers once per day, could have that effect....

No more replication conflicts than a couple of weeks ago.

>Look first in the Notes.log database of your server, under Usage, by Size, and open the >document of your database. It shows the sizes of the views, so you can see what view is the l>argest. There is an option in the Compact function of the administrator to throw away one or all
 view indexes.

I threw them all away , but when i access the view again , it is still growing to the 850 Mb mentioned above.

> Is this db in a cluster ?

It was in a cluster , but they took it out of it because at first we thought that was being the reason for all this to happen.

What i will do next is to recreate a view from scratch and see if this one also is growing out of proportion. I still think something is corrupt in the database.
On the other hand the same has happened to other databases using the same template but on a different day. This would mean a problem with the templae.
On the other hand : also the shared views in each particular database has grown approx. 400 %
Also we have contacted IBM with this issue , but apparently nothing is documented in their knowledge base concerning this problem.
This is driving me crazy.
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 24

Expert Comment

by:HemanthaKumar
Comment Utility
Did you try compact - B option which can reduce the size of the db ?

Also are there any advanced flags set for the db ? Like Don't overwrite free space ?? or any other . If so disable it and see . Most of the time these options create much trouble than relieving it...
0
 
LVL 46

Expert Comment

by:Sjef Bosman
Comment Utility
Last resort strategy: mail me, or anyone asking for it, a part of the template, e.g. some views that grow beyond the horizon. My mail adderss is in my profile.
0
 

Expert Comment

by:buoroj
Comment Utility
I had the same problem where the views were too large. That was with a 5.0.11 server.

Try looking at the design of the views:
1) DO NOT use @Now or complex formulas in the view selection.
2) remove as many of the unwanted columns as possible.
3) do not use column sorting
4) do not use complex formulas in columns

In my case I managed to reduce the size of most of the views except two. In their case the view tries to display multiple forms in a particular order. The ordering was done using a formula (if form = a then 1 else if b then 2...)

IBM could not help. And I had to schedule outages to compact the database.
After upgrading to 6.0.3 server and performing a compact -c -D and then an updall the database settled down.
0
 

Author Comment

by:Steamer
Comment Utility
Kinda forgot about this

Actually we finally found the causing for this abnormal increase in size.
In these databases there was an * ( asteriks )  in a reader/author field of numerous documents.
NOtes does handle these documents very bad, because apparently it attaches the complete addressbook to the views showing these documents.
The more documents had the asteriks , the more the database was growing.
We had people of IBM looking on this and they didn't find it was a problem.
 Anyway , i removed all * and ran an UPDAL and compact and everything is back to normal.
0
 
LVL 46

Expert Comment

by:Sjef Bosman
Comment Utility
That's good news! I think this information is very valuable, so may I suggest you ask for the question to be PAQ'ed and for the points to be refunded, since you found the answer yourself? Thanks for letting us know.

Sjef
0
 
LVL 13

Expert Comment

by:CRAK
Comment Utility
Ok with me.

Thanks for sharing that Steamer. Like Sjef mentioned: very valuable info! I am working on this application where I intended to translate empty reader fields into "*". I guess I'll better adjust a few ACL's ald use e.g. an [All] role instead.
0
 

Accepted Solution

by:
ee_ai_construct earned 0 total points
Comment Utility
Question answered by asker or dialog valuable.
Closed, 300 points refunded.
ee_ai_construct (replacement part #xm34)
Community Support Admin
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

Problem "Can you help me recover my changes?  I double-clicked the attachment, made changes, and then hit Save before closing it.  But when I try to re-open it, my changes are missing!"    Solution This solution opens the Outlook Secure Temp Fold…
Article by: Rob
Notes 8.5 Archiving Steps and Tips This article covers setting up a Notes archive, and helps understand some of the menu choices making setting up and maintaining a Notes archive file easier.
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now