?
Solved

Network Guardian /var/log Disk Usage Slowly Disappearing...

Posted on 2008-11-04
5
Medium Priority
?
1,516 Views
Last Modified: 2013-12-04
The /var/log partition is steadily losing disk space despite setting logging settings to only retain a days worth of logs.

I have the option set for "Delete old logs when free space is low:"
checked and the amount of disk space to use for logging is configured to the lowest value, 50%.

Free space does vary (for example 3.5 gigs then back up to 4.3 gigs) but the total free space appears to be gradually disappearing. Last week, the variation was between 5.8 and 6.2 gigs free but this week it is between 3.5 and 4.3.

There are about 150 users actively using the proxy at the same time. The proxy is running on a 1GB VM (Virtual Machine) with 20 gigs of HDD space which is larger than SmoothWall's recommendation.

I have tried once before with a 10 GB VM and the /var/log folder filled up completely (despite the same log settings mentioned above) and the web proxy and web content filter daemons completely stopped.

I also used the command "du -hs /var/log/* | sort -nr | head" to find the largest folder but the largest folder in /var/log is only 105 megs.

I am hesitant to perform any "hacks" because when logging in via SSH, the following message is displayed:

"WARNING: Any modifications to files or configurations made outside the
SmoothWall Web Interface or Setup Program will void any support
entitlement unless said change was requested in writing by an authorized
SmoothWall Support Engineer."

Has anyone ever run into this issue and found a solution?
0
Comment
Question by:Jeremy_in_Japan
  • 3
  • 2
5 Comments
 
LVL 12

Expert Comment

by:Hugh Fraser
ID: 22875515
Can you do a du -s  /var/log/*. The space is going somewhere.

Keep in mind that if you delete a file that a process has open, the file still exists even though the directory entry is removed. If the process continues running and writing to the file, the available disk space will continue to drop. until the process exits at which point the file is deleted. So if you're unable to locate the file(s) that are growing, you may want to stop the process to see if the space is released.

You can check for this situation with the lsof command. If the process has a deleted file open, it will show up like this:

processname    3963 root    3w   REG  253,0 127606784 835681 /var/log/test.log (deleted)

0
 
LVL 3

Author Comment

by:Jeremy_in_Japan
ID: 22893642
hfraser,

Thank you for the tip - I now see where the space is going. I was able to deduce that the following path is filling up:

/var/log/postgres/data/base

# du -sh * returns the following folders:

3.7M    1
3.7M    10818
3.7M    10819
4.3G    16384

These are postgre folders and apparently, Network Guardian 2008 has a database "feature" that logs everything. In the database settings tab, there are pruning options but the shortest is "monthly". I can see the following message:

"Warning - There is old data in the reporting database which isn't being used. Please backup and restore an archive to include the data"

There is an option to backup the database but it never works and of course, I do not see any error messages. Either the page times out or the page refreshes with no backup listed.

I have also tried logging into the database directly from localhost but cannot.

Any suggestions other than hard deleting the folders?
0
 
LVL 12

Accepted Solution

by:
Hugh Fraser earned 2000 total points
ID: 22893940
I don't know how complex the db schema is, but you might be able to do some optimization manually. Install a copy of pgadmin to examine the database. The username/password are on the same screen you used to adjust the retention period.

I doubt, though, that you will recover much space, since the varying free space indicates you're already doing pruning. You might want to disable some of the logging, especially the proxy logging if the options aren't something you need.
0
 
LVL 3

Author Comment

by:Jeremy_in_Japan
ID: 22931086
Thank you very much for your help - you pointed me in the right direction! Actually, the space issue seems to have stabilized - it doesn't go below 2.5 Gigs free space. It looks like I just needed a larger partition.

Thank you again.
0
 
LVL 3

Author Closing Comment

by:Jeremy_in_Japan
ID: 31513019
Thank you for your help!
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

With the evolution of technology, we have finally reached a point where it is possible to have home automation features like having your thermostat turn up and door lock itself when you leave, as well as a complete home security system. This is a st…
What monsters are hiding in your child's room? In this article I will share with you a tech horror story that could happen to anyone, along with some tips on how you can prevent it from happening to you.
With Secure Portal Encryption, the recipient is sent a link to their email address directing them to the email laundry delivery page. From there, the recipient will be required to enter a user name and password to enter the page. Once the recipient …
When cloud platforms entered the scene, users and companies jumped on board to take advantage of the many benefits, like the ability to work and connect with company information from various locations. What many didn't foresee was the increased risk…
Suggested Courses

850 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question