• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1546
  • Last Modified:

Network Guardian /var/log Disk Usage Slowly Disappearing...

The /var/log partition is steadily losing disk space despite setting logging settings to only retain a days worth of logs.

I have the option set for "Delete old logs when free space is low:"
checked and the amount of disk space to use for logging is configured to the lowest value, 50%.

Free space does vary (for example 3.5 gigs then back up to 4.3 gigs) but the total free space appears to be gradually disappearing. Last week, the variation was between 5.8 and 6.2 gigs free but this week it is between 3.5 and 4.3.

There are about 150 users actively using the proxy at the same time. The proxy is running on a 1GB VM (Virtual Machine) with 20 gigs of HDD space which is larger than SmoothWall's recommendation.

I have tried once before with a 10 GB VM and the /var/log folder filled up completely (despite the same log settings mentioned above) and the web proxy and web content filter daemons completely stopped.

I also used the command "du -hs /var/log/* | sort -nr | head" to find the largest folder but the largest folder in /var/log is only 105 megs.

I am hesitant to perform any "hacks" because when logging in via SSH, the following message is displayed:

"WARNING: Any modifications to files or configurations made outside the
SmoothWall Web Interface or Setup Program will void any support
entitlement unless said change was requested in writing by an authorized
SmoothWall Support Engineer."

Has anyone ever run into this issue and found a solution?
0
Jeremy_in_Japan
Asked:
Jeremy_in_Japan
  • 3
  • 2
1 Solution
 
Hugh FraserConsultantCommented:
Can you do a du -s  /var/log/*. The space is going somewhere.

Keep in mind that if you delete a file that a process has open, the file still exists even though the directory entry is removed. If the process continues running and writing to the file, the available disk space will continue to drop. until the process exits at which point the file is deleted. So if you're unable to locate the file(s) that are growing, you may want to stop the process to see if the space is released.

You can check for this situation with the lsof command. If the process has a deleted file open, it will show up like this:

processname    3963 root    3w   REG  253,0 127606784 835681 /var/log/test.log (deleted)

0
 
Jeremy_in_JapanAuthor Commented:
hfraser,

Thank you for the tip - I now see where the space is going. I was able to deduce that the following path is filling up:

/var/log/postgres/data/base

# du -sh * returns the following folders:

3.7M    1
3.7M    10818
3.7M    10819
4.3G    16384

These are postgre folders and apparently, Network Guardian 2008 has a database "feature" that logs everything. In the database settings tab, there are pruning options but the shortest is "monthly". I can see the following message:

"Warning - There is old data in the reporting database which isn't being used. Please backup and restore an archive to include the data"

There is an option to backup the database but it never works and of course, I do not see any error messages. Either the page times out or the page refreshes with no backup listed.

I have also tried logging into the database directly from localhost but cannot.

Any suggestions other than hard deleting the folders?
0
 
Hugh FraserConsultantCommented:
I don't know how complex the db schema is, but you might be able to do some optimization manually. Install a copy of pgadmin to examine the database. The username/password are on the same screen you used to adjust the retention period.

I doubt, though, that you will recover much space, since the varying free space indicates you're already doing pruning. You might want to disable some of the logging, especially the proxy logging if the options aren't something you need.
0
 
Jeremy_in_JapanAuthor Commented:
Thank you very much for your help - you pointed me in the right direction! Actually, the space issue seems to have stabilized - it doesn't go below 2.5 Gigs free space. It looks like I just needed a larger partition.

Thank you again.
0
 
Jeremy_in_JapanAuthor Commented:
Thank you for your help!
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

  • 3
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now