Network Guardian /var/log Disk Usage Slowly Disappearing...

Posted on 2008-11-04
Last Modified: 2013-12-04
The /var/log partition is steadily losing disk space despite setting logging settings to only retain a days worth of logs.

I have the option set for "Delete old logs when free space is low:"
checked and the amount of disk space to use for logging is configured to the lowest value, 50%.

Free space does vary (for example 3.5 gigs then back up to 4.3 gigs) but the total free space appears to be gradually disappearing. Last week, the variation was between 5.8 and 6.2 gigs free but this week it is between 3.5 and 4.3.

There are about 150 users actively using the proxy at the same time. The proxy is running on a 1GB VM (Virtual Machine) with 20 gigs of HDD space which is larger than SmoothWall's recommendation.

I have tried once before with a 10 GB VM and the /var/log folder filled up completely (despite the same log settings mentioned above) and the web proxy and web content filter daemons completely stopped.

I also used the command "du -hs /var/log/* | sort -nr | head" to find the largest folder but the largest folder in /var/log is only 105 megs.

I am hesitant to perform any "hacks" because when logging in via SSH, the following message is displayed:

"WARNING: Any modifications to files or configurations made outside the
SmoothWall Web Interface or Setup Program will void any support
entitlement unless said change was requested in writing by an authorized
SmoothWall Support Engineer."

Has anyone ever run into this issue and found a solution?
Question by:Jeremy_in_Japan
    LVL 12

    Expert Comment

    Can you do a du -s  /var/log/*. The space is going somewhere.

    Keep in mind that if you delete a file that a process has open, the file still exists even though the directory entry is removed. If the process continues running and writing to the file, the available disk space will continue to drop. until the process exits at which point the file is deleted. So if you're unable to locate the file(s) that are growing, you may want to stop the process to see if the space is released.

    You can check for this situation with the lsof command. If the process has a deleted file open, it will show up like this:

    processname    3963 root    3w   REG  253,0 127606784 835681 /var/log/test.log (deleted)

    LVL 3

    Author Comment


    Thank you for the tip - I now see where the space is going. I was able to deduce that the following path is filling up:


    # du -sh * returns the following folders:

    3.7M    1
    3.7M    10818
    3.7M    10819
    4.3G    16384

    These are postgre folders and apparently, Network Guardian 2008 has a database "feature" that logs everything. In the database settings tab, there are pruning options but the shortest is "monthly". I can see the following message:

    "Warning - There is old data in the reporting database which isn't being used. Please backup and restore an archive to include the data"

    There is an option to backup the database but it never works and of course, I do not see any error messages. Either the page times out or the page refreshes with no backup listed.

    I have also tried logging into the database directly from localhost but cannot.

    Any suggestions other than hard deleting the folders?
    LVL 12

    Accepted Solution

    I don't know how complex the db schema is, but you might be able to do some optimization manually. Install a copy of pgadmin to examine the database. The username/password are on the same screen you used to adjust the retention period.

    I doubt, though, that you will recover much space, since the varying free space indicates you're already doing pruning. You might want to disable some of the logging, especially the proxy logging if the options aren't something you need.
    LVL 3

    Author Comment

    Thank you very much for your help - you pointed me in the right direction! Actually, the space issue seems to have stabilized - it doesn't go below 2.5 Gigs free space. It looks like I just needed a larger partition.

    Thank you again.
    LVL 3

    Author Closing Comment

    Thank you for your help!

    Featured Post

    Enabling OSINT in Activity Based Intelligence

    Activity based intelligence (ABI) requires access to all available sources of data. Recorded Future allows analysts to observe structured data on the open, deep, and dark web.

    Join & Write a Comment

    Suggested Solutions

    SHARE your personal details only on a NEED to basis. Take CHARGE and SECURE your IDENTITY. How do I then PROTECT myself and stay in charge of my own Personal details (and) - MY own WAY...
    Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
    Sending a Secure fax is easy with eFax Corporate ( First, Just open a new email message.  In the To field, type your recipient's fax number You can even send a secure international fax — just include t…
    Polish reports in Access so they look terrific. Take yourself to another level. Equations, Back Color, Alternate Back Color. Write easy VBA Code. Tighten space to use less pages. Launch report from a menu, considering criteria only when it is filled…

    732 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    22 Experts available now in Live!

    Get 1:1 Help Now