root volume "/" running out of space....

Solaris newbie here. We have a Solaris 10 box running oracle databases. The disk space usage on the root volume "/" is continuing to climb into the danger zone. Last night it was 93% utlized and this morning it's up to 94%. How can I find out what is chewing up this space before there is a problem? Please help!

 df -k screenshot
Who is Participating?

Improve company productivity with a Business Account.Sign Up

Brian UtterbackConnect With a Mentor Principle Software EngineerCommented:
The most likely locations for unbounded disk space usage is in /var and in $ORACLE_BASE. You can't use the df command very effectively to
find the usages, but the du command can come in handy for this. I use this command to find out where the file space goes:

du -sdk * | sort -rn

First, go to /var and run the above command and save the output. Then do the same at $ORACLE_BASE. Look at the output for each and see if there
is anything that is using more space than you expect.

Once you have done this, if you still do not know where the space is going, wait a while and repeat those two commands and see what is increasing.
If you still do not see where the space is going, trying doing it from the "/" directory.
use df -h from root directory and see which directory is using more space and do the same in subdirectories which is using more space.
Alexey KomarovChief Project EngineerCommented:
Look at $ORACLE_BASE\admin
my be oracle logs and trace
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

check /var/log /var/adm to make sure all the files that are there are on a logadm rotation schedule.
find /var/log -size +300000c
repeat for /var/adm

The problem is that you have a single partition which make it harder to isolate the cause.

what does the system do? web server, ftp server, mail server, database server, Virtualization?

IT_TelephonicsAuthor Commented:
Thanks for the help everyone. Now I think I'm getting somewhere. Using the 'du -sdk * | sort -rn' command in the /var directory I can see that the audit dir is by far the largest.

41214449        audit
697144            tmp
87349              sadm
41106               webconsole
37768              opt

The following files are in that directory:
# pwd
# ls -l
total 82437200
-rw-------   1 root     root     6952053749 Aug 27 13:37 20110709000121.20110827173755.<server name>
-rw-------   1 root     root     14215319161 Sep  7 21:30 20110829105451.20110908013026.<server name>
-rw-------   1 root     root     13009525154 Sep 16 03:23 20110908013026.20110916072330.<server name>
-rw-------   1 root     root     8010284957 Sep 21 11:50 20110916072330.not_terminated.<server name>

We did have a problem recently where someone set our /etc/security/audit_control file to log everything under the sun. We have since tuned that down a lot. Can these logs be trimmed\purged\deleted?


Brian UtterbackConnect With a Mentor Principle Software EngineerCommented:
You can look at this link for some ideas on how to set this up:
IT_TelephonicsAuthor Commented:
I did some reading to trying to come up to speed on how auditing works. Looks like I only have one "active" log file in that dir which is the one that includes "not_terminated" in its name. The others can be backedup and removed according to this read:

Anyone see any problems if I backup\delete the three "closed" audit files?

IT_TelephonicsAuthor Commented:
Just another note. I'm at 95% utilized now. I ran another 'du -sdk * | sort -rn' and have verified that the audit dir is what is chewing up my resources when comparing the two results together.
Brian UtterbackPrinciple Software EngineerCommented:
You can delete the oldest, assuming that you no longer need it. You can use the scripts from the link I gave above to keep it from happening again.

You should ask yourself whether or not you really need auditing and if so, what events you really need.
Joseph GanSystem AdminCommented:
YOu can move the old audit files to other directory, ie. /u01. Usually /var/audit should be on a seperated filesystem if you have spare disk space availible. Otherwise, you can create a link to /u01.
IT_TelephonicsAuthor Commented:
Back down to 34%. Crisis averted!!
Thanks again.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.