Solved

root volume "/" running out of space....

Posted on 2011-09-21
11
531 Views
Last Modified: 2013-12-28
Solaris newbie here. We have a Solaris 10 box running oracle databases. The disk space usage on the root volume "/" is continuing to climb into the danger zone. Last night it was 93% utlized and this morning it's up to 94%. How can I find out what is chewing up this space before there is a problem? Please help!


 df -k screenshot
0
Comment
Question by:IT_Telephonics
11 Comments
 
LVL 9

Expert Comment

by:amolg
ID: 36573603
use df -h from root directory and see which directory is using more space and do the same in subdirectories which is using more space.
0
 
LVL 4

Expert Comment

by:alexeykomarov
ID: 36573648
Hi
Look at $ORACLE_BASE\admin
my be oracle logs and trace
0
 
LVL 77

Expert Comment

by:arnold
ID: 36573671
check /var/log /var/adm to make sure all the files that are there are on a logadm rotation schedule.
find /var/log -size +300000c
repeat for /var/adm

The problem is that you have a single partition which make it harder to isolate the cause.

what does the system do? web server, ftp server, mail server, database server, Virtualization?

0
Does Powershell have you tied up in knots?

Managing Active Directory does not always have to be complicated.  If you are spending more time trying instead of doing, then it's time to look at something else. For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why

 
LVL 22

Accepted Solution

by:
blu earned 500 total points
ID: 36574301
The most likely locations for unbounded disk space usage is in /var and in $ORACLE_BASE. You can't use the df command very effectively to
find the usages, but the du command can come in handy for this. I use this command to find out where the file space goes:

du -sdk * | sort -rn

First, go to /var and run the above command and save the output. Then do the same at $ORACLE_BASE. Look at the output for each and see if there
is anything that is using more space than you expect.

Once you have done this, if you still do not know where the space is going, wait a while and repeat those two commands and see what is increasing.
If you still do not see where the space is going, trying doing it from the "/" directory.
0
 

Author Comment

by:IT_Telephonics
ID: 36574990
Thanks for the help everyone. Now I think I'm getting somewhere. Using the 'du -sdk * | sort -rn' command in the /var directory I can see that the audit dir is by far the largest.

41214449        audit
697144            tmp
87349              sadm
41106               webconsole
37768              opt

The following files are in that directory:
# pwd
/var/audit
# ls -l
total 82437200
-rw-------   1 root     root     6952053749 Aug 27 13:37 20110709000121.20110827173755.<server name>
-rw-------   1 root     root     14215319161 Sep  7 21:30 20110829105451.20110908013026.<server name>
-rw-------   1 root     root     13009525154 Sep 16 03:23 20110908013026.20110916072330.<server name>
-rw-------   1 root     root     8010284957 Sep 21 11:50 20110916072330.not_terminated.<server name>

We did have a problem recently where someone set our /etc/security/audit_control file to log everything under the sun. We have since tuned that down a lot. Can these logs be trimmed\purged\deleted?

Thanks

Thanks
0
 
LVL 22

Assisted Solution

by:blu
blu earned 500 total points
ID: 36575143
You can look at this link for some ideas on how to set this up:

http://otoh.org/xwiki/bin/view/Blog/2008-08-29-audit-logadm
0
 

Author Comment

by:IT_Telephonics
ID: 36575319
I did some reading to trying to come up to speed on how auditing works. Looks like I only have one "active" log file in that dir which is the one that includes "not_terminated" in its name. The others can be backedup and removed according to this read: http://download.oracle.com/docs/cd/E19109-01/tsolaris7/805-8057/6j7iuhh33/index.html


Anyone see any problems if I backup\delete the three "closed" audit files?

Thanks
0
 

Author Comment

by:IT_Telephonics
ID: 36575329
Just another note. I'm at 95% utilized now. I ran another 'du -sdk * | sort -rn' and have verified that the audit dir is what is chewing up my resources when comparing the two results together.
0
 
LVL 22

Expert Comment

by:blu
ID: 36575978
You can delete the oldest, assuming that you no longer need it. You can use the scripts from the link I gave above to keep it from happening again.

You should ask yourself whether or not you really need auditing and if so, what events you really need.
0
 
LVL 16

Expert Comment

by:Joseph Gan
ID: 36576641
YOu can move the old audit files to other directory, ie. /u01. Usually /var/audit should be on a seperated filesystem if you have spare disk space availible. Otherwise, you can create a link to /u01.
0
 

Author Comment

by:IT_Telephonics
ID: 36577494
Back down to 34%. Crisis averted!!
Thanks again.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
prtdiag report hdd failure 10 152
what is the meaning of filesystem loop detected? 7 1,059
nodeip 9 80
Most secure Linux or x86 Unix that are least prone to ransomware/malware 24 112
Introduction Regular patching is part of a system administrator's tasks. However, many patches require that the system be in single-user mode before they can be installed. A cluster patch in particular can take quite a while to apply if the machine…
Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

809 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question