Solved

Solaris directory ull

Posted on 2013-06-17
16
418 Views
Last Modified: 2013-06-19
Hi,

I have a Solaris Sparc system where df -h command shows the one partition almost full.

/dev/vx/dsk/dba_DG/dba_oraswVol
                        51G    49G   1.8G    97%    /opt/oracle

However, when I check with another command, du -sh, if just shows 17G used.

 du -sh oracle
  17G   oracle

Do you have any idea why there is such difference?

thanks!
0
Comment
Question by:joaotelles
  • 8
  • 8
16 Comments
 

Author Comment

by:joaotelles
ID: 39253577
memory and CPU usage seems OK in prstat and top commands.

dpa@DP-NODE-1> top
load averages:  4.64,  4.34,  3.96;                    up 4916+15:03:59                                                                                        09:27:07
543 processes: 541 sleeping, 2 on cpu
CPU states: 86.4% idle,  4.0% user,  9.6% kernel,  0.0% iowait,  0.0% swap
Memory: 16G phys mem, 1996M free mem, 16G total swap, 5057M free swap
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253592
Do you have other directories in /opt that might be taking up the space?  'df' shows the full disk, but you're calling 'du' on only one directory in /opt.

What does this tell you:

du -sh /opt
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253597
Also, run this (as root):

/usr/sbin/quot /opt

(yes, there is no 'e' there)
0
 

Author Comment

by:joaotelles
ID: 39253619
the problem is inside /opt/oracle.There is no other directory inside /opt/oracle that is taking the space.

See the df -h output below:

Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          19G    15G   3.7G    81%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   5.8G   1.5M   5.8G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/platform/SUNW,Sun-Fire-T200/lib/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/sparcv9/libc_psr.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                    16G   9.8G   5.8G    64%    /tmp
swap                   5.8G    48K   5.8G     1%    /var/run
/dev/vx/dsk/dm_DG/dm_log_Vol
                       5.0G   705M   4.0G    15%    /var/opt/smarttrust/dpa/dm/log
/dev/vx/dsk/dm_DG/dm_data_Vol
                       5.0G    18M   4.7G     1%    /var/opt/smarttrust/dpa/dm/data
/dev/vx/dsk/dba_DG/dba_ArchVol
                        45G   118M    42G     1%    /opt/arch
/dev/vx/dsk/dba_DG/dba_Vol1
                       111G    86G    25G    78%    /opt/mnt1
/dev/vx/dsk/dba_DG/dba_Vol4
                       146G   100G    45G    69%    /opt/mnt4
/dev/vx/dsk/dba_DG/dba_Vol3
                       245G   232G    12G    96%    /opt/mnt3
/dev/vx/dsk/dba_DG/dba_Vol2
                       167G   164G   3.4G    98%    /opt/mnt2
/dev/vx/dsk/dba_DG/dba_oraswVol
                        51G    49G   1.8G    97%    /opt/oracle
/dev/vx/dsk/dp6a_DG/dp6a_dataVol
                       172G   143G    28G    84%    /opt/rman


/usr/sbin/quot /opt
quot: /opt doesn't appear to be a filesystem.
ufs usage: quot [-nfcvha] [filesystem ...]


 /usr/sbin/quot /opt/oracle
quot: /opt/oracle not ufs filesystem


du -sh /opt
 733G   /opt


 du -sh /opt/oracle
  17G   /opt/oracle
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253663
Ah - sorry.  I see much more clearly now what is going on.

I'd like to this the difference has something to do with sparse files, but then things would be turned around (du would report a lot of usage and 'df' would *not*).

Can you try a usage tree utility?

Like this:
http://xdiskusage.sourceforge.net/

Or maybe dutree (perl - I use this all the time):
http://www.prasoonagrawal.com/library/oreilly/perl3/cookbook/ch05_17.htm
0
 

Author Comment

by:joaotelles
ID: 39253714
I am sending attached the ouput from the dutree as you asked.

Let me know what you find.
oracle.log
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253743
What's crazy is we now have a 3rd number.  Your dump says 35 gigs of disk is used.  Closer to 49 but still way off.  Something very odd is going on here. :(

Can you run 'df -h' again right now?  I'm wondering if you have some process that is using/freeing up disk space as you're working on it right now.
0
 

Author Comment

by:joaotelles
ID: 39253759
sure!
here is df -h command again.

df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          19G    15G   3.7G    81%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   5.7G   1.5M   5.7G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/platform/SUNW,Sun-Fire-T200/lib/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/sparcv9/libc_psr.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                    16G   9.8G   5.7G    64%    /tmp
swap                   5.7G    48K   5.7G     1%    /var/run
/dev/vx/dsk/dm_DG/dm_log_Vol
                       5.0G   705M   4.0G    15%    /var/opt/smarttrust/dpa/dm/log
/dev/vx/dsk/dm_DG/dm_data_Vol
                       5.0G    18M   4.7G     1%    /var/opt/smarttrust/dpa/dm/data
/dev/vx/dsk/dba_DG/dba_ArchVol
                        45G   712M    42G     2%    /opt/arch
/dev/vx/dsk/dba_DG/dba_Vol1
                       111G    86G    25G    78%    /opt/mnt1
/dev/vx/dsk/dba_DG/dba_Vol4
                       146G   100G    45G    69%    /opt/mnt4
/dev/vx/dsk/dba_DG/dba_Vol3
                       245G   232G    12G    96%    /opt/mnt3
/dev/vx/dsk/dba_DG/dba_Vol2
                       167G   164G   3.4G    98%    /opt/mnt2
/dev/vx/dsk/dba_DG/dba_oraswVol
                        51G    49G   1.7G    97%    /opt/oracle
/dev/vx/dsk/dp6a_DG/dp6a_dataVol
                       172G   143G    28G    84%    /opt/rman
0
Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

 
LVL 23

Expert Comment

by:nemws1
ID: 39253776
Dangit. The same as before. Let me email my buddy that works more with Oracle than I do to see if he has any ideas.

Can you also run this and post the output:  mount | grep oracle
0
 

Author Comment

by:joaotelles
ID: 39253786
here is the mount output:

mount | grep oracle
/opt/oracle on /dev/vx/dsk/dba_DG/dba_oraswVol read/write/setuid/devices/log/largefiles/ioerror=mwdisable/dev=4a0c355 on Sun May 26 19:30:21 2013
0
 

Author Comment

by:joaotelles
ID: 39253795
searching on the internet, I saw a difference between how du and df commands work.

du is an example of a user-level program that is not aware of file system meta data, while df looks at the file system disk allocation maps and is aware of file system meta data. df obtains the true file system statistics, whereas du sees only a partial picture.

Do you think this explains the difference?
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253829
That is all correct, but it can't explain a multi-gigabyte difference.

Are you running 'du' as the root user?
0
 

Author Comment

by:joaotelles
ID: 39253880
The output from du is the same for both root and my user. I tried run with both.
0
 
LVL 23

Accepted Solution

by:
nemws1 earned 500 total points
ID: 39254430
Hmm.. I was hoping 'du' running as root would return something closer to 50 gigs.

I see these are all vx filesystems, I assume Veritas VXFS.  Unfortunately, its been forever since I worked with VXes - could there be some issue with that?  Is it possible you have a pooled filesystem (like you can with ZFS) where 2 drives belong to the same pool and usage by one affects the other?  That's the only type of thing I can think of that's going on with this.
0
 

Author Comment

by:joaotelles
ID: 39260793
the problem was a zombie file in the /opt/oracle directory. The file was already deleted but some processes in the machine were using it, and the space was not released.
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39260796
Ah!  Very difficult to track down.  Glad you got it figured out.
0

Featured Post

Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hyper-convergence systems have taken the IT world by storm and have quickly started to change our point of view of how the data center should and could be architected. In this article, I’ll explain the benefits of employing a hyper-converged system …
Moving your enterprise fax infrastructure from in-house fax machines and servers to the cloud makes sense — from both an efficiency and productivity standpoint. But does migrating to a cloud fax solution mean you will no longer be able to send or re…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

919 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now