Solaris directory ull

Hi,

I have a Solaris Sparc system where df -h command shows the one partition almost full.

/dev/vx/dsk/dba_DG/dba_oraswVol
                        51G    49G   1.8G    97%    /opt/oracle

However, when I check with another command, du -sh, if just shows 17G used.

 du -sh oracle
  17G   oracle

Do you have any idea why there is such difference?

thanks!
joaotellesAsked:
Who is Participating?
 
nemws1Connect With a Mentor Database AdministratorCommented:
Hmm.. I was hoping 'du' running as root would return something closer to 50 gigs.

I see these are all vx filesystems, I assume Veritas VXFS.  Unfortunately, its been forever since I worked with VXes - could there be some issue with that?  Is it possible you have a pooled filesystem (like you can with ZFS) where 2 drives belong to the same pool and usage by one affects the other?  That's the only type of thing I can think of that's going on with this.
0
 
joaotellesAuthor Commented:
memory and CPU usage seems OK in prstat and top commands.

dpa@DP-NODE-1> top
load averages:  4.64,  4.34,  3.96;                    up 4916+15:03:59                                                                                        09:27:07
543 processes: 541 sleeping, 2 on cpu
CPU states: 86.4% idle,  4.0% user,  9.6% kernel,  0.0% iowait,  0.0% swap
Memory: 16G phys mem, 1996M free mem, 16G total swap, 5057M free swap
0
 
nemws1Database AdministratorCommented:
Do you have other directories in /opt that might be taking up the space?  'df' shows the full disk, but you're calling 'du' on only one directory in /opt.

What does this tell you:

du -sh /opt
0
Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
nemws1Database AdministratorCommented:
Also, run this (as root):

/usr/sbin/quot /opt

(yes, there is no 'e' there)
0
 
joaotellesAuthor Commented:
the problem is inside /opt/oracle.There is no other directory inside /opt/oracle that is taking the space.

See the df -h output below:

Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          19G    15G   3.7G    81%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   5.8G   1.5M   5.8G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/platform/SUNW,Sun-Fire-T200/lib/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/sparcv9/libc_psr.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                    16G   9.8G   5.8G    64%    /tmp
swap                   5.8G    48K   5.8G     1%    /var/run
/dev/vx/dsk/dm_DG/dm_log_Vol
                       5.0G   705M   4.0G    15%    /var/opt/smarttrust/dpa/dm/log
/dev/vx/dsk/dm_DG/dm_data_Vol
                       5.0G    18M   4.7G     1%    /var/opt/smarttrust/dpa/dm/data
/dev/vx/dsk/dba_DG/dba_ArchVol
                        45G   118M    42G     1%    /opt/arch
/dev/vx/dsk/dba_DG/dba_Vol1
                       111G    86G    25G    78%    /opt/mnt1
/dev/vx/dsk/dba_DG/dba_Vol4
                       146G   100G    45G    69%    /opt/mnt4
/dev/vx/dsk/dba_DG/dba_Vol3
                       245G   232G    12G    96%    /opt/mnt3
/dev/vx/dsk/dba_DG/dba_Vol2
                       167G   164G   3.4G    98%    /opt/mnt2
/dev/vx/dsk/dba_DG/dba_oraswVol
                        51G    49G   1.8G    97%    /opt/oracle
/dev/vx/dsk/dp6a_DG/dp6a_dataVol
                       172G   143G    28G    84%    /opt/rman


/usr/sbin/quot /opt
quot: /opt doesn't appear to be a filesystem.
ufs usage: quot [-nfcvha] [filesystem ...]


 /usr/sbin/quot /opt/oracle
quot: /opt/oracle not ufs filesystem


du -sh /opt
 733G   /opt


 du -sh /opt/oracle
  17G   /opt/oracle
0
 
nemws1Database AdministratorCommented:
Ah - sorry.  I see much more clearly now what is going on.

I'd like to this the difference has something to do with sparse files, but then things would be turned around (du would report a lot of usage and 'df' would *not*).

Can you try a usage tree utility?

Like this:
http://xdiskusage.sourceforge.net/

Or maybe dutree (perl - I use this all the time):
http://www.prasoonagrawal.com/library/oreilly/perl3/cookbook/ch05_17.htm
0
 
joaotellesAuthor Commented:
I am sending attached the ouput from the dutree as you asked.

Let me know what you find.
oracle.log
0
 
nemws1Database AdministratorCommented:
What's crazy is we now have a 3rd number.  Your dump says 35 gigs of disk is used.  Closer to 49 but still way off.  Something very odd is going on here. :(

Can you run 'df -h' again right now?  I'm wondering if you have some process that is using/freeing up disk space as you're working on it right now.
0
 
joaotellesAuthor Commented:
sure!
here is df -h command again.

df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          19G    15G   3.7G    81%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   5.7G   1.5M   5.7G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/platform/SUNW,Sun-Fire-T200/lib/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/sparcv9/libc_psr.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                    16G   9.8G   5.7G    64%    /tmp
swap                   5.7G    48K   5.7G     1%    /var/run
/dev/vx/dsk/dm_DG/dm_log_Vol
                       5.0G   705M   4.0G    15%    /var/opt/smarttrust/dpa/dm/log
/dev/vx/dsk/dm_DG/dm_data_Vol
                       5.0G    18M   4.7G     1%    /var/opt/smarttrust/dpa/dm/data
/dev/vx/dsk/dba_DG/dba_ArchVol
                        45G   712M    42G     2%    /opt/arch
/dev/vx/dsk/dba_DG/dba_Vol1
                       111G    86G    25G    78%    /opt/mnt1
/dev/vx/dsk/dba_DG/dba_Vol4
                       146G   100G    45G    69%    /opt/mnt4
/dev/vx/dsk/dba_DG/dba_Vol3
                       245G   232G    12G    96%    /opt/mnt3
/dev/vx/dsk/dba_DG/dba_Vol2
                       167G   164G   3.4G    98%    /opt/mnt2
/dev/vx/dsk/dba_DG/dba_oraswVol
                        51G    49G   1.7G    97%    /opt/oracle
/dev/vx/dsk/dp6a_DG/dp6a_dataVol
                       172G   143G    28G    84%    /opt/rman
0
 
nemws1Database AdministratorCommented:
Dangit. The same as before. Let me email my buddy that works more with Oracle than I do to see if he has any ideas.

Can you also run this and post the output:  mount | grep oracle
0
 
joaotellesAuthor Commented:
here is the mount output:

mount | grep oracle
/opt/oracle on /dev/vx/dsk/dba_DG/dba_oraswVol read/write/setuid/devices/log/largefiles/ioerror=mwdisable/dev=4a0c355 on Sun May 26 19:30:21 2013
0
 
joaotellesAuthor Commented:
searching on the internet, I saw a difference between how du and df commands work.

du is an example of a user-level program that is not aware of file system meta data, while df looks at the file system disk allocation maps and is aware of file system meta data. df obtains the true file system statistics, whereas du sees only a partial picture.

Do you think this explains the difference?
0
 
nemws1Database AdministratorCommented:
That is all correct, but it can't explain a multi-gigabyte difference.

Are you running 'du' as the root user?
0
 
joaotellesAuthor Commented:
The output from du is the same for both root and my user. I tried run with both.
0
 
joaotellesAuthor Commented:
the problem was a zombie file in the /opt/oracle directory. The file was already deleted but some processes in the machine were using it, and the space was not released.
0
 
nemws1Database AdministratorCommented:
Ah!  Very difficult to track down.  Glad you got it figured out.
0
All Courses

From novice to tech pro — start learning today.