[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

Solaris directory ull

Posted on 2013-06-17
16
Medium Priority
?
447 Views
Last Modified: 2013-06-19
Hi,

I have a Solaris Sparc system where df -h command shows the one partition almost full.

/dev/vx/dsk/dba_DG/dba_oraswVol
                        51G    49G   1.8G    97%    /opt/oracle

However, when I check with another command, du -sh, if just shows 17G used.

 du -sh oracle
  17G   oracle

Do you have any idea why there is such difference?

thanks!
0
Comment
Question by:joaotelles
  • 8
  • 8
16 Comments
 

Author Comment

by:joaotelles
ID: 39253577
memory and CPU usage seems OK in prstat and top commands.

dpa@DP-NODE-1> top
load averages:  4.64,  4.34,  3.96;                    up 4916+15:03:59                                                                                        09:27:07
543 processes: 541 sleeping, 2 on cpu
CPU states: 86.4% idle,  4.0% user,  9.6% kernel,  0.0% iowait,  0.0% swap
Memory: 16G phys mem, 1996M free mem, 16G total swap, 5057M free swap
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253592
Do you have other directories in /opt that might be taking up the space?  'df' shows the full disk, but you're calling 'du' on only one directory in /opt.

What does this tell you:

du -sh /opt
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253597
Also, run this (as root):

/usr/sbin/quot /opt

(yes, there is no 'e' there)
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 

Author Comment

by:joaotelles
ID: 39253619
the problem is inside /opt/oracle.There is no other directory inside /opt/oracle that is taking the space.

See the df -h output below:

Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          19G    15G   3.7G    81%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   5.8G   1.5M   5.8G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/platform/SUNW,Sun-Fire-T200/lib/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/sparcv9/libc_psr.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                    16G   9.8G   5.8G    64%    /tmp
swap                   5.8G    48K   5.8G     1%    /var/run
/dev/vx/dsk/dm_DG/dm_log_Vol
                       5.0G   705M   4.0G    15%    /var/opt/smarttrust/dpa/dm/log
/dev/vx/dsk/dm_DG/dm_data_Vol
                       5.0G    18M   4.7G     1%    /var/opt/smarttrust/dpa/dm/data
/dev/vx/dsk/dba_DG/dba_ArchVol
                        45G   118M    42G     1%    /opt/arch
/dev/vx/dsk/dba_DG/dba_Vol1
                       111G    86G    25G    78%    /opt/mnt1
/dev/vx/dsk/dba_DG/dba_Vol4
                       146G   100G    45G    69%    /opt/mnt4
/dev/vx/dsk/dba_DG/dba_Vol3
                       245G   232G    12G    96%    /opt/mnt3
/dev/vx/dsk/dba_DG/dba_Vol2
                       167G   164G   3.4G    98%    /opt/mnt2
/dev/vx/dsk/dba_DG/dba_oraswVol
                        51G    49G   1.8G    97%    /opt/oracle
/dev/vx/dsk/dp6a_DG/dp6a_dataVol
                       172G   143G    28G    84%    /opt/rman


/usr/sbin/quot /opt
quot: /opt doesn't appear to be a filesystem.
ufs usage: quot [-nfcvha] [filesystem ...]


 /usr/sbin/quot /opt/oracle
quot: /opt/oracle not ufs filesystem


du -sh /opt
 733G   /opt


 du -sh /opt/oracle
  17G   /opt/oracle
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253663
Ah - sorry.  I see much more clearly now what is going on.

I'd like to this the difference has something to do with sparse files, but then things would be turned around (du would report a lot of usage and 'df' would *not*).

Can you try a usage tree utility?

Like this:
http://xdiskusage.sourceforge.net/

Or maybe dutree (perl - I use this all the time):
http://www.prasoonagrawal.com/library/oreilly/perl3/cookbook/ch05_17.htm
0
 

Author Comment

by:joaotelles
ID: 39253714
I am sending attached the ouput from the dutree as you asked.

Let me know what you find.
oracle.log
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253743
What's crazy is we now have a 3rd number.  Your dump says 35 gigs of disk is used.  Closer to 49 but still way off.  Something very odd is going on here. :(

Can you run 'df -h' again right now?  I'm wondering if you have some process that is using/freeing up disk space as you're working on it right now.
0
 

Author Comment

by:joaotelles
ID: 39253759
sure!
here is df -h command again.

df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/md/dsk/d0          19G    15G   3.7G    81%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   5.7G   1.5M   5.7G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
/platform/SUNW,Sun-Fire-T200/lib/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
                        19G    15G   3.7G    81%    /platform/sun4v/lib/sparcv9/libc_psr.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                    16G   9.8G   5.7G    64%    /tmp
swap                   5.7G    48K   5.7G     1%    /var/run
/dev/vx/dsk/dm_DG/dm_log_Vol
                       5.0G   705M   4.0G    15%    /var/opt/smarttrust/dpa/dm/log
/dev/vx/dsk/dm_DG/dm_data_Vol
                       5.0G    18M   4.7G     1%    /var/opt/smarttrust/dpa/dm/data
/dev/vx/dsk/dba_DG/dba_ArchVol
                        45G   712M    42G     2%    /opt/arch
/dev/vx/dsk/dba_DG/dba_Vol1
                       111G    86G    25G    78%    /opt/mnt1
/dev/vx/dsk/dba_DG/dba_Vol4
                       146G   100G    45G    69%    /opt/mnt4
/dev/vx/dsk/dba_DG/dba_Vol3
                       245G   232G    12G    96%    /opt/mnt3
/dev/vx/dsk/dba_DG/dba_Vol2
                       167G   164G   3.4G    98%    /opt/mnt2
/dev/vx/dsk/dba_DG/dba_oraswVol
                        51G    49G   1.7G    97%    /opt/oracle
/dev/vx/dsk/dp6a_DG/dp6a_dataVol
                       172G   143G    28G    84%    /opt/rman
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253776
Dangit. The same as before. Let me email my buddy that works more with Oracle than I do to see if he has any ideas.

Can you also run this and post the output:  mount | grep oracle
0
 

Author Comment

by:joaotelles
ID: 39253786
here is the mount output:

mount | grep oracle
/opt/oracle on /dev/vx/dsk/dba_DG/dba_oraswVol read/write/setuid/devices/log/largefiles/ioerror=mwdisable/dev=4a0c355 on Sun May 26 19:30:21 2013
0
 

Author Comment

by:joaotelles
ID: 39253795
searching on the internet, I saw a difference between how du and df commands work.

du is an example of a user-level program that is not aware of file system meta data, while df looks at the file system disk allocation maps and is aware of file system meta data. df obtains the true file system statistics, whereas du sees only a partial picture.

Do you think this explains the difference?
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39253829
That is all correct, but it can't explain a multi-gigabyte difference.

Are you running 'du' as the root user?
0
 

Author Comment

by:joaotelles
ID: 39253880
The output from du is the same for both root and my user. I tried run with both.
0
 
LVL 23

Accepted Solution

by:
nemws1 earned 1000 total points
ID: 39254430
Hmm.. I was hoping 'du' running as root would return something closer to 50 gigs.

I see these are all vx filesystems, I assume Veritas VXFS.  Unfortunately, its been forever since I worked with VXes - could there be some issue with that?  Is it possible you have a pooled filesystem (like you can with ZFS) where 2 drives belong to the same pool and usage by one affects the other?  That's the only type of thing I can think of that's going on with this.
0
 

Author Comment

by:joaotelles
ID: 39260793
the problem was a zombie file in the /opt/oracle directory. The file was already deleted but some processes in the machine were using it, and the space was not released.
0
 
LVL 23

Expert Comment

by:nemws1
ID: 39260796
Ah!  Very difficult to track down.  Glad you got it figured out.
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded…
FreeBSD on EC2 FreeBSD (https://www.freebsd.org) is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Suggested Courses
Course of the Month18 days, 16 hours left to enroll

834 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question