getting file system full message even when space is 70%

I get

ID 845546 kern.notice] NOTICE: alloc: /opt/IBM/WebSphere: file system full
even when the file system is 70% full. inode usage is about 70%.

When the message occurs , my websphere application hangs.
Any idea why this happens


OS is Sun OS 5.10
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

what does it show for all your file systems?
wasadmin11Author Commented:
$ df -h
Filesystem             size   used  avail capacity  Mounted on
/dev/dsk/c1t0d0s0       15G   8.2G   7.0G    54%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                    25G   1.1M    25G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
                        15G   8.2G   7.0G    54%    /platform/sun4v/lib/
                        15G   8.2G   7.0G    54%    /platform/sun4v/lib/sparcv9/
fd                       0K     0K     0K     0%    /dev/fd
/dev/dsk/c1t0d0s3      9.6G   5.9G   3.6G    63%    /var
swap                    25G   573M    25G     3%    /tmp
swap                    25G    56K    25G     1%    /var/run
/dev/md/dsk/d170       9.8G   1.4G   8.3G    15%    /oracle
/dev/md/dsk/d120        39G    13G    26G    35%    /apps
/dev/md/dsk/d130       1.9G   168M   1.7G     9%    /opt/mqm
/dev/md/dsk/d110       1.9G   487M   1.4G    26%    /opt/patrol
/dev/md/dsk/d140       1.9G   453M   1.4G    24%    /var/mqm
/dev/md/dsk/d160        30G    21G   8.2G    73%    /opt/IBM/WebSphere
                        30G    17G    12G    59%    /mbimount
                        79G    77G   1.4G    99%    /mnt
/dev/md/dsk/d180       9.8G   671M   9.1G     7%    /mbicode

$ df -F ufs -o i
Filesystem             iused   ifree  %iused  Mounted on
/dev/dsk/c1t0d0s0     256220 1688996    13%   /
/dev/dsk/c1t0d0s3      33823 1183393     3%   /var
/dev/md/dsk/d170       23469 1222867     2%   /oracle
/dev/md/dsk/d120        1189 4972507     0%   /apps
/dev/md/dsk/d130        3118  494418     1%   /opt/mqm
/dev/md/dsk/d110        5002  492534     1%   /opt/patrol
/dev/md/dsk/d140         507  497029     0%   /var/mqm
/dev/md/dsk/d160     2627179 1100181    70%   /opt/IBM/WebSphere
/dev/md/dsk/d180       23564 1222772     2%   /mbicode
Have you taken a look at                        79G    77G   1.4G    99%    /mnt

It may be that is causing the problem
Cloud Class® Course: MCSA MCSE Windows Server 2012

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

wasadmin11Author Commented:
the /mnt is only used to store the software .
 the error is for /opt/IBM/WebSphere
ufs: [ID 845546 kern.notice] NOTICE: alloc: /opt/IBM/WebSphere: file system full
A few things. Can you check the messages file for errors and post back if any notable ones found.
If you can, run lsof (not part of the default solaris tool set) against the file system to see what processes are writing to which files. You may have an open file that is filling up the file system.

Also, you could check for available inodes. post back the output of:

df -F ufs -o i /opt/IBM/WebSphere

Finally, you could check for fragmentation by running fsck.

Brian UtterbackPrinciple Software EngineerCommented:
He already posted the inode info above. One question, are the df commands above being done at the same time that the message says that the problem is occurring? It could be a temp file or some other transient condition causes the problem and then goes away. Other than the message, does anything happen, like a \n
application failing?

When the problem is actually happening, run "df -gl" and post the output.
Clean up some big files, core files and do a fsck ( after bring it to
single-user mode).

Also tune fs to optimize space

For e.g.
tunefs -o space /dev/md/dsk/d10

Note : Changing the tuning option on the filesystem does not affect
existing layout. It modifies the behaviour going forward. That being
said, moving from  time  to  space  optimization will make the partial
blocks available for other fragments.

 From tunefs(1M):

     -o space|time
           The file system can either be  instructed  to  try  to
           minimize  the  time spent allocating blocks, or to try
           to minimize the space fragmentation on the disk.  The
           default is time.


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
wasadmin11Author Commented:
Thanks for ur inputs. Recently the inode usage was 100% and i deleted a large number of small size files.
Is there a way to check if the fileystem is fragramented without unmounting. Is there a way for checking any log if the filesystem is fragmented (like errpt in AIX) ?
"fsck -n" while the filesystem is unmounted.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Unix OS

From novice to tech pro — start learning today.