?
Solved

Swap space issue in Solaris

Posted on 2008-11-10
5
Medium Priority
?
4,613 Views
Last Modified: 2013-12-27
Hello,

Sometimes I'm really going mad with Swap space in Solaris? Please look at the below output, based on this i have some questions about Swap space?

root@hotsun# uname -a
SunOS hotsun.travelclick.net 5.8 Generic_117350-14 sun4u sparc SUNW,Sun-Fire-880
root@hotsun# prtconf | grep Mem
Memory size: 16384 Megabytes
root@hotsun# swap -s
total: 13790024k bytes allocated + 99080k reserved = 13889104k used, 10667080k available
root@hotsun# swap -l
swapfile             dev  swaplo blocks   free
/dev/md/dsk/d1      85,1      16 23163984 22928464
root@hotsun.travelclick.net# df -k
Filesystem            kbytes    used        avail      capacity  Mounted on
/dev/md/dsk/d0   4691952   2882534 1762499    63%    /
/proc                       0           0       0                      0%        /proc
fd                            0           0       0                      0%    /dev/fd
mnttab                     0           0       0                      0%    /etc/mnttab
/dev/md/dsk/d3  14219619 5033441 9043982     36%    /var
swap                 10662024 32           10661992    1%    /var/run
swap                 10784344 122352   10661992    2%    /tmp
.
.
root@hotsun# cat /var/adm/messages.0 | grep 'Nov  9'
.
.

Nov  9 03:00:17 hotsun.travelclick.net sendmail[10822]: [ID 801593 mail.alert] mA990F210822: SYSERR(root): savemail: cannot save rejected email anywhere
Nov  9 03:01:41 hotsun.travelclick.net tmpfs: [ID 518458 kern.warning] WARNING: /tmp: File system full, swap space limit exceeded                                 <----------------------

root@hotsun.travelclick.net# top
last pid: 27680;  load averages:  3.62,  3.59, 3.70  
135 processes: 131 sleeping, 4 on cpu
CPU states: 13.1% idle, 39.1% user,  4.4% kernel, 43.3% iowait,  0.0% swap
Memory: 16G real, 14G swap in use, 10G swap free                   <-----------------------------------

  PID USERNAME THR PRI NICE  SIZE   RES STATE   TIME    CPU COMMAND
25462 oracle     1  20    0   10G   10G cpu4   48:35 12.45% oracle
24883 oracle     1  20    0 9182M 9136M cpu3   57:07 12.39% oracle
21364 oracle    11  20    0 8406M 8358M cpu1   38:09  3.61% oracle
21362 oracle    11  51    0 8406M 8358M sleep  39:32  3.23% oracle
21366 oracle    11  41    0 8406M 8358M sleep  37:18  2.98% oracle
21360 oracle    11  52    0 8406M 8358M sleep  37:06  2.91% oracle
25423 root      11  60    0   78M   67M sleep   4:32  1.91% bpbkar
27607 oracle    11  58    0 8393M 8345M sleep   0:22  0.38% oracle
27611 oracle    11  58    0 8393M 8345M sleep   0:21  0.38% oracle
27605 oracle    11  58    0 8392M 8347M sleep   0:07  0.21% oracle
27609 oracle    11  58    0 8392M 8344M sleep   0:07  0.20% oracle
27668 root       1  58    0 3168K 2272K cpu6    0:00  0.11% top
  148 root      14  59    0   11M 7504K sleep  75:25  0.02% picld
27646 oracle    11  58    0 8391M 8343M sleep   0:00  0.02% oracle
27642 oracle    11  58    0 8391M 8343M sleep   0:00  0.02% oracle


1. Now what is the Total Swap swap space available in the system? From 'swap -l',  I could see it is 11 GB. If that is the case, why the 'top' output showing 14G swap in use, 10G swap free?

2. 'top' output shows 14G swap in use, 10G swap free. Please explain this.

3. Due to the following error " /tmp: File system full, swap space limit exceeded ", the Oracle DB running in the server got hung and we were forced to restart the Oracle database. Can you explain why it is and how to prevent it?

4. I persume /tmp filesystem is the Total size of Physical and Swap memory. If that is the case, it should show as 16 GB physical memory + Swap space. But 'df' output just shows approx. around 10 GB.  Can you clarify me this?

Thanks,
AR





0
Comment
Question by:rdashokraj
  • 2
  • 2
5 Comments
 
LVL 38

Assisted Solution

by:yuzh
yuzh earned 800 total points
ID: 22926973
It might have something to to with your kernel settings, you need to configure parameter related to semaphores or shared memory need to increase. You need to configure the kernel parametrers to run Oracle.
have a look at: "Solaris kernel has parameters " settings in:
http://www.dbspecialists.com/files/presentations/oracle817solaris.html
http://www.akadia.com/services/ora_sol_install_9i.html
 http://www.idevelopment.info/data/Oracle/DBA_tips/Sun_Solaris/SUNSOLARIS_10.shtml
http://blogs.sun.com/glennf/resource/oracle_perf_LnL_v2.pdf
It is better to check the installation guide for your version of Oracle.
To learn details about the Solaris swap space, please read:
http: Q_21630919.html
http:Q_23584919.html
PS: "top" is 3rd command, for Solaris 8 or newer, you should run "prstat", it is better than "top"
man prstat
to learn more
0
 
LVL 22

Accepted Solution

by:
Brian Utterback earned 1200 total points
ID: 22929116
Questions 1, 2 and 4 all really have the same explaination. Swap is accounted as the total of the
available memory plus the space on the swap devices. As you noted, the swap device is 11GB. However, not all of memory is available as swap. First, the memory used by the Kernel is subtracted. So according to the outputs, you have a total of 24GB of swap total.  At the time these commands were run, all the commands are in agreement that you have 10GB of swap space free.

>3. Due to the following error " /tmp: File system full, swap space limit exceeded ",

Ah, now this one is trickier. This says that there was a write which returned ENOSPC. Either /tmp was actually full or it has a quota placed on it. If it does have a quota, I would think that the df would indicate it in the available column, but it might not, I am not sure.

It is certainly possible to fill up /tmp in a transitory manner. It is often the case that a program will open a temporary file and then delete it while it is still open. This means that when the program terminates, the OS will automatically delete the file. This used to be a common practice, but it is now considered a bit antisocial, since it makes it impossible for an adminstrator to deal with errant programs. So, it would be trivial to create such a temporary file, and then keep writing to it until /tmp was full and then exit, freeing up the space again. In fact, the write that failed might easily have been just such an occurance.

Also notice, that it wasn't actually a program that is reporting that /tmp if full, it is the kernel. Another thing to note, is that sendmail does not save rejected email in /tmp, it should be writing that to the /var file system.
 
I hope that helps.  If you are using Solaris 10, then you might be able touse Dtrace to hel pyou find out where your swap space goes, if it happens again.
0
 

Author Comment

by:rdashokraj
ID: 22933588
Before I accept your solutions, I just to want confirm onething:

So if someone creates files of huge size in /tmp filesystem, they can fill up the Swap space and it would potentially can crash the OS itself right? (due to insufficient swap space).  If yes, don't you think it is a vulnerability in Solaris OS, that anyone normal user can bring down the application or OS just by creatiing some huge files in /tmp ?
0
 
LVL 22

Expert Comment

by:Brian Utterback
ID: 22933959
As I noted, you can set a quota on the size of /tmp, enforcing a buffer that will allow the system to keep running without running out of swap space. Of course applications that need free space in /tmp will start to fail, but it is up to the developers of the applications to handle error conditions gracefully.

See the mount_tmpfs man page for details.
0
 

Author Comment

by:rdashokraj
ID: 22935345
Thankyou so much for your explanation.
0

Featured Post

[Webinar] Kill tickets & tabs using PowerShell

Are you tired of cycling through the same browser tabs everyday to close the same repetitive tickets? In this webinar JumpCloud will show how you can leverage RESTful APIs to build your own PowerShell modules to kill tickets & tabs using the PowerShell command Invoke-RestMethod.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
Suggested Courses

608 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question