[Webinar] Streamline your web hosting managementRegister Today

x
?
Solved

segmentation fault, high server load

Posted on 2009-04-10
7
Medium Priority
?
795 Views
Last Modified: 2013-12-16
I'm having very high server loads, 8 to 14.4.

In investigating, I find this in the httpd/error_log

[Fri Apr 10 08:28:53 2009] [notice] child pid 4591 exit signal Segmentation fault (11)
[Fri Apr 10 08:38:50 2009] [notice] child pid 20513 exit signal Segmentation fault (11)
[Fri Apr 10 08:39:03 2009] [notice] child pid 20032 exit signal Segmentation fault (11)
[Fri Apr 10 08:39:43 2009] [notice] child pid 6393 exit signal Segmentation fault (11)
[Fri Apr 10 08:40:38 2009] [notice] child pid 20525 exit signal Segmentation fault (11)
[Fri Apr 10 08:40:50 2009] [notice] child pid 6394 exit signal Segmentation fault (11)
[Fri Apr 10 08:41:08 2009] [notice] child pid 6910 exit signal Segmentation fault (11)
[Fri Apr 10 08:41:34 2009] [notice] child pid 20233 exit signal Segmentation fault (11)
[Fri Apr 10 08:41:53 2009] [notice] child pid 21073 exit signal Segmentation fault (11}

Is this likely to be related to the problem of the high server load, and what is the next step I should take?

Thank you,

Chris

(this is an older machine btw, Red Hat Linux release 7.3 (Valhalla)) running ensim
0
Comment
Question by:St_Aug_Beach_Bum
  • 5
7 Comments
 

Author Comment

by:St_Aug_Beach_Bum
ID: 24115645
Rebooting brings the server load down, and the segmentation fault goes away after rebooting, but this has been going on for two days now, after rebooting it will eventually start happening again (after several hours usually).  

0
 
LVL 40

Accepted Solution

by:
omarfarid earned 2000 total points
ID: 24115773
if you run top at the problem time, what do you see the processes cpu and memory utilization? Also, what about swap? you could be out of physical memory and swap space.

Are you running apache ? I have see some comments about too many modules loaded and they were pointing to reduce the number of modules loaded.
0
 

Author Comment

by:St_Aug_Beach_Bum
ID: 24116485
Thank you for responsding omarfarid.

top confirms low memory, but I can't tell why it's running low on memory.  Is running out of memory causing the segmentation fault, or is that fault eating up the memory?

The load average has been ok for several hours after I rebooted, it just started climbing back up a few minutes ago.

10:44am  up  1:51,  1 user,  load average: 1.89, 1.90, 1.29
94 processes: 91 sleeping, 3 running, 0 zombie, 0 stopped
CPU states:  1.2% user,  3.0% system,  0.0% nice, 95.8% idle
Mem:   506044K av,  500612K used,    5432K free,       0K shrd,   60984K buff
Swap: 2048276K av,   37456K used, 2010820K free                  302772K cached

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
 2839 root       5 -10 21372  11M  1208 S <   0.0  2.3   0:00 ensim-python
 4646 root       5 -10 21372  11M  1208 S <   0.0  2.3   0:00 ensim-python
 4647 root       5 -10 21372  11M  1208 S <   0.0  2.3   0:00 ensim-python
 4648 root       5 -10 21372  11M  1208 S <   0.0  2.3   0:00 ensim-python
 4649 root       5 -10 21372  11M  1208 S <   0.0  2.3   0:00 ensim-python
 4650 root       5 -10 21372  11M  1208 S <   0.0  2.3   0:00 ensim-python
 4651 root       5 -10 21372  11M  1208 S <   0.0  2.3   0:00 ensim-python
 2738 apache     9   0 12852  10M  3188 S     0.0  2.0   0:03 httpd
 2695 apache     9   0 12428 9968  3064 S     0.0  1.9   0:04 httpd
 2698 apache     9   0 10380 7932  2636 S     0.0  1.5   0:04 httpd
 6428 apache     9   0 10260 7916  2708 S     0.0  1.5   0:02 httpd
 2696 apache     9   0 10584 7712  2528 S     0.0  1.5   0:03 httpd
 2694 apache    10   0 10264 7660  2620 S     0.4  1.5   0:03 httpd
 2737 apache     9   0 10248 7632  2772 S     0.0  1.5   0:04 httpd
 2697 apache     9   0  9788 7492  2564 S     0.0  1.4   0:03 httpd
 2743 apache     9   0 10084 7380  2724 S     0.0  1.4   0:03 httpd
 2715 apache     9   0  9820 7372  2648 S     0.0  1.4   0:04 httpd
13041 apache     9   0  8624 5788  3212 S     0.0  1.1   0:00 httpd
13275 apache    10   0  8504 5672  3468 S     0.2  1.1   0:00 httpd
13039 apache     9   0  8136 5236  3044 S     0.0  1.0   0:00 httpd
13277 apache     9   0  8012 5076  3176 S     0.0  1.0   0:00 httpd
12772 root      17   0  3676 3452   332 R     1.6  0.6   0:04 sendmail
 2444 mysql     15  10  5336 2924  1520 S N   0.0  0.5   0:00 mysqld
 2446 mysql     13  10  5336 2924  1520 S N   0.0  0.5   0:00 mysqld
 2447 mysql     15  10  5336 2924  1520 S N   0.0  0.5   0:00 mysqld
 2448 mysql     15  10  5336 2924  1520 S N   0.0  0.5   0:00 mysqld
13293 apache    11   0  6696 2852  1968 S     0.0  0.5   0:00 httpd
 2905 root       9   0  2448 2448  1036 S     0.0  0.4   0:00 poprelayd
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 

Author Comment

by:St_Aug_Beach_Bum
ID: 24116506
I don't see any one pid taking up a lot of memory, but there are several instances of ensim-python running, and I don't remember seeing that before, so I don't know if that's normal. Or what to do about it.
0
 

Author Comment

by:St_Aug_Beach_Bum
ID: 24116560
Oh, and I do have apache, but I haven't made any changes to the server software or added any new programs for it to run.
0
 

Author Closing Comment

by:St_Aug_Beach_Bum
ID: 31568896
Omarfarid, thank you.  The problem pretty much resolved itself, and why we did not really get a 'solution' here, you seem to be correct that it was memory related. So I will award you the points.  Thank you for your assistance, Chris
0
 
LVL 1

Expert Comment

by:zmagyar
ID: 24704849
Actually I got the same problem and started on the memory route. The free memory is low here as well. But free memory means nothing on linux as the free memory is usually used as FS cache and freed up immediately as an application needs it.
0

Featured Post

The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
This article explains how to use the rsync command to create backups and sync data across hosts. Rsync is a very useful command that is often used to copy data, make backups, migrate hosts, and bridge the gap between site staging and production envi…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses
Course of the Month11 days, 7 hours left to enroll

640 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question