We use an Perl handler for one of our web sites. Thorught the day, apache parent processes cconsume more and more memory until eventually the serverr uns out of memeory. We now reboot nightly.
We suspect mod_perl is leaking somewhere. Having refined the httpd config
If you look the sar -r output below you will see that the processes get much larger before the reboot.
# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_prefork_module>
StartServers 8
MinSpareServers 10
MaxSpareServers 50
ServerLimit 512
MaxClients 512
MaxRequestsPerChild 60
</IfModule>
So as to stop child processes consuming too much memory.
It may be a better idea to kill off children when they reach a certain size. However this does not reduce the size of the parent process. I am suspecting that dirty chidren are messing up the server processes memory.
Has anyone else come accorss this and might know a possibel fix.
uname -a
Linux esrh02.eskills.local 2.6.9-5.ELsmp #1 SMP Wed Jan 5 19:30:39 EST 2005 i686 i686 i386 GNU/Linux
top
top - 12:54:04 up 4:37, 1 user, load average: 1.17, 1.02, 0.68
Tasks: 90 total, 1 running, 86 sleeping, 0 stopped, 3 zombie
Cpu(s): 13.9% us, 2.6% sy, 0.0% ni, 57.6% id, 4.1% wa, 0.2% hi, 21.5% si
Mem: 4150004k total, 770960k used, 3379044k free, 37464k buffers
Swap: 2096472k total, 0k used, 2096472k free, 357216k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8573 daemon 15 0 65308 54m 6236 S 2 1.3 0:01.48 /etc/httpd/apache2/bin/htt
pd -k start
8308 daemon 16 0 64800 53m 6188 S 1 1.3 0:01.38 /etc/httpd/apache2/bin/htt
pd -k start
8593 daemon 16 0 59952 48m 6184 S 0 1.2 0:01.12 /etc/httpd/apache2/bin/htt
pd -k start
8208 daemon 15 0 54044 42m 6192 S 3 1.1 0:00.94 /etc/httpd/apache2/bin/htt
pd -k start
8373 daemon 16 0 50652 39m 6232 S 0 1.0 0:00.51 /etc/httpd/apache2/bin/htt
pd -k start
9306 daemon 16 0 47636 36m 6164 S 1 0.9 0:00.38 /etc/httpd/apache2/bin/htt
pd -k start
9252 daemon 15 0 47428 36m 6220 S 0 0.9 0:00.45 /etc/httpd/apache2/bin/htt
pd -k start
9332 daemon 15 0 47416 36m 6152 S 0 0.9 0:00.49 /etc/httpd/apache2/bin/htt
pd -k start
9331 daemon 20 0 47416 36m 6152 S 0 0.9 0:00.51 /etc/httpd/apache2/bin/htt
pd -k start
9333 daemon 16 0 47416 36m 6152 S 0 0.9 0:00.50 /etc/httpd/apache2/bin/htt
pd -k start
9438 daemon 16 0 47260 36m 6156 S 10 0.9 0:00.30 /etc/httpd/apache2/bin/htt
pd -k start
9334 daemon 16 0 42384 31m 6156 S 1 0.8 0:00.10 /etc/httpd/apache2/bin/htt
pd -k start
9437 daemon 17 0 42244 31m 6184 S 3 0.8 0:00.09 /etc/httpd/apache2/bin/htt
pd -k start
9307 daemon 15 0 42120 31m 6156 S 0 0.8 0:00.08 /etc/httpd/apache2/bin/htt
pd -k start
9511 daemon 16 0 41732 30m 6156 S 1 0.8 0:00.04 /etc/httpd/apache2/bin/htt
pd -k start
9399 daemon 15 0 40884 30m 6044 S 0 0.7 0:00.00 /etc/httpd/apache2/bin/htt
pd -k start
9510 daemon 16 0 40884 30m 6044 S 0 0.7 0:00.00 /etc/httpd/apache2/bin/htt
pd -k start
9512 daemon 16 0 40884 30m 6044 S 0 0.7 0:00.00 /etc/httpd/apache2/bin/htt
pd -k start
9509 daemon 16 0 40884 30m 6040 S 0 0.7 0:00.00 /etc/httpd/apache2/bin/htt
pd -k start
9171 root 16 0 40748 30m 5972 S 0 0.7 0:06.94 /etc/httpd/apache2/bin/htt
pd -k start
sar -r
06:45:01 1194156 2955848 71.23 146428 2054992 2096472 0 0.00 0
06:50:01 1161716 2988288 72.01 146432 2054988 2096472 0 0.00 0
06:55:01 1080868 3069136 73.96 146432 2055248 2096472 0 0.00 0
07:00:01 1076268 3073736 74.07 146432 2055248 2096472 0 0.00 0
07:05:01 1058716 3091288 74.49 146444 2055496 2096472 0 0.00 0
07:10:01 1050348 3099656 74.69 146448 2055232 2096472 0 0.00 0
07:15:01 1048812 3101192 74.73 146452 2055488 2096472 0 0.00 0
07:20:01 1006484 3143520 75.75 146460 2055740 2096472 0 0.00 0
07:25:01 1004812 3145192 75.79 146464 2055736 2096472 0 0.00 0
07:30:01 996316 3153688 75.99 146484 2055716 2096472 0 0.00 0
07:35:01 973940 3176064 76.53 146484 2055976 2096472 0 0.00 0
07:40:01 881820 3268184 78.75 146496 2055704 2096472 0 0.00 0
07:45:01 802020 3347984 80.67 146496 2055964 2096472 0 0.00 0
07:50:01 744364 3405640 82.06 146500 2056220 2096472 0 0.00 0
07:55:01 697444 3452560 83.19 146516 2056464 2096472 0 0.00 0
08:00:01 607076 3542928 85.37 146516 2056724 2096472 0 0.00 0
08:05:01 417588 3732416 89.94 146780 2097020 2096472 0 0.00 0
08:10:01 868324 3281680 79.08 146780 2097280 2096472 0 0.00 0
08:15:01 1134988 3015016 72.65 146780 2097800 2096472 0 0.00 0
Average: 1765621 2384383 57.45 106704 1505992 2096472 0 0.00 0
08:17:03 LINUX RESTART
08:20:01 kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree kbswpused %swpused kbswpcad
08:25:01 3823604 326400 7.87 14980 114500 2096472 0 0.00 0
08:30:01 3699612 450392 10.85 15916 122144 2096472 0 0.00 0
08:35:01 3732116 417888 10.07 16760 128060 2096472 0 0.00 0
08:40:01 3377988 772016 18.60 17488 134352 2096472 0 0.00 0
08:45:01 3617596 532408 12.83 18116 137104 2096472 0 0.00 0
08:50:01 3816556 333448 8.03 18504 138276 2096472 0 0.00 0
08:55:01 3746780 403224 9.72 18972 143008 2096472 0 0.00 0
09:00:01 3615420 534584 12.88 19408 143612 2096472 0 0.00 0
09:05:01 3578780 571224 13.76 19716 143824 2096472 0 0.00 0
09:10:01 3811036 338968 8.17 20160 145200 2096472 0 0.00 0
09:15:01 3726708 423296 10.20 20552 145588 2096472 0 0.00 0
09:20:01 3632604 517400 12.47 20896 146024 2096472 0 0.00 0
09:25:01 3516716 633288 15.26 21224 148816 2096472 0 0.00 0
09:30:01 3441916 708088 17.06 21564 158876 2096472 0 0.00 0
09:35:01 3355004 795000 19.16 21892 164008 2096472 0 0.00 0
09:40:01 3353068 796936 19.20 22256 166244 2096472 0 0.00 0
09:45:01 3683916 466088 11.23 23188 172072 2096472 0 0.00 0
09:50:01 3603108 546896 13.18 23776 172524 2096472 0 0.00 0
09:55:01 3534132 615872 14.84 24112 173748 2096472 0 0.00 0
10:00:01 3564852 585152 14.10 24480 174420 2096472 0 0.00 0
vmstat
procs -----------memory---------
- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 0 3157604 37580 363340 0 0 10 21 551 122 4 1 93 3
free
total used free shared buffers cached
Mem: 4150004 954320 3195684 0 37608 363832
-/+ buffers/cache: 552880 3597124
Swap: 2096472 0 2096472
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.
# Controls IP packet forwarding
net.ipv4.ip_forward = 0
# Controls source route verification
net.ipv4.conf.default.rp_f
ilter = 1
# Do not accept source routing
net.ipv4.conf.default.acce
pt_source_
route = 0
# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0
# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1