Tuning PHP5 and FCGID on Debian etch4
Hi all,
it's first time i post here so i hope to respect rules of this board.
I have many server web apache2-mpm-worker with php5 as module.
I want increase performance and i want limit some websites trying mod_fcgid.
Now i have about 6000-8000 websites on each server (2 X AMD Opteron 275 Dual Core) and with php5 as module they work good ( load less than 5 score daily)
Problem is that, sometimes, one website cause high load web server.
I want to preview high load problem so i thought that with mod_fcgid or just fastcgi, i can limit processes for each website so that ONLY website that cause high load will have problem and not all others.
After some tuning I tried mod_fcgid with following options:
cat /etc/apache2/mods-enabled/
##############
AddHandler fcgid-script .fcgi .php
SocketPath /var/lib/apache2/fcgid/soc
IdleTimeout 30
BusyTimeout 30
ProcessLifeTime 360
SpawnScoreUpLimit 21000
MaxProcessCount 21000
DefaultMaxClassProcessCoun
IPCConnectTimeout 3
IPCCommTimeout 6
##############
And php-cgi script as follow:
##############
export PHPRC=/var/www/domain.ext/
export TMP=/var/www/domain.ext/ww
#export PHP_FCGI_CHILDREN=1
export PHP_FCGI_MAX_REQUESTS=300
exec /usr/bin/php5-cgi $@
##############
This is not enough cause "looklikes" that process will never die
I reached about 1000-1200 sleeping process( most of them php5-cgi ) and 8GB DDR RAM were full and start swapping.
CPU is ok but RAM is full (on each php.ini memory limit is 24M)
If i stop apache2, php5-cgi still alive unless i kill them with kill command, manually.
With mod_php i do not have these probs so i would like to know how tuning fcgid with php5 to consume less memory and be sure that process are killed after timeout.
On error.log i can see these messages:
[notice] mod_fcgid: process /var/www/domain.ext/www/sy
or
[warn] mod_fcgid: process 13274 graceful shutdown timeouted, sending SIGKILL
or
[notice] mod_fcgid: process /var/www/domain.ext/www/sy
so looklikes that processes are killed but anyway there still be too many processes and after few hours websites are not visible anymore(apache2 and php5-cgi still running).
So i hope that someone can show me light
Sorry for this "long" question.
I hope to be clear.
Waiting for your answer
Regards
it's first time i post here so i hope to respect rules of this board.
I have many server web apache2-mpm-worker with php5 as module.
I want increase performance and i want limit some websites trying mod_fcgid.
Now i have about 6000-8000 websites on each server (2 X AMD Opteron 275 Dual Core) and with php5 as module they work good ( load less than 5 score daily)
Problem is that, sometimes, one website cause high load web server.
I want to preview high load problem so i thought that with mod_fcgid or just fastcgi, i can limit processes for each website so that ONLY website that cause high load will have problem and not all others.
After some tuning I tried mod_fcgid with following options:
cat /etc/apache2/mods-enabled/
##############
AddHandler fcgid-script .fcgi .php
SocketPath /var/lib/apache2/fcgid/soc
IdleTimeout 30
BusyTimeout 30
ProcessLifeTime 360
SpawnScoreUpLimit 21000
MaxProcessCount 21000
DefaultMaxClassProcessCoun
IPCConnectTimeout 3
IPCCommTimeout 6
##############
And php-cgi script as follow:
##############
export PHPRC=/var/www/domain.ext/
export TMP=/var/www/domain.ext/ww
#export PHP_FCGI_CHILDREN=1
export PHP_FCGI_MAX_REQUESTS=300
exec /usr/bin/php5-cgi $@
##############
This is not enough cause "looklikes" that process will never die
I reached about 1000-1200 sleeping process( most of them php5-cgi ) and 8GB DDR RAM were full and start swapping.
CPU is ok but RAM is full (on each php.ini memory limit is 24M)
If i stop apache2, php5-cgi still alive unless i kill them with kill command, manually.
With mod_php i do not have these probs so i would like to know how tuning fcgid with php5 to consume less memory and be sure that process are killed after timeout.
On error.log i can see these messages:
[notice] mod_fcgid: process /var/www/domain.ext/www/sy
or
[warn] mod_fcgid: process 13274 graceful shutdown timeouted, sending SIGKILL
or
[notice] mod_fcgid: process /var/www/domain.ext/www/sy
so looklikes that processes are killed but anyway there still be too many processes and after few hours websites are not visible anymore(apache2 and php5-cgi still running).
So i hope that someone can show me light
Sorry for this "long" question.
I hope to be clear.
Waiting for your answer
Regards
ASKER
Yes it's load average and for "less than 5" i mean that server reach 5 score few times.
Mrtg reports that load has score around 2 all day(24h)
Anyway i do not think this is question
Mrtg reports that load has score around 2 all day(24h)
Anyway i do not think this is question
I understand that is not the question. However if your server load indicates how busy your system is. Is the 2 the avg. over 24 hours or the actual number for each 5 minute interval?
With dual core CPU's I would say that anything over 3 or 3.5 would be bad.
What I am attempting to see if maybe you load is too much for a single computer with two dual-core CPU's.
With dual core CPU's I would say that anything over 3 or 3.5 would be bad.
What I am attempting to see if maybe you load is too much for a single computer with two dual-core CPU's.
ASKER
2 is load each 5 interval so i think is good enough.
Problem is that memory finish too fast (8GB DDR RAM) so it start swap.
I mean if it could be a hw problem, then problem is about memory and not CPU.
Also CPU is 70%-80% free instead of memory usage.
Anyway i think (and i hope ) that i can tune better config.
I notice for example that another hosting provider host on one single server ( Dual Xeon 3150 if i am not wrong, with 8GB RAM for sure) 12000 websites while i host 6000 websites.
This hosting provider did not finish RAM while i finish.
That's why i think that my config could be tune better.
Problem is that memory finish too fast (8GB DDR RAM) so it start swap.
I mean if it could be a hw problem, then problem is about memory and not CPU.
Also CPU is 70%-80% free instead of memory usage.
Anyway i think (and i hope ) that i can tune better config.
I notice for example that another hosting provider host on one single server ( Dual Xeon 3150 if i am not wrong, with 8GB RAM for sure) 12000 websites while i host 6000 websites.
This hosting provider did not finish RAM while i finish.
That's why i think that my config could be tune better.
ASKER
2 is load each 5 minutes interval so i think is good enough.
I wrote wrong on my last comment without specify minutes.
I wrote wrong on my last comment without specify minutes.
If the load is 2 for each 5 min. interval that is fine. However, unless I am missing something a total system CPU avg. utilization 70-80% should mean that your load avg. should be above 2.
With "4" CPU's a system CPU busy of 25% would mean that you are basically using 100% of 1 CPU, a system CPU busy of 50% would mean you are using 100% of 2 CPU's and a system CPU busy of 75% would mean 100% of 3 CPU's.
Since you are avg. 70-80% that means you are using 100% of 2 CPU's all of the time 80%+ of the 3rd CPU and sometimes 10% of the 4th. If you are using that much CPU I would expect your load to be higher.
Now you could be having memory issues. If you are limting each PHP process to 24GB of memory and you only have 8GB of RAM, that means you are allowing each process to use 3 times the amount of physical memory. Which will cause swaping and swaping is a BAD thing. How much RAM is normally used? How much of this is cache and how much is buffers?
You can't necessary compare what somebody else is doing with what you are doing. The other hosting provider may not have any PHP running, the PHP they are running may not be as complex, or they could have less traffic than you do. Its like saying I can carry 10 boxes, but you can only carry 5. Why? Easy, each of my boxes way 1 pound, but each of your's weighs 3 pounds.
Do you control the PHP code? If not, then there may not be much you can do. If you do control the code, you may want to look at it to verify that it is cleaning up properly.
How are you running Apache? MPM or pre-fork? How do you have these parameters coded? It could be you are letting the child processes run for to many requests and they are chewing up too much memory.
With "4" CPU's a system CPU busy of 25% would mean that you are basically using 100% of 1 CPU, a system CPU busy of 50% would mean you are using 100% of 2 CPU's and a system CPU busy of 75% would mean 100% of 3 CPU's.
Since you are avg. 70-80% that means you are using 100% of 2 CPU's all of the time 80%+ of the 3rd CPU and sometimes 10% of the 4th. If you are using that much CPU I would expect your load to be higher.
Now you could be having memory issues. If you are limting each PHP process to 24GB of memory and you only have 8GB of RAM, that means you are allowing each process to use 3 times the amount of physical memory. Which will cause swaping and swaping is a BAD thing. How much RAM is normally used? How much of this is cache and how much is buffers?
You can't necessary compare what somebody else is doing with what you are doing. The other hosting provider may not have any PHP running, the PHP they are running may not be as complex, or they could have less traffic than you do. Its like saying I can carry 10 boxes, but you can only carry 5. Why? Easy, each of my boxes way 1 pound, but each of your's weighs 3 pounds.
Do you control the PHP code? If not, then there may not be much you can do. If you do control the code, you may want to look at it to verify that it is cleaning up properly.
How are you running Apache? MPM or pre-fork? How do you have these parameters coded? It could be you are letting the child processes run for to many requests and they are chewing up too much memory.
ASKER
I say that:
Also CPU is 70%-80% free <- then CPU usage is about 20%
So forget CPU.
I can and i must compare to other hosting provider cause php is same package version, os same version, apache2 worker same version and i can access to server-status and server-info about this other hosting provider.
PHP environment is the same and they do more traffic than me (i can see by mod_cband with public access on each server).
Apache environment is the same(StartServers and so on).
Configuration is different only about fcgid directives.
I know that you are trying to help me but please, believe me when i tell you that i replicate same environment of other hosting company, a part of fcgid directives.
Also CPU is 70%-80% free <- then CPU usage is about 20%
So forget CPU.
I can and i must compare to other hosting provider cause php is same package version, os same version, apache2 worker same version and i can access to server-status and server-info about this other hosting provider.
PHP environment is the same and they do more traffic than me (i can see by mod_cband with public access on each server).
Apache environment is the same(StartServers and so on).
Configuration is different only about fcgid directives.
I know that you are trying to help me but please, believe me when i tell you that i replicate same environment of other hosting company, a part of fcgid directives.
Sorry about the confusion on the CPU utilization, normally people put the CPU busy, not the CPU free.
As I do not know the exact setup of your enviroment or of the other hosting provider I want to make as few assumptions as possible. You know the environments, I don't. If you have ever tried to help somebody without knowing the details, then you should know that as soon as you make an assumptions, you start making the wrong recommendations.
What is your RAM utilization? How much is buffer and cache?
Just because you are using the same "everything" if the customers are writing PHP code, you don't control that and so what your customers may be doing may not be what the other customers are doing.
I don't know how many times I have been told "I have two things that are setup exactly the same." To find out there was a list of 20 things that were different.
If the only difference (other than processor type) if the fcgid directives, have you tried using what they are using?
Are you using MPM or fork?
As I do not know the exact setup of your enviroment or of the other hosting provider I want to make as few assumptions as possible. You know the environments, I don't. If you have ever tried to help somebody without knowing the details, then you should know that as soon as you make an assumptions, you start making the wrong recommendations.
What is your RAM utilization? How much is buffer and cache?
Just because you are using the same "everything" if the customers are writing PHP code, you don't control that and so what your customers may be doing may not be what the other customers are doing.
I don't know how many times I have been told "I have two things that are setup exactly the same." To find out there was a list of 20 things that were different.
If the only difference (other than processor type) if the fcgid directives, have you tried using what they are using?
Are you using MPM or fork?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
What does this mean? The load average on the server? If you load average is 5 for a 24 hour period, then the box is over utilized.