Solved

High CPU load, slow page serving

Posted on 2004-08-24
8
246 Views
Last Modified: 2010-03-04
I hope I am posting this in the right section, but I do believe the reason for the problem is Apache/MySQL.

I'm running rh9 on a Celeron 1.7. I'm currently serving about 100,000 pages per day, all of them are php pages and rely (more or less heavily) on a MySQL database. My problem is that with this volume (roughly translates to 3 gigs per day), the CPU is reaching 100% and processes start building up during peak times, pushing the server load beyond 10 up to as much as 50 (according to top).

 18:55:22  up 5 days, 22:56,  1 user,  load average: 7.31, 6.30, 5.92
162 processes: 141 sleeping, 21 running, 0 zombie, 0 stopped
CPU states:  84.0% user  15.9% system   0.0% nice   0.0% iowait   0.0% idle
Mem:  1022796k av,  792168k used,  230628k free,       0k shrd,    7900k buff
                    470300k actv,  122056k in_d,   18972k in_c
Swap: 1052248k av,   77848k used,  974400k free                  497896k cached

This snapshot is rather harmless, it can get much worse. As you can see, I have 1 gig of RAM installed and no, I am not swapping (so memory doesn't seem to be the problem). I do use the server for my own mail (qmail) and have to fight off a lot of spam (using spam assassin), but a basic and reliable test (apachectl stop :) reveals that the vast majority of the load is due to Apache/mySQL.

I am aware that php is CPU intensive, and my CPU is not the fastest around, but I believe it should be able to handle the load without the problems I am having. True?

P.S. I have installed Zend Accelerator to solve the problem - that get's me up to 150,00 pageviews and 4,5 gigs per day, but then the problem resurfaces.
0
Comment
Question by:yosmc
  • 3
  • 2
8 Comments
 
LVL 15

Expert Comment

by:samri
ID: 11894320
hi yosmc,

Since you pretty much isolated that Apache may be the the main contributor to the excessive load on the server, perhaps you may want to take a look at how Apache is doing, in term of it's processes.  Apache comes with a built-in "monitor", which is available using mod_status.  Take a look at Apache docs on mod_status (http://httpd.apache.org/docs-2.0/mod/mod_status.html) to enable it, and how to interpret the output.

Another thing that you could take a look at is Apache general tuning tips (http://httpd.apache.org/docs-2.0/misc/perf-tuning.html) which should give you an overview on what to look for in optimizing your Apache for performance.  I personally found the pages a bit too technical (at some times), but it is a good place to start with.

Some pages that I can search thru (http://www.google.com/search?q=%22apache+tuning%22+performance&hl=en&lr=&ie=UTF-8&start=10&sa=N)
- http://www.ece.concordia.ca/~daniel/tips/apache_tuning.html
- http://builder.com.com/5100-6387-1045523.html
- http://phplens.com/phpeverywhere/tuning-apache-php
- http://www.redhat.com/archives/tux-list/2001-November/msg00062.html
- http://www.redhat.com/docs/manuals/linux/RHL-9-Manual/custom-guide/s1-httpd-tuning-settings.html
- http://people.redhat.com/alikins/system_tuning.html#apache

* and a bunch of other.  I hope it could give you a good kick-start.

I personally would be happy to learn the best way to *boost* apache to the max :)

Cheers
0
 

Author Comment

by:yosmc
ID: 11895461
Here goes my server-status (it roughly corresponds to s server-load of 7 via top):

Current Time: Wednesday, 25-Aug-2004 12:51:30 CDT
Restart Time: Wednesday, 25-Aug-2004 10:57:21 CDT
Parent Server Generation: 30
Server uptime: 1 hour 54 minutes 8 seconds
Total accesses: 205869 - Total Traffic: 669.8 MB
CPU Usage: u104.46 s8.27 cu.02 cs.01 - 1.65% CPU load
30.1 requests/sec - 100.2 kB/second - 3411 B/request
18 requests currently being processed, 10 idle workers
__CRWCCCC.W__CR____C...R..C._WW..W.WR._.........................
................................................................
................................................................
................................................................

Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process

Srv PID Acc M CPU  SS Req Conn Child Slot Client VHost Request
0-30 8913 0/627/7143 _  9.71 0 0 0.0 1.67 22.96  192.234.106.2 www.mysite.com GET / HTTP/1.1
1-30 9556 0/0/8520 _  1.10 0 0 0.0 0.00 28.03  198.103.111.110 www.mysite.com GET /forums/templates/subSilver/images/icon_mini_profile.gif HT
2-30 9010 1/451/8145 C  8.28 0 0 0.7 1.21 25.04  66.43.162.74 www.mysite.com GET /files/8right_up.gif HTTP/1.1
3-30 9540 0/19/7068 R  0.39 0 0 0.0 0.04 24.06  ? ? ..reading..  
4-30 8536 0/19/6125 W  0.34 889 0 0.0 0.03 19.61  66.36.74.231 www.mysite.com GET /forums/images/avatars/60.gif HTTP/1.
5-30 9541 1/17/5595 C  0.30 0 0 0.1 0.03 18.01  66.43.162.74 www.mysite.com GET /files/8ugang2.gif HTTP/1.1
6-30 9167 1/287/6997 C  5.14 0 0 0.4 0.65 22.64  199.0.65.2 www.mysite.com GET /files/sign.gif HTTP/1.1
7-30 9542 1/17/5596 C  0.07 0 0 0.2 0.02 16.39  199.0.65.2 www.mysite.com GET /files/links.gif HTTP/1.1
8-30 9145 1/262/5190 C  6.78 0 0 0.3 0.81 16.97  199.0.65.2 www.mysite.com GET /files/chips.gif HTTP/1.1

---SNIP--- (the list continues through 39)

As far as tuning is concerned, I believe I'm pretty much upto date, at least what the basic stuff is concerned. Here's a snippet from my httpd.conf:

KeepAlive Off
<IfModule prefork.c>
StartServers       5
MinSpareServers    5
MaxSpareServers   15
MaxClients       40
MaxRequestsPerChild  10000
</IfModule>
HostnameLookups Off

I even avoid checking for .htaccess files to make things faster, and for the same reason I don't check follow SymLinks. Maybe I should add at this point that output is gzipped, which I guess puts some more strain on the CPU.
0
 

Author Comment

by:yosmc
ID: 11908567
UPDATE: I ran chkrootkit today just to be on the safe side, but nothing bad was found.

Here's a snippet from my ps -aux output:

apache   24752  2.2  1.3 56084 13504 ?       S    18:30   0:04 /usr/sbin/httpd -k start
apache   24778  3.5  1.1 55076 12032 ?       S    18:31   0:04 /usr/sbin/httpd -k start
apache   24779  1.5  1.2 55324 13032 ?       S    18:31   0:01 /usr/sbin/httpd -k start
apache   24780  2.8  1.2 55256 12516 ?       S    18:31   0:03 /usr/sbin/httpd -k start
apache   24795  3.3  1.2 55776 12496 ?       S    18:32   0:03 /usr/sbin/httpd -k start
apache   24796  1.9  1.2 55436 12672 ?       S    18:32   0:01 /usr/sbin/httpd -k start
apache   24806  1.5  1.1 55276 11804 ?       S    18:32   0:01 /usr/sbin/httpd -k start

As you can see, each Apache process takes 2-3% of CPU time. So if I get my maths right, with mysql and an occasional qmail process running, it takes 30 clients to eat up the rest of the CPU. Would you agree that I'm at least barking up the right tree?
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 15

Expert Comment

by:samri
ID: 11917211
yosmc,

I would suspect that you are right.  However, i can comment much on that since I do not have much expertise on Linux/Unix admin (only knew a few command -- "rm -rf *" :)

but what are there so many "httpd -k start".  It appears that you apache is spawning itself (dies, and restarting).

Try to compare the ps -aux output, and those on /server-status page, and compare the PID?

0
 

Author Comment

by:yosmc
ID: 12145898
Um, I didn't abandon the question, but the number of responses was limited. ;) (samri was very kind, but his input didn't help much.) I have concluded that my celeron processor simply wasn't good enough to handle the load, so I am upgrading to a faster CPU. Would it seem appropriate to close this thread and refund the points?
0
 
LVL 1

Accepted Solution

by:
Computer101 earned 0 total points
ID: 12207778
PAQed, with points refunded (500)

Computer101
E-E Admin
0

Featured Post

Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Maintain Parameters on URL throughout Site (PHP) 6 38
Need assistance modifying php script 3 48
PHPmailer and exchange online(Office365) 3 360
htaccess file 3 79
Over the last year I have answered a couple of basic URL rewriting questions several times so I thought I might as well have a stab at: explaining the basics, providing a few useful links and consolidating some of the most common queries into a sing…
If you've heard about htaccess and it sounds like it does what you want, but you're not sure how it works... well, you're in the right place. Read on. Some Basics #1. It's a file and its filename is .htaccess (yes, with a dot in the front). #…
This Micro Tutorial demonstrates using Microsoft Excel pivot tables, how to reverse engineer competitors' marketing strategies through backlinks.
Two types of users will appreciate AOMEI Backupper Pro: 1 - Those with PCIe drives (and haven't found cloning software that works on them). 2 - Those who want a fast clone of their boot drive (no re-boots needed) and it can clone your drive wh…

773 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question