I hope I am posting this in the right section, but I do believe the reason for the problem is Apache/MySQL.
I'm running rh9 on a Celeron 1.7. I'm currently serving about 100,000 pages per day, all of them are php pages and rely (more or less heavily) on a MySQL database. My problem is that with this volume (roughly translates to 3 gigs per day), the CPU is reaching 100% and processes start building up during peak times, pushing the server load beyond 10 up to as much as 50 (according to top).
18:55:22 up 5 days, 22:56, 1 user, load average: 7.31, 6.30, 5.92
162 processes: 141 sleeping, 21 running, 0 zombie, 0 stopped
CPU states: 84.0% user 15.9% system 0.0% nice 0.0% iowait 0.0% idle
Mem: 1022796k av, 792168k used, 230628k free, 0k shrd, 7900k buff
470300k actv, 122056k in_d, 18972k in_c
Swap: 1052248k av, 77848k used, 974400k free 497896k cached
This snapshot is rather harmless, it can get much worse. As you can see, I have 1 gig of RAM installed and no, I am not swapping (so memory doesn't seem to be the problem). I do use the server for my own mail (qmail) and have to fight off a lot of spam (using spam assassin), but a basic and reliable test (apachectl stop :) reveals that the vast majority of the load is due to Apache/mySQL.
I am aware that php is CPU intensive, and my CPU is not the fastest around, but I believe it should be able to handle the load without the problems I am having. True?
P.S. I have installed Zend Accelerator to solve the problem - that get's me up to 150,00 pageviews and 4,5 gigs per day, but then the problem resurfaces.