4000-5000 Hits per hour. Suggested apache hardware & settings

Well let's start with your current configuatation, 'cos 5000 "hits" does not sound like a lot.

What hardware: CPU, Memory, Network card, type of network.
Apache version
Application? CGI Scripts? or just plain old HTML pages?

Current Configuration:
Celeron 900
256 Ram
Netgear 10/100
Apache 2
PHP/MySQL Backend

I had a look at the conf, and bumped up MinSpareServers from 4 to 30 and MaxSpareServers from 8 to 60.

I also set KeepAlive On for 15 seconds/1000 req's.

Comments?

Paul.

If this is a Linux box it should go like the clappers. Basically however the more memory you throw in the faster these things go. I presume that the network is running at 100MB around the school and this is not a bottleneck. But before we start getting you to spend money on more memory, what OS are you using, what browser and what *looks slow* on the browser.

I agree with samri about mysql and memory. But I can't imagine a database large enough to cause problems.

I had an Apache running with 1GB+256MB with MySQL table about (65536  * 4) rows , and each rows had foreign key reference to another 4-5 tables.  Pages are generated using CGI (Perl).  Typical searches took almost 1 minutes.  And the machine is running (you can already guess) Win2K Advance Server.  Pretty neat huh!  Why not Unix/Linux -- talk to the guy in the glass window.  basically company policy.

my main point is, the problem may lies on the OS.

RedHat Linux 7.3

I will give apache benchmark a go.

4000-5000 requests per hour is the average of course, but do note that there will be up to 30 requests all at the exact same time, as students will be following a teachers commands.

The network is not a bottleneck, and I will consider getting more ram :)

The thing everyone is clicking on is PHP, with MySQL db access. I was observing top and not much swapping was going on, however it was definitely apache doing all the hard work! Didnt note any outrageous mysql processes.

Cheers,

Paul.

The browser we are using is IE 6. Looks slow is defined as pages not loading, timing out etc.

Server load has been up at 140 at times!!!

Paul.

"it was definitely apache doing all the hard work"

Actually this is the build-in PHP processor. The Mysql should be at least 4.0.

It does seem that the CPU is running out of power, if the performance show little disk activity. I'd suspect this (with reservations) since all requests would be for the same data. The first request which got to the Mysql server would cause the data to be transfered from the disk, all others would reuse the same data. It's then simply a CPU matter to format the data and throw it out onto the net. But a 900MHz Celeron should have enough power to do this.

Is Mysql at least 4.0? Do you have any Apache modules active (like gzip)? What tool are you using to measure CPU and Disk activity?

gzip is compiled, my current performance measurement has been only from the client machines.

I found out today that if I hold down F5 (refresh), the server pretty much freezes... That was on one machine. Load average jumped up to 150.. ram swapped to disk... so on.

I estimate perhaps 25 - 30 connections at one moment. What should the settings be for MinSpareServers and MaxSpareServers.

Cheers,

Paul.

"I found out today that if I hold down F5 (refresh), the server pretty much freezes..."

That's what I'd expect on a LAN where the network speed is so high that it doesn't slow down the request rate.

If mod_gzip is active it's compressing the returned output and this is not necessary on a LAN, and it will in fact overload the CPU. So I'd turn it off if its on.

The MinSpareServers really does not make an awfull lot of difference. Set at 30 it should be able to handle all your requests. If too many come in, Apache creates new servers up to max.

I'd still like to know how you are measuring the performance. With what tool? I have Suse Linux here not RedHat.

I think you might consider turning down or even off the KeepAlive. This option keeps a permanent connection between browser and server into which requests are placed.  The difference will be marginal though.

I'd like to see a vmstat 1 100 (say) from when the requests come in until the machine goes quiet.

If all the students are loading the same pages, you might consider setting up Squid proxy cache on the machine to cache a lot of the static content on the pages.  We have seen this improve performance by orders of magnitude.

-Larry

[root@router root]# ab -n 30 -c 10 http://localhost/moodle/

Benchmarking localhost (be patient).....done
Server Software:        Apache/1.3.27
Server Hostname:        localhost
Server Port:            80

Document Path:          /moodle/
Document Length:        15925 bytes

Concurrency Level:      10
Time taken for tests:   17.439 seconds
Complete requests:      30
Failed requests:        0
Broken pipe errors:     0
Total transferred:      490980 bytes
HTML transferred:       477750 bytes
Requests per second:    1.72 [#/sec] (mean)
Time per request:       5813.00 [ms] (mean)
Time per request:       581.30 [ms] (mean, across all concurrent requests)
Transfer rate:          28.15 [Kbytes/sec] received

Connnection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    97  175.8      0   576
Processing:  4159  5235  525.0   5277  6494
Waiting:     4159  5214  516.9   5216  6430
Total:       4159  5332  575.8   5377  6630

Percentage of the requests served within a certain time (ms)
  50%   5377
  66%   5464
  75%   5665
  80%   5927
  90%   6183
  95%   6205
  98%   6630
  99%   6630
 100%   6630 (last request)

--------------

[root@router root]# vmstat 1 100
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 8  0  0  53568  14596  30032  61160   1   2     1     9   61    24   2   1  36
10  0  0  53568   9168  30032  61160   0   0     0     0  383   109  90  10   0
10  0  0  53568   3596  30032  60852   0   0     0     0  487   116  89  11   0
10  0  0  53700   3372  29884  61000   0 348     0   508  466  1672  92   8   0
10  0  0  53796   7272  29728  61184   0 356     0   356  352  1503  91   9   0
17  2  1  53796  14384  29732  61192   0   0     0   216  415   289  93   7   0
16  0  0  53796  14620  29744  61192   0   0     0   160  429   148  92   8   0
11 25  1  53796   5976  29752  61208   0   0    16   236  504   218  86  14   0
13  5  0  53928   4256  29608  61400  16 332    56  1452  667  1132  80  13   7
14  0  0  53928   4096  29608  61400   0   0     0     0  384  1281  92   8   0
15  0  0  54060   6736  29460  61568   0 356     0   356  372   788  93   7   0
14  0  0  54192   8380  29336  61708   0 132     0   132  416   487  91   9   0
14  0  0  54192   5724  29336  61708   0   0     0     0  413   175  88  12   0
14  0  1  54720   3632  28652  61804   0 1392     0  1720  459   213  90  10   0
14  0  0  55776   3444  27604  62388  52 1504    52  1504  446   893  90  10   0
13  0  0  55896   5536  27536  62448   0 328    72   328  496  1400  89  11   0
12  0  0  55888  11232  27544  62580 180   0   184     0  377   773  93   7   0
12  0  0  56020   9048  27540  62588 148 132   148   132  341   431  90  10   0
12  0  0  56020   7396  27540  62596  44   0    44   428  471   377  89  11   0
11  0  0  56164   4468  27480  62656   0 292     0   292  423   421  91   9   0
11  0  0  57432   7252  27232  62740   0 1416     0  1416  426  1181  88  12   0
   procs                      memory    swap          io     system         cpu
 r  b  w   swpd   free   buff  cache  si  so    bi    bo   in    cs  us  sy  id
 2  0  0  57428  13660  27252  63084   0   0   204   240  431   727  75  10  15
 0  0  0  57352  19920  27252  63088   0   0     0     0  362    53   2   2  96
 0  0  0  57352  19928  27260  63088   0   0     0   368  410    44   1   2  97
 0  0  0  57352  19928  27260  63088   0   0     0     0  338    19   1   1  98
 0  0  0  57352  19928  27260  63088   0   0     0     0  399    23   1   1  98

What version of MySQL are you running? Older versions lack query cacheing, which would speed up repeated access to dynamic pages massively (similar to the Squid proxy suggestions, above.)

Also, are your SQL queries optimised? Not only should the SQL be efficient, but you need to make sure your tables are stored efficiently, with proper indexes. In Postgres there are commands to analyse SQL and find out whether it will do a linear scan or an index lookup, but I don't know how to do it in MySQL; any MySQL experts care to comment? If you are doing a scan of the tables every time you get a request, it would certainly explain the poor performance.

If you need to upgrade hardware (although I think that should not be necessary) I would upgrade the RAM first - 256mb is not a lot for disk cache if you have a lot of large MySQL tables to access. Also, what kind of hard disk configuration are you using? SCSI/IDE? RAID level? RPM? And are you doing mostly read or mostly write access? Doing a lot of writes in RAID level 0 (mirroring) would slow you down a lot.

In reply to The--Captain's post regarding his statistics, you were only downloading a static image in your test.

When I download a static file (58kb):
Connnection Times (ms)
              min  mean[+/-sd] median   max
Connect:        4    10    4.1     10    18
Processing:    12    26    7.3     24    36
Waiting:       10    24    7.8     22    34
Total:         30    36    4.0     37    41

So I thus conclude that it is either PHP or MySQL that is causing the slowdown here.

We are using a package called Moodle (http://moodle.org), and I would assume it has been written fairly efficiently, as it is designed for a large number of users. The version of MySQL running is: Ver 12.22 Distrib 4.0.16, for pc-linux (i686)

When watching in top, as all the students click on, the processes using the most CPU are the httpd ones. So I'm guessing that its specificaly the php library that is using the processor here. This installation of apache was the one that came with RedHat 7.3, and PHP was included in an rpm update from the redhat site.

Would it make any difference if I compiled apache from source, and php as well?

Thanks for all the help!!

Paul.

The machine is dedicated only to hosting this site. There are no large files. The system is not compromised.

The hard disk configuration is simple. A single IDE hdd of about 20GB.

There is only about 400k of data in the database.

Paul.

I havent modified moodle at all. I will definitely investigate more ram, and possibly in 6-12 months, a new server. In the meanwhile I will do performance tests on MySQL, and await the arrival of the new version of Moodle.

Thanks everyone for your contributions!!

Paul.