Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

PHP dealing with high load spikes.

Posted on 2011-09-13
6
Medium Priority
?
588 Views
Last Modified: 2013-11-08
Hi,
We have several servers each with arround 50 -80 sites on them. Each site runs an instace of our large ecommerce system.
The problem we are getting is dealing with load on the server, but this seems mainly to be becuase of backups.
The backups start very early in the morning arround 4am, but some days have been continueing running right up till 1 or 2pm. at random times during the backups, the load will spike and the application will not be able to run, but then there are other times when the load is really high and the app can sort of still run, but 1. its slow, and 2. its probably not helping as it may be helping it reach such high loads. Load can spike from 3.4 - 5 right up to 500!
Aside from any ideas of how to fix the issue with the load from the backups, does anyone have any ideas about how the application should handle this. E.g. Should we try to detect the load at the start of the php application, and get it to sleep for a bit till its better, or exit/redirect to a static error page?
I realize that dealing with it at all at the php app it self level seems like avoiding the problem, but i think we may always get spikes in the load anyway from maybe spikes in visitors at random times and so this would help with all cases.
Thanks is anyone has any ideas.
0
Comment
Question by:stilliard
6 Comments
 
LVL 24

Accepted Solution

by:
slyong earned 668 total points
ID: 36533710
Hi,

I think you understand the disadvantage dealing with such situation using PHP.  However, if you want to have a script to check server load and redirect, you might want to try:

<?php
$load = sys_getloadavg();
if ($load[0] > 80) {
    header('HTTP/1.1 503 Too busy, try again later');
    die('Server too busy. Please try again later.');
}
?>

Open in new window


ref: http://de.php.net/manual/en/function.sys-getloadavg.php
0
 
LVL 24

Expert Comment

by:slyong
ID: 36533714
Sorry another way to do it is to set a timeout and register_shutdown_function to make PHP redirect to somewhere:

set_time_limit(1);
function shutdown () {
    if ( ! defined('FINISHED')) {
        header('Location: /timeout');
    }
}

register_shutdown_function('shutdown');

while (true) {}

define('FINISHED', 1);

Open in new window


ref: http://stackoverflow.com/questions/6759735/can-i-configure-apache-to-redirect-at-a-specified-timeout
0
 
LVL 10

Assisted Solution

by:ray-solomon
ray-solomon earned 664 total points
ID: 36534496
Some random insights/tips that may help.

I also have several servers as big as yours. The process of backing up sites to tar.gz files is very CPU intense and mysql consumes a lot of memory when databases are being dumped. This is normal. So during the backup process I make sure there's no other script on any of our sites that is running any big mysql queries.

I also adjust my backup to start late at night at 10pm when traffic is lower because it takes about 9 hours to complete. Better than during the day.

Make sure you are not actually running out of RAM and using swap during the backup process. If so, add more RAM or else apache and other things will start dying randomly. That will help keep your sites operating normally during a backup. Can't have too much ram.

I adjust most cron jobs around the backup process.

As for the php applications. If your applications are doing database queries, make sure you have the proper indexes on your tables where applicable. A lot of people I help get this wrong.

Raid 10 is the way to go if you can afford it. Otherwise throwing more RAM and better CPU is useless if your Hard drive read/write speeds cant keep up with all those sites doing things during the backup or heavy traffic.
0
Configuration Guide and Best Practices

Read the guide to learn how to orchestrate Data ONTAP, create application-consistent backups and enable fast recovery from NetApp storage snapshots. Version 9.5 also contains performance and scalability enhancements to meet the needs of the largest enterprise environments.

 
LVL 26

Expert Comment

by:arober11
ID: 36538744
What are you backing up?
0
 
LVL 6

Author Comment

by:stilliard
ID: 36542002
@slyong Thanks sys_getloadavg is great for telling the point when the app may need to be stoped, thanks i'll look into it more to decide the best cut off point.
@ray-solomon Thanks for the tips, i'll definetly be looking to through more RAM at it and look at some better hardware to support it.
@arober11 All the sites files and mysql db backups. However im glad you've asked hat question, as we could definetly look to cut down the parts we actually need backups of in terms of files. The sites currently all run the same system, so i gues the backups would only be needed for there own user files folder and any config files.
0
 
LVL 26

Assisted Solution

by:arober11
arober11 earned 668 total points
ID: 36545546
You could also set up another mySQL instance, on a separate server and use basic mySQL replication (master / slave) to maintain a remote copy of your DB's, then simply break the mirror, backup from the slave, then re-enable the replication. Would shift the majority of the load elsewhere along with offering some redundancy.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article discusses how to create an extensible mechanism for linked drop downs.
Originally, this post was published on Monitis Blog, you can check it here . In business circles, we sometimes hear that today is the “age of the customer.” And so it is. Thanks to the enormous advances over the past few years in consumer techno…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses

886 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question