Throttle shell_exec?

We have a script that processes very large files occasionally. (EPS graphics for print). Some of the files reach 500M to 1G.

When we process those files on our development server, the server becomes unresponsive until the processing is complete.

Is there a way to limit the amount of CPU consumed when we use shell_exec to call imagemagick?

Is there a way to limit the cpu usage in any other way of calling imagemagick? (Perhaps spinning off a separate thread of apache or PHP an calling imagemagick the PHP way?)
LVL 32
DrDamnitAsked:
Who is Participating?
 
DrDamnitConnect With a Mentor Author Commented:
bump?
0
 
hankknightCommented:
PHP does not control the CPU usage of processes run using shell_exec.

However, you may have better success using  mogrify instead of convert.

http://www.imagemagick.org/www/mogrify.html
0
 
Ray PaseurCommented:
Full disclosure: I have never tried this, but it seems intuitively feasible, and easy to test.

Try starting the processing script with a cURL POST-method request.  The script should be able to run asynchronously.  You may want some kind of a checkpoint file to indicate that the script is complete.

Here is a teaching example showing how to use cURL POST to start a separate script.

<?php // RAY_curl_post_async.php
error_reporting(E_ALL);


// DEMONSTRATE HOW TO USE CURL POST TO START AN ASYNCHRONOUS PROCESS


function curl_post($url, $post_array=array(), $timeout=2, $error_report=FALSE)
{
    // PREPARE THE POST STRING
    $post_string = NULL;
    foreach ($post_array as $key => $val)
    {
        $post_string .= $key . '=' . urlencode($val) . '&';
    }
    $post_string = rtrim($post_string, '&');

    // PREPARE THE CURL CALL
    $curl = curl_init();
    curl_setopt( $curl, CURLOPT_URL,            $url         );
    curl_setopt( $curl, CURLOPT_HEADER,         FALSE        );
    curl_setopt( $curl, CURLOPT_POST,           TRUE         );
    curl_setopt( $curl, CURLOPT_POSTFIELDS,     $post_string );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout     );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE         );

    // EXECUTE THE CURL CALL
    $htm = curl_exec($curl);
    $err = curl_errno($curl);
    $inf = curl_getinfo($curl);

    // ON FAILURE
    if (!$htm)
    {
        // PROCESS ERRORS HERE
        if ($error_report)
        {
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            echo "<pre>\n";
            var_dump($inf);
            echo "</pre>\n";
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS
    curl_close($curl);
    return $htm;
}


// USAGE EXAMPLE CREATES ASSOCIATIVE ARRAY OF KEY=>VALUE PAIRS
$args["name"]  = 'Ray';
$args["email"] = 'Ray.Paseur@Gmail.com';

// ACTIVATE THIS TO SEE THE ARRAY OF ARGS
// var_dump($args);

// SET THE URL
$url = "http://LAPRBass.com/RAY_bounce_post.php";

// CALL CURL TO POST THE DATA
$htm = curl_post($url, $args, 3, TRUE);

// SHOW WHAT CAME BACK, IF ANYTHING
if ($htm)
{
    echo "<pre>";
    echo htmlentities($htm);
}
else
{
    echo "NO RESPONSE YET FROM $url -- MAYBE BECAUSE IT IS RUNNING ASYNCHRONOUSLY";
}

Open in new window

HTH, ~Ray
0
Cloud Class® Course: Certified Penetration Testing

This CPTE Certified Penetration Testing Engineer course covers everything you need to know about becoming a Certified Penetration Testing Engineer. Career Path: Professional roles include Ethical Hackers, Security Consultants, System Administrators, and Chief Security Officers.

 
hankknightCommented:
Ray, running this asynchronously will not improve CPU performance.
0
 
Ray PaseurCommented:
Right, I understand that.  But it may make the task subject to load balancing on the server.  Like I said, it's easy to test (and by implication easy to adopt or discard, depending on the outcome of the tests).
0
 
DrDamnitAuthor Commented:
@hank - I'll try mogrify, but I think the problem stems from the hashing that is done when we use identify to determine the resolution and other information. We don't need all the information from identify, is there a way to only pull small information like the resolution without pulling all the identify data?

@ray - we'll certainly try this, but I think imagemagick is running the CPU on the dev box to 100% causing it to be unresponsive. Unless there is a way I can throttle the CPU, starting it with cURL would (likewise) run it to 100%.
0
 
DrDamnitAuthor Commented:
What do you guys think of this?

1. Setup a separate instance of apache running as a different user and responding to a different port.
2. Use CPULimit (http://www.howtoforge.com/how-to-limit-cpu-usage-of-a-process-with-cpulimit-debian-ubuntu) to limit the cpu usage of this instance to some value less than 100%.
3. Use cURL to trigger the mogrify command on that instance.

Long way around the barn or good solution?
0
 
hankknightCommented:
Your three-step plan just might work.  Give it a try!
0
 
DrDamnitAuthor Commented:
This is what we did.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.