Solved

Throttle shell_exec?

Posted on 2014-02-15
10
431 Views
Last Modified: 2014-05-12
We have a script that processes very large files occasionally. (EPS graphics for print). Some of the files reach 500M to 1G.

When we process those files on our development server, the server becomes unresponsive until the processing is complete.

Is there a way to limit the amount of CPU consumed when we use shell_exec to call imagemagick?

Is there a way to limit the cpu usage in any other way of calling imagemagick? (Perhaps spinning off a separate thread of apache or PHP an calling imagemagick the PHP way?)
0
Comment
Question by:DrDamnit
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 4
  • 2
10 Comments
 
LVL 16

Expert Comment

by:hankknight
ID: 39862881
PHP does not control the CPU usage of processes run using shell_exec.

However, you may have better success using  mogrify instead of convert.

http://www.imagemagick.org/www/mogrify.html
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39862895
Full disclosure: I have never tried this, but it seems intuitively feasible, and easy to test.

Try starting the processing script with a cURL POST-method request.  The script should be able to run asynchronously.  You may want some kind of a checkpoint file to indicate that the script is complete.

Here is a teaching example showing how to use cURL POST to start a separate script.

<?php // RAY_curl_post_async.php
error_reporting(E_ALL);


// DEMONSTRATE HOW TO USE CURL POST TO START AN ASYNCHRONOUS PROCESS


function curl_post($url, $post_array=array(), $timeout=2, $error_report=FALSE)
{
    // PREPARE THE POST STRING
    $post_string = NULL;
    foreach ($post_array as $key => $val)
    {
        $post_string .= $key . '=' . urlencode($val) . '&';
    }
    $post_string = rtrim($post_string, '&');

    // PREPARE THE CURL CALL
    $curl = curl_init();
    curl_setopt( $curl, CURLOPT_URL,            $url         );
    curl_setopt( $curl, CURLOPT_HEADER,         FALSE        );
    curl_setopt( $curl, CURLOPT_POST,           TRUE         );
    curl_setopt( $curl, CURLOPT_POSTFIELDS,     $post_string );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout     );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE         );

    // EXECUTE THE CURL CALL
    $htm = curl_exec($curl);
    $err = curl_errno($curl);
    $inf = curl_getinfo($curl);

    // ON FAILURE
    if (!$htm)
    {
        // PROCESS ERRORS HERE
        if ($error_report)
        {
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            echo "<pre>\n";
            var_dump($inf);
            echo "</pre>\n";
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS
    curl_close($curl);
    return $htm;
}


// USAGE EXAMPLE CREATES ASSOCIATIVE ARRAY OF KEY=>VALUE PAIRS
$args["name"]  = 'Ray';
$args["email"] = 'Ray.Paseur@Gmail.com';

// ACTIVATE THIS TO SEE THE ARRAY OF ARGS
// var_dump($args);

// SET THE URL
$url = "http://LAPRBass.com/RAY_bounce_post.php";

// CALL CURL TO POST THE DATA
$htm = curl_post($url, $args, 3, TRUE);

// SHOW WHAT CAME BACK, IF ANYTHING
if ($htm)
{
    echo "<pre>";
    echo htmlentities($htm);
}
else
{
    echo "NO RESPONSE YET FROM $url -- MAYBE BECAUSE IT IS RUNNING ASYNCHRONOUSLY";
}

Open in new window

HTH, ~Ray
0
 
LVL 16

Expert Comment

by:hankknight
ID: 39862906
Ray, running this asynchronously will not improve CPU performance.
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39862916
Right, I understand that.  But it may make the task subject to load balancing on the server.  Like I said, it's easy to test (and by implication easy to adopt or discard, depending on the outcome of the tests).
0
 
LVL 32

Author Comment

by:DrDamnit
ID: 39865082
@hank - I'll try mogrify, but I think the problem stems from the hashing that is done when we use identify to determine the resolution and other information. We don't need all the information from identify, is there a way to only pull small information like the resolution without pulling all the identify data?

@ray - we'll certainly try this, but I think imagemagick is running the CPU on the dev box to 100% causing it to be unresponsive. Unless there is a way I can throttle the CPU, starting it with cURL would (likewise) run it to 100%.
0
 
LVL 32

Author Comment

by:DrDamnit
ID: 39868776
What do you guys think of this?

1. Setup a separate instance of apache running as a different user and responding to a different port.
2. Use CPULimit (http://www.howtoforge.com/how-to-limit-cpu-usage-of-a-process-with-cpulimit-debian-ubuntu) to limit the cpu usage of this instance to some value less than 100%.
3. Use cURL to trigger the mogrify command on that instance.

Long way around the barn or good solution?
0
 
LVL 32

Accepted Solution

by:
DrDamnit earned 0 total points
ID: 39901145
bump?
0
 
LVL 16

Expert Comment

by:hankknight
ID: 39902085
Your three-step plan just might work.  Give it a try!
0
 
LVL 32

Author Closing Comment

by:DrDamnit
ID: 40059445
This is what we did.
0

Featured Post

Secure Your WordPress Site: 5 Essential Approaches

WordPress is the web's most popular CMS, but its dominance also makes it a target for attackers. Our eBook will show you how to:

Prevent costly exploits of core and plugin vulnerabilities
Repel automated attacks
Lock down your dashboard, secure your code, and protect your users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Build an array called $myWeek which will hold the array elements Today, Yesterday and then builds up the rest of the week by the name of the day going back 1 week.   (CODE) (CODE) Then you just need to pass your date to the function. If i…
3 proven steps to speed up Magento powered sites. The article focus is on optimizing time to first byte (TTFB), full page caching and configuring server for optimal performance.
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.

724 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question