How to queue and run PHP Imagick script on Windows Server for background image processing

benners70
benners70 used Ask the Experts™
on
I'm running Codeigniter on a Windows Web Server 2008 and need to queue and run a PHP, Imagemagick script to process hi-res images. I need help designing the best solution for this.

My current thinking is I could run Task Scheduler every 10 mins to run WGET  to make an http request to a PHP script that checks for queued jobs.

If this is a sensible solution I have the following queries:

1. If I have 100 jobs queued and each jobs runs the PHP script for about 3 minutes I can't keep looping as the script will timeout. How do I close the script and then check if there is another job to process and ensure I'm only processing 1 job at a time on the server.

2. I've checked here http://php.net/manual/en/function.set-time-limit.php and I'm not quite clear what is the limit on Windows server. max_execution_time is not set in my php.ini but the script seems to run OK even though it 2-3 minutes.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Most Valuable Expert 2017
Distinguished Expert 2018

Commented:
Firstly set_time_limit can be used indefinitely in your script - call it regularly to extend the timeout for the script. However if there is a process that takes a long time that you don't have access to you will need to set this to a high value rather than for instance processing a loop where you can set it to say 30seconds each time you loop.

However, I would not use that approach here - it will work but it is inefficient.

I would instead use a folder watcher - an application that monitors a specific folder for a create / update event and then executes a script based on that event.

I wrote a C# service a while back I could probably dig out that allows you to do this - you specify in the registry what folders you want to watch for what events and what script to run when an event is triggered. It was created for a company in the movie industry wanting to process frames when they were complete from a rendering process - sounds similar to what you are doing.

I am sure there are othe similar utilities on the web - mine is a bit nuts and bolts - no nice GUI for configuring it but it works. Alternatively you can build a .Net app with a FileSystemWatcher - here are some links

http://www.techrepublic.com/article/use-the-net-filesystemwatcher-object-to-monitor-directory-changes-in-c/6165137

http://creatingcode.com/2010/02/using-net-to-monitor-a-directory-for-changes/

Author

Commented:
Hi JulianH

Thanks for your comments. I don't actually need to watch a folder which I store images to be processsed. My PHP script seems to handle the large image processing OK, I just need something that checks a DB queue folder and then runs the script with the parameters stored in the DB. I assume PHP can be run via CMD but I'm not familiar with this so I was thinking of using WGET.

I've done a bit more reading and it seems that a Windows version of beanstalkd might do the job, although I've not been able to find something.

Thank you for the offer of the C# service but I need something a little less hardcore, perhaps something with a nice GUI :)
Most Valuable Expert 2017
Distinguished Expert 2018

Commented:
The FileSystemWatcher solution does not have to be used on the actual images - what you do is when jobs are ready you write them to a manifest file in a specified folder - this wakes up the FSW which then processes the job. The job files can be processed sequentially by the script so you won't have it so that more than one job is executing at a time.

PHP can be run from the command line

php myscript.php [param1 [param2 ..]]

The advantage to the above is that when there is nothing to do your server does not execute jobs unnecessarily

Maybe I am not understanding exactly what you want to do?

Author

Commented:
No you do understand and that does sound ideal. What is the manifest file? Just a text document with PHP commands?

php myscript.php [param1 [param2 ..]]
php myscript.php [param1 [param2 ..]]
php myscript.php [param1 [param2 ..]]

I don't know anything about C# and adding a service though. Can you tell me how to do it or is there an something else that does this that has some documentation? I don't mind paying a little if it save me time and is easier for me to manage.

Thanks
Most Valuable Expert 2017
Distinguished Expert 2018
Commented:
Yes - sorry - a manifest file is just a text file - You can download a version of NimbleFolders (the software I mentioned before) here

http://www.marcorpsa.com/downloads/nimblefolders.1.0.zip

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial