Avatar of benners70
benners70
 asked on

How to queue and run PHP Imagick script on Windows Server for background image processing

I'm running Codeigniter on a Windows Web Server 2008 and need to queue and run a PHP, Imagemagick script to process hi-res images. I need help designing the best solution for this.

My current thinking is I could run Task Scheduler every 10 mins to run WGET  to make an http request to a PHP script that checks for queued jobs.

If this is a sensible solution I have the following queries:

1. If I have 100 jobs queued and each jobs runs the PHP script for about 3 minutes I can't keep looping as the script will timeout. How do I close the script and then check if there is another job to process and ensure I'm only processing 1 job at a time on the server.

2. I've checked here http://php.net/manual/en/function.set-time-limit.php and I'm not quite clear what is the limit on Windows server. max_execution_time is not set in my php.ini but the script seems to run OK even though it 2-3 minutes.
PHP

Avatar of undefined
Last Comment
Julian Hansen

8/22/2022 - Mon
Julian Hansen

Firstly set_time_limit can be used indefinitely in your script - call it regularly to extend the timeout for the script. However if there is a process that takes a long time that you don't have access to you will need to set this to a high value rather than for instance processing a loop where you can set it to say 30seconds each time you loop.

However, I would not use that approach here - it will work but it is inefficient.

I would instead use a folder watcher - an application that monitors a specific folder for a create / update event and then executes a script based on that event.

I wrote a C# service a while back I could probably dig out that allows you to do this - you specify in the registry what folders you want to watch for what events and what script to run when an event is triggered. It was created for a company in the movie industry wanting to process frames when they were complete from a rendering process - sounds similar to what you are doing.

I am sure there are othe similar utilities on the web - mine is a bit nuts and bolts - no nice GUI for configuring it but it works. Alternatively you can build a .Net app with a FileSystemWatcher - here are some links

http://www.techrepublic.com/article/use-the-net-filesystemwatcher-object-to-monitor-directory-changes-in-c/6165137

http://creatingcode.com/2010/02/using-net-to-monitor-a-directory-for-changes/
benners70

ASKER
Hi JulianH

Thanks for your comments. I don't actually need to watch a folder which I store images to be processsed. My PHP script seems to handle the large image processing OK, I just need something that checks a DB queue folder and then runs the script with the parameters stored in the DB. I assume PHP can be run via CMD but I'm not familiar with this so I was thinking of using WGET.

I've done a bit more reading and it seems that a Windows version of beanstalkd might do the job, although I've not been able to find something.

Thank you for the offer of the C# service but I need something a little less hardcore, perhaps something with a nice GUI :)
Julian Hansen

The FileSystemWatcher solution does not have to be used on the actual images - what you do is when jobs are ready you write them to a manifest file in a specified folder - this wakes up the FSW which then processes the job. The job files can be processed sequentially by the script so you won't have it so that more than one job is executing at a time.

PHP can be run from the command line

php myscript.php [param1 [param2 ..]]

The advantage to the above is that when there is nothing to do your server does not execute jobs unnecessarily

Maybe I am not understanding exactly what you want to do?
Your help has saved me hundreds of hours of internet surfing.
fblack61
benners70

ASKER
No you do understand and that does sound ideal. What is the manifest file? Just a text document with PHP commands?

php myscript.php [param1 [param2 ..]]
php myscript.php [param1 [param2 ..]]
php myscript.php [param1 [param2 ..]]

I don't know anything about C# and adding a service though. Can you tell me how to do it or is there an something else that does this that has some documentation? I don't mind paying a little if it save me time and is easier for me to manage.

Thanks
ASKER CERTIFIED SOLUTION
Julian Hansen

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question