I need to run a PHP script for a while - how Does PHP Maximum Execution Time Work?

I have a PHP script that I am testing that works as it should and completes in a little under 3 minutes with a certain size data input test file.

When I move to production, the data input file could potentially be 15 times this large (making execution almost 45 minutes for a million records.)

What do I need to consider? I'm not sure why the script is not being interrupted now - in Webmin > PHP Configuration > Resource Limits, my Maximum Execution Time is set to 30 seconds, and the Maximum Input Parsing Time is set at 60 seconds.

Am I considering all PHP Config options involved here?
What should I increase these options to?
What pro's/con's do I need to consider if I do increase these parameters?
Joel BuhrPresidentAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Dave BaldwinFixer of ProblemsCommented:
PHP max execution time doesn't apply to PHP on the command line, only when running thru a web server.  

http://php.net/manual/en/info.configuration.php#ini.max-execution-time

http://php.net/manual/en/features.commandline.differences.php
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ray PaseurCommented:
You can set and reset the time limit at any time.  If you put this into a loop, you may be able to create a long-running job.
http://php.net/manual/en/function.set-time-limit.php

I emphasize may because PHP is not really made for what you're describing, and the server that is running the PHP script may have time limits, too.  What is the application and why does it take so long to run?
0
Bernard S.CTOCommented:
Problem with running a long job is that it might gobble a significant part of the available CPU resources: that's why there are these limits: if a web site or some other process is running on the machine... it might slow down for a long time.

We have some jobs that need lots of cpu cycles: we have chosen that they will run only 5 sec (which is in fact HUGE!), but have put in place some kind of "progress counter" which allow the job to resume later.
With a cron job, we run this every 15 minutes: we are them pretty sure that user experience on our sites in not impacyed heavily.

Usually such jobs are, as in our case, database related: use transactions (so that your data is kept consistant even if the program is interrupted) and update the counter just at the end of the transaction.

That's the idea, now as Ray suggested you should explain what you want to achieve: brute force computing? batch updates of data?
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

GaryCommented:
Check you have the right columns indexed and not columns that don't need it
Optimize your sql
What are some samples of your queries?
0
MurfurFull Stack DeveloperCommented:
To add to DB's comment, this sounds like a cronjob waiting to happen!

I would create a results table in the database and get the cronjob to post the results of the lengthy query to the results table. That way a site page can simply query the results table in a fraction of the time.

Commonly a cronjob will use wget to call a web page but wget will timeout just as a browser so as Dave has said, you use PHP on the command line. The caveat is that this needs to be run on the same server as the web site so that the php command refers to a local file e.g. this:

* */2 * * * php /www/sitename.com/html/filename.php  >/dev/null

One thing to consider is how often you want to re-run the query to update the results which is probably  driven by how often the source data changes. If it the query takes 45 mins to generate results then a frequency of a few hours may seem appropriate and probably not less than 2. The example above would run every two hours and subject to your server configuration would only "reply" if there are errors to report i.e. cron failed to run the job but there is nothing to stop you making your own reporting service within the php file. For example, at the simple end of things, you only need to echo result statements and these will typically be emailed to the user account who is running the job. Or you can build your own more complex process to use sendmail etc. to deliver a more complexly structured message such as a summary of the query result or formatting the email content so that the company execs get a graph of sales figures etc. The world is, as they say, your oyster. Code away...

Finally, you mention "Webmin > PHP Configuration" so I'm guessing that you are using a GUI front end to administer the server like Webmin or PLESK. If so then look for the cronjob options or "scheduled jobs" in there which will give you an easier method of administering the schedule.
0
Joel BuhrPresidentAuthor Commented:
Thanks, Dave.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
PHP

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.