Solved

PHP bulk inserting records into multiple tables from a scheduled task/cron job

Posted on 2014-10-22
8
261 Views
Last Modified: 2014-10-24
I have a pure php page that is set up to run about  a 100 transactions where I am adding and updating to several tables and making a call to a payment processor.  Everything is working nicely.

I have done this many times on the asp/vb side.  What do I need to do in order to make sure the page does not time out.  Just the set time limit?  http://php.net/manual/en/function.set-time-limit.php. I  would prefer to make this change in the scripting than an ini setting.

This is something that is going to be scheduled to run every night so I don't really expect any more than 100 transactions and it will probably be a lot less and I don't have to worry about this. But if I want to make sure I could run more.  Any tips and gotchas to know.

This is on windows and my plan is to have scheduled task hit a vbs script that will post to the php page.   Would it be best to just use CLI?  Instead of hitting the vbs, just run http://php.net/manual/en/install.windows.commandline.php C:\PHP5\php.exe -f "C:\PHP Scripts\script.php" -- -arg1 -arg2 -arg3
0
Comment
Question by:Scott Fell,  EE MVE
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
  • 2
  • +1
8 Comments
 
LVL 9

Accepted Solution

by:
Brian Tao earned 200 total points
ID: 40398567
1) yes, just add the following line to your script
set_time_limit(0);

Open in new window


2) I've been using a .bat file to run my PHP script for quite a long time.  The batch file should be similar to the following:
REM usage: php.exe -q processEmpData.php Instance ArchivedAfterLoad
REM valid values for Instance: PROD, STAGING, DEV
REM valid values for ArchivedAfterLoad: true, false;
php.exe -q processEmpData.php PROD true

Open in new window


Note: The lines start with "REM" are just comments in Windows batch file.
0
 
LVL 83

Assisted Solution

by:Dave Baldwin
Dave Baldwin earned 100 total points
ID: 40398572
Command line PHP doesn't have a timeout.  More info here: http://us1.php.net/manual/en/info.configuration.php#ini.max-execution-time
0
 
LVL 9

Expert Comment

by:Brian Tao
ID: 40398583
@Dave Baldwin,
Command line PHP does have a timeout.  It's just default to 0.  It's specified in the link you provided.
When running PHP from the command line the default setting is 0.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 83

Expert Comment

by:Dave Baldwin
ID: 40398747
'0' means no time out.  Another reference here: http://us1.php.net/manual/en/features.commandline.differences.php
0
 
LVL 110

Assisted Solution

by:Ray Paseur
Ray Paseur earned 200 total points
ID: 40399131
I don't know all the answers here, especially not on Windows, but I can tell you what has worked for me most of the time (in a shared hosting environment on Linux).  With each transaction I do two things:

1. set_time_limit(10) (not zero)
2. create browser output, perhaps echo the SQL query string.

Using this strategy I've often been able to run several thousand queries over an hour or more without interruption.  Another piece of the strategy might appear if you know what should happen with each query.  You might set up an internal "checklist" for each query and check it off as it is run successfully.  Then at the end, if there is anything left undone, send yourself and email message.

With a zero timeout your script can run forever.  You definitely do not want that; you want an interruption if you've got a loop or a hung external process.  And 10 seconds is long enough for just about anything.  So for each query or transaction or other unit-of-work, I just set the time limit over and over again.  This extends the effective time limit of the script without the risk of an endless loop.
0
 
LVL 53

Author Comment

by:Scott Fell, EE MVE
ID: 40399264
Ray, are you flushing the output?
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 40399863
That's a whole different question with a lot of moving parts.  You probably can't go wrong with flush() in the script, but there are a lot of ways that PHP, initialization settings, your server and the client's browser can interact to frustrate your efforts.  Some of this is described on the flush() man page.
0
 
LVL 53

Author Comment

by:Scott Fell, EE MVE
ID: 40401193
For some reason, I couldn't use  the format suggested and it must have something to do with plesk.  I tried many variations and nothing worked.   It's even outlined in the plesk KB.  http://kb.sp.parallels.com/en/115292.  My web host support suggested to use the powershell version and that's what I did.

Path to executable file: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
    Arguments: -c "(new-object system.net.webclient).downloadstring('http://domain.test/folder/page.php')"

Open in new window

0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Popularity Can Be Measured Sometimes we deal with questions of popularity, and we need a way to collect opinions from our clients.  This article shows a simple teaching example of how we might elect a favorite color by letting our clients vote for …
I imagine that there are some, like me, who require a way of getting currency exchange rates for implementation in web project from time to time, so I thought I would share a solution that I have developed for this purpose. It turns out that Yaho…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

632 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question