ParadyneDesigns
asked on
Starting point in a foreach()
Hello
I have a tab delimited text file which contains 9,000 rows and 23 columns. I am importing this into a MySQL database. However, the script which imports it is timing out around row 4860.
What I want to do is make it so the script gets to row 1000, then stops and refreshes and continues from that point to break the process down a bit. I can store the row number in a SQL database ready to be read and continued from by the script on refresh.
However, the method I am using to break the file down is as follows.
if (file_exists($file_name)) { // Check the file can be opened
$file_to_process = file($file_name); // put file into an array
Then once I have the file in an array I do the following to process each row
foreach($file_to_process as $row) {
$data = explode("\t",$row);
// From here I do my validation and insert into the database
However, if I am restarting the script, my current script is going to always start from row zero and will never get anywhere. How can I do the foreach starting at row 1001 (or whatever value I call from the database to continue from) so that only the data which has not been processed is then processed.
Also, each time the script runs I need to call row 0 to get the column names as this is part of my verification system.
Note: The 9,000 rows is an example. The file could contains 23,000 rows, or just 10 rows. I depends on what my customers upload to be processed.
I did wonder if it was possible to do the following
foreach($file_to_process[i
and then i++ after processing although I have a strong feeling that would not work as it would only retrieve one row.
I am sure I am missing something simple but for the life of me cannot figure it out.
Cheers
I have a tab delimited text file which contains 9,000 rows and 23 columns. I am importing this into a MySQL database. However, the script which imports it is timing out around row 4860.
What I want to do is make it so the script gets to row 1000, then stops and refreshes and continues from that point to break the process down a bit. I can store the row number in a SQL database ready to be read and continued from by the script on refresh.
However, the method I am using to break the file down is as follows.
if (file_exists($file_name)) { // Check the file can be opened
$file_to_process = file($file_name); // put file into an array
Then once I have the file in an array I do the following to process each row
foreach($file_to_process as $row) {
$data = explode("\t",$row);
// From here I do my validation and insert into the database
However, if I am restarting the script, my current script is going to always start from row zero and will never get anywhere. How can I do the foreach starting at row 1001 (or whatever value I call from the database to continue from) so that only the data which has not been processed is then processed.
Also, each time the script runs I need to call row 0 to get the column names as this is part of my verification system.
Note: The 9,000 rows is an example. The file could contains 23,000 rows, or just 10 rows. I depends on what my customers upload to be processed.
I did wonder if it was possible to do the following
foreach($file_to_process[i
and then i++ after processing although I have a strong feeling that would not work as it would only retrieve one row.
I am sure I am missing something simple but for the life of me cannot figure it out.
Cheers
$starting_offset = "0";
$stopping_offset = "999";
$current_count = "0";
for($i = $starting_offset; $stopping_offset; $i ++){
blabla
$current_count ++;
}
use the current count to check if all was succesfull and to know where the seccond run shoudl start...
$stopping_offset = "999";
$current_count = "0";
for($i = $starting_offset; $stopping_offset; $i ++){
blabla
$current_count ++;
}
use the current count to check if all was succesfull and to know where the seccond run shoudl start...
You might also want to read this function (do be carefull though)
http://nl3.php.net/manual/en/function.set-time-limit.php
regards,
http://nl3.php.net/manual/en/function.set-time-limit.php
regards,
oeps, that for will walk forever :P
for($i = $starting_offset; $i <= $stopping_offset; $i ++)
for($i = $starting_offset; $i <= $stopping_offset; $i ++)
ASKER
Thanks for the ideas but the loop suggested still would not split the array. Each iteration would still commence from the begining of the array. What I needed was something to move the internal pointer to the new starting point in the array when the script restarts.
What I have done in the time being is add an array slice to see if that will work. I am just uploading the file to my FTP to run the tasks and seeing it that solved it.
The time set limit would not really have worked either as a 16,000 row file would take my server around 1 hour to process (the validation in my cron script is extremely intensive (I forgot to add that this was a cron)). There is still the time set limit of the server which in my case overrules the time set limit added in php.
What I have done in the time being is add an array slice to see if that will work. I am just uploading the file to my FTP to run the tasks and seeing it that solved it.
The time set limit would not really have worked either as a 16,000 row file would take my server around 1 hour to process (the validation in my cron script is extremely intensive (I forgot to add that this was a cron)). There is still the time set limit of the server which in my case overrules the time set limit added in php.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
next set a for loop delimiter and work with a for loop instead?
regards,