I have a 212MB text file containing about 600,000 records I am trying to import into mySQL using php. There is a lot of manipulation that must be done so I can not do a straight import into mySQL. The import works great and handles about 8,000 recrods per minute. My problem is I am running the php script by calling it thorough a web browser. My issues are:
1) When it gets to about the 93,000 record, it stops. I have set the set_time_limit property to 0 so I'm not sure why this is happening
2) How can I monitor the progress of the script? I am currently sending output to the webbrowser with the count of the record number but I'm thinking there has to be a better way. Is there a way to run/monitor the script locally on the server? If so, can you pause the execution somehow?