PHP / MySQL - Processing a 9MB-25MB .csv file
Posted on 2011-02-11
1. PHP / MySQL website on dedicated server.
2. Customer uploads a large file (9MB - 25MB)
3. This file is a .csv of product_id,cust_id,product_price
4. Whats the fastest and best way to efficiently import this data into our data table.
We have successfully created scripting that does it pretty fast, but we're having trouble with the larger files.
We have been able to get it to work by putting this at the beginning of our processing PHP script: ini_set("memory_limit","375M");
...but that seems like a bit of a hack, right?
There must be a more efficient wat, or is this ok since we're on a dedicated server?
Should we just set the memory_limit back to something like 64MB when we're done processing, or is it ok to leave it at 375MB?