csv file too large

I am entering info from a csv file into a database. this has been working and still works on most files but I believe a file is too large now to work. the program is crashing on this line.

$file = file($filename, FILE_IGNORE_NEW_LINES );

any help would be appreciated.
LVL 6
mmarthAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Nathan RileyFounderCommented:
What's you post_max_size limit set to in your php settings?  Chances are if default then it's 2mb, you'd need to up that to bring in the larger files and that should resolve the issue.
mmarthAuthor Commented:
post_max_size  8M

also the file has already been uploaded to the same folder the program is in prior to running.
Ray PaseurCommented:
The program is crashing...
What does that mean?  Is there some symptom you can share with us?  Is there a link to your test data so we can try it on  our own?
OWASP: Threats Fundamentals

Learn the top ten threats that are present in modern web-application development and how to protect your business from them.

mmarthAuthor Commented:
Several things. I'm sorry, it just means that the program stops when it encounters this line. Also,  the file is 57 MB. if i cut it down it will run.
$file = file($filename, FILE_IGNORE_NEW_LINES );

 I displayed errors and the result is:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 285 bytes) in /var/www/clients/client9/web14/web/Medicare/uploads/fileToDB_SAM.php on line 21
Ray PaseurCommented:
Ahh, now we are getting somewhere.  PHP arrays use about 20X the storage space needed for the raw data.  So 20X 57MB is your baseline storage demand: 1.2 Gigabytes.  That explains why it's dying after 128 Megabytes!  My recommendation would be to refactor the application so that it can read the file one line at a time.  When you use file() you're telling PHP to read the entire file into an array.  If you use fgetcsv() you can read each row and process the rows incrementally.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
mmarthAuthor Commented:
Thanks Ra.
Ray PaseurCommented:
Thanks for the points -- it's a great question!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
PHP

From novice to tech pro — start learning today.