Solved

PHP: fopen and large file

Posted on 2009-07-02
8
358 Views
Last Modified: 2012-05-07
Hi,

I'm trying to open large (ish) text file using fopen and then read it line by line to create mysql queries. The size of the file is 264.8 MB. The script times out every time despite increasing max_execution_time to 1000. Are there any other variables that I should be looking at or is 1000 simply to low?

Thank you in advance!
$handle = fopen($file, "rb");
 
		if ($handle)
		{
			$line = fgets($handle);

Open in new window

0
Comment
Question by:tomcupr
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
  • 2
  • +1
8 Comments
 
LVL 5

Expert Comment

by:mallcore
ID: 24764478
There is a function called
file($file) or
file_get_contents($file)

The first one puts the entire file into array, the other one reads the file and puts it into a string. In my opinion its easier.
0
 

Author Comment

by:tomcupr
ID: 24764541
@mallcore: will it solve my problem though? I am not looking for an easier but working solution and I'm not too sure that 264 MB string or array would work...
0
 
LVL 36

Expert Comment

by:Loganathan Natarajan
ID: 24764650
Increasing 264M with  max_execution_time won't help. you should see in php.ini file and keep it less than 128M and try it out...
0
PeopleSoft Has Never Been Easier

PeopleSoft Adoption Made Smooth & Simple!

On-The-Job Training Is made Intuitive & Easy With WalkMe's On-Screen Guidance Tool.  Claim Your Free WalkMe Account Now

 
LVL 5

Accepted Solution

by:
mallcore earned 500 total points
ID: 24764653
Just noticed you are reading the file binary
ini_set('max_execution_time', '10000');
0
 
LVL 36

Expert Comment

by:Loganathan Natarajan
ID: 24764657
also check, set_time_limit from php.ini and work on.
0
 
LVL 36

Expert Comment

by:Loganathan Natarajan
ID: 24764673
consider to read chunk and open it.,

ref. http://php.net/manual/en/function.readfile.php
0
 
LVL 34

Expert Comment

by:Beverley Portlock
ID: 24764856
What is in the file? Is it CSV data or SQL queries or something else?

0
 

Author Comment

by:tomcupr
ID: 24764862
@bportlock it's tab delimited data
0

Featured Post

Secure Your WordPress Site: 5 Essential Approaches

WordPress is the web's most popular CMS, but its dominance also makes it a target for attackers. Our eBook will show you how to:

Prevent costly exploits of core and plugin vulnerabilities
Repel automated attacks
Lock down your dashboard, secure your code, and protect your users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Generating table dynamically is the most common issue faced by php developers.... So it seems there is a need of an article that explains the basic concept of generating tables dynamically. It just requires a basic knowledge of html and little maths…
Since pre-biblical times, humans have sought ways to keep secrets, and share the secrets selectively.  This article explores the ways PHP can be used to hide and encrypt information.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to count occurrences of each item in an array.

691 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question