Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Windows 7 > PHP 5.3.6 > Cannot set memory limit

Posted on 2011-09-27
16
Medium Priority
?
536 Views
Last Modified: 2012-05-12
Tried setting the memory limit everywhere but no matter where I set it, it doesn't "take."  What am I missing?

C:\Users\Public>php -f md553_to_csv.php -d memory_limit=1512927291

Fatal error: Out of memory (allocated 524288) 
(tried to allocate 1512927290 bytes) in 
C:\Users\Public\md553_to_csv.php on line 3

Open in new window


Here's the PHP code:
<?php
ini_set('memory_limit', '1512927291');
$lines = file('C:\Users\Public\CRMD3836.mask', FILE_IGNORE_NEW_LINES);
?>

Open in new window

0
Comment
Question by:Geoff Millikan
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 10
  • 5
16 Comments
 
LVL 17

Expert Comment

by:psimation
ID: 36711577
try to edit the php.ini file, and you will probably need to run your text editor as Administrator.
0
 

Author Comment

by:Geoff Millikan
ID: 36713551
Further debugging shows the memory_limit is in fact getting changed - the issue has something to do with how big the file is (about 1.5GB) and the function, file().  The error seems to point to memory_limit bing the issue, but it's not.  

So not sure what the issue is, this is very strange.
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 36713910
A file of 1.5GB is kind of large for in-memory storage, even in this day and age.  Can you give us a URL link to the file and a sense of what you want to get out of the file, please?
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 

Author Comment

by:Geoff Millikan
ID: 36714325
Cannot give link but can tell you it's a big ASCII text file that we're parsing content out of, line-by-line.

In the mean time, we're using fopen(), fgets() and that works fine but still curious why file() doesn't work.
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 36716689
OK, please create a big ASCII text file that looks like what you want to work on.  Put it on a server where you can give the link.  To understand why we need to see the test data, please see the article here about how professional programmers approach problems with test-driven development.
http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/A_7830-A-Quick-Tour-of-Test-Driven-Development.html

Fopen() and fgets() read one line at a time, and this process uses relatively little memory.  File() reads the entire file into an array, thus ensuring that the largest possible representation of the file is loaded all at once.
0
 

Author Comment

by:Geoff Millikan
ID: 36907950
This works:  Run this at Windows command line:
php -r "while (filesize('C:\Users\Public\I_AM_A_2GB_FILE.txt')<=2147483648) {file_put_contents('C:\Users\Public\I_AM_A_2GB_FILE.txt',substr(str_shuffle(str_repeat('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 ',5000)),0,262144), FILE_APPEND);}"

Open in new window


Then run this:
C:\Users\Public>php -r "echo strlen(file_get_contents('C:\Users\Public\I_AM_A_2GB_FILE.txt'));" -d memory_limit=2500000000

Open in new window


And you get this error:
Fatal error: Out of memory (allocated 524288) (tried to allocate 2147483648 bytes) in Command line code on line 1

Merry Christmas!
0
 

Author Comment

by:Geoff Millikan
ID: 36907981
Actually, better to use this to make a 1GB file so we don't run into the 32bit integer issue described at  link below:
C:\Users\Public>php -r "while (filesize('C:\Users\Public\I_AM_A_1GB_FILE.txt')<=1073741824) {file_put_contents('C:\Users\Public\I_AM_A_1GB_FILE.txt',substr(str_shuffle(str_repeat('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 ',5000)),0,262144), FILE_APPEND);}"

Open in new window



http://php.net/manual/en/function.filesize.php
Note:  Because PHP's integer type is signed and many platforms use 32bit integers, filesize() may return unexpected results for files which are larger than 2GB. For files between 2GB and 4GB in size this can usually be overcome by using sprintf("%u", filesize($file)).
0
 

Author Comment

by:Geoff Millikan
ID: 36908009
Hold the phone, it's working now for 1GB file...
0
 

Author Comment

by:Geoff Millikan
ID: 36908029
file_get_contents() works for me on files up to about 1,182,531,584 bytes but if the files go over 1,606,156,288 bytes it fails consistently:

Fatal error: Out of memory (allocated 524288) (tried to allocate 1606164480 bytes) in Command line code on line 1

And just to confirm, my system has that much RAM available:
 System RAM
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 36909126
Where is the link to the test file we asked for?

PHP has limits on the amount of memory it uses.  These may not be directly related to the amount of memory on the system -- it just depends on the PHP settings.
0
 

Author Comment

by:Geoff Millikan
ID: 36913350
Ray_Paseur
Where is the link to the test file we asked for?

Cut and paste the script below and you will have your test data:
C:\Users\Public>php -r "while (filesize('C:\Users\Public\I_AM_A_1GB_FILE.txt')<=1073741824) {file_put_contents('C:\Users\Public\I_AM_A_1GB_FILE.txt',substr(str_shuffle(str_repeat('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 ',5000)),0,262144), FILE_APPEND);}"

Open in new window


Follow instructions above to reproduce.  I don't believe this is a memory_limit ini file issue, much more complicated.  
0
 
LVL 111

Assisted Solution

by:Ray Paseur
Ray Paseur earned 2000 total points
ID: 36913460
Using either file_get_contents() or file() I get this:
Fatal error: Allowed memory size of 134,217,728 bytes exhausted (tried to allocate 1,406,935,040 bytes) in /home/websitet/public_html/RAY_temp_t1shopper.php on line 3

But when I change to the script below, allowing unlimited memory usage, I get this (commas inserted in output):
http://www.laprbass.com/RAY_temp_t1shopper.php
Outputs
int(2,815,164,416)
<?php // RAY_temp_t1shopper.php
error_reporting(E_ALL);
ini_set('memory_limit', -1);
$x = file('I_AM_A_1GB_FILE.txt');
$m = memory_get_peak_usage(TRUE);
var_dump($m);

Open in new window

0
 

Author Comment

by:Geoff Millikan
ID: 36921608
What version of PHP are you running?  I'm running the Windows version of PHP 5.3.6 32bit on Windows 7 64bit.

Running this code:
<?php
$file='C:\Users\Public\I_AM_A_1GB_FILE.txt';
error_reporting(E_ALL);
ini_set('memory_limit', -1);
echo filesize($file), "\r\n";
echo ini_get('memory_limit'),"\r\n"; 
$x = file($file);
$m = memory_get_peak_usage(TRUE);
var_dump($m);
?>

Open in new window


Produces this result:
c:\Users\Public>php -f "c:\Users\Public\memory_limit.php"
1548746752
-1

Fatal error: Out of memory (allocated 524288) (tried to allocate 1548754944 bytes) in C:\Users\Public\memory_limit.php on line 7

Open in new window

0
 
LVL 111

Accepted Solution

by:
Ray Paseur earned 2000 total points
ID: 36922283
I am running PHP on Linux.  Version 5.3.8...
http://www.laprbass.com/phpinfo.php
0
 

Author Comment

by:Geoff Millikan
ID: 36922374
Ah ha!  Yes, this works for me on Linux no problem.  On Windows, it doesn't.  Must be a Windows issue with how RAM is allocated.
0
 

Author Closing Comment

by:Geoff Millikan
ID: 37015366
Bug in Windows version of PHP...
0

Featured Post

Tech or Treat! - Giveaway

Submit an article about your scariest tech experience—and the solution—and you’ll be automatically entered to win one of 4 fantastic tech gadgets.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Build an array called $myWeek which will hold the array elements Today, Yesterday and then builds up the rest of the week by the name of the day going back 1 week.   (CODE) (CODE) Then you just need to pass your date to the function. If i…
This article discusses how to implement server side field validation and display customized error messages to the client.
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
Suggested Courses

618 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question