Fatal error: Out of memory

I have a problem, i am using a php script file to generate something and put in my mysql db, the problem is that the info is very much and i get something likt this :

Fatal error: Out of memory (allocated 503054336) at /usr/ports/lang/php5/work/php-5.2.3/Zend/zend_hash.c:610 (tried to allocate 1048576 bytes) in /usr/home/gtoplive/public_html/php/inc/__mysql.php on line 375


How can i compress or do something to reduce the memory allocation ?
rares_dumitrescuAsked:
Who is Participating?
 
etullyConnect With a Mentor Commented:
9 times out of 10, you don't need to read all the records from the database, process them, and write them back out.

The remaining 1 time out of 10 is when record #1 is altered in a way that is dependent on the values of records #2-250,000.

A good programming model would be:

Set $i = 0;
Start Loop
read records $i through $i+X  (where X is a number from 0 to maybe 1000)
process those records
write those records
increment $i by X
Finish Loop

If you pick a value of 0 for X,  then you are saying:  Read ONE record, process it, write it.  Repeat.  This is very disk intensive.

If you pick a value of 1000 for X,  then you are saying:  Read 1000 records, process them write them.  Repeat.  This is less disk intensive BUT you are moving lots of data into RAM and you might run out of memory.  Also, things process more slowly when you push RAM to its limits.

You will need to experiment a little to find a good value for X.  I suspect it will be somewhere between 50 and 500.  You might even do it scientifically by trying a dozen different values for X and timing them all.  This way,  you should be able to optimize the code fairly well.
0
 
itsbthCommented:
#1 Use unset() on all the variables you no longer need.
#2 Use  ini_set('memory_limit', '-1'); to disable the memory limit.
#3 Write to a file rather than store it in a variable.
0
 
etullyCommented:
I would need a little more specific information about what you're trying to do but in general, the approach is going to probably be to break up the job and do one piece at a time.  For example, if you are reading 1000 records into memory, altering the values, and writing them back to the database, then instead of reading all 1000 into an array you would instead read one record, alter it, and write it back to the database - and then loop to do that 1000 times.

Can you give me a little more detail about how the logic of your script works and what kind of data you're reading into PHP?
0
 
rares_dumitrescuAuthor Commented:
The scripts works like this...

I get from db about 250 000 records, and then i  combine the information from it in a array, afther that i read the array and insert into another table...
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.