PHP 5.3.5 memory leak
After solving a problem with a php memory leak in question https://www.experts-exchange.com/questions/26929549/PHP-cron-script-in-production-for-5-years-suddenly-develops-memory-leak.html?anchorAnswerId=35306748#a35306748, I am coming across other memory leaks that are not solved by the same workaround.
I just ran the following code:
while($r = mysqli_fetch_assoc($rs))
{
echo 'userid=', $r['userid'], "\n";
echo ' mem=' . memory_get_usage(false), "\n";
}
The SQL executed immediately prior to the above is "SELECT * FROM user ORDER BY userid LIMIT 10". This program is in test mode, and I would not normal use the asterisk.
In any case, here is the output:
php ./sanitize-data.php
Sanitize user
mem=1106704
userid=1
mem=1133296
userid=2
mem=1136496
userid=3
mem=1139728
userid=5
mem=1142960
userid=6
mem=1146208
userid=7
mem=1149440
userid=8
mem=1152672
userid=10
mem=1155904
userid=11
mem=1159136
userid=12
mem=1162368
As you can see, there is an obvious memory leak as there is no obvious reason why memory usage show grow from the operations in this simple loop.
Previously I was able to work around this issue by unsetting the variable at the end of the loop, like this:
while($r = mysqli_fetch_assoc($rs))
{
echo 'userid=', $r['userid'], "\n";
echo ' 0 mem=' . memory_get_usage(false), "\n";
unset($r);
}
The output now changes to the following:
php ./sanitize-data.php
Sanitize user
mem=1106816
userid=1
mem=1133408
userid=2
mem=1136608
userid=3
mem=1139808
userid=5
mem=1143008
userid=6
mem=1146208
userid=7
mem=1149408
userid=8
mem=1152656
userid=10
mem=1155856
userid=11
mem=1159056
userid=12
mem=1162256
This makes no sense to me, as it appears that even unsetting the variable is having no effect. Previously this workaround prevented the memory leak.
Any ideas?
I just ran the following code:
while($r = mysqli_fetch_assoc($rs))
{
echo 'userid=', $r['userid'], "\n";
echo ' mem=' . memory_get_usage(false), "\n";
}
The SQL executed immediately prior to the above is "SELECT * FROM user ORDER BY userid LIMIT 10". This program is in test mode, and I would not normal use the asterisk.
In any case, here is the output:
php ./sanitize-data.php
Sanitize user
mem=1106704
userid=1
mem=1133296
userid=2
mem=1136496
userid=3
mem=1139728
userid=5
mem=1142960
userid=6
mem=1146208
userid=7
mem=1149440
userid=8
mem=1152672
userid=10
mem=1155904
userid=11
mem=1159136
userid=12
mem=1162368
As you can see, there is an obvious memory leak as there is no obvious reason why memory usage show grow from the operations in this simple loop.
Previously I was able to work around this issue by unsetting the variable at the end of the loop, like this:
while($r = mysqli_fetch_assoc($rs))
{
echo 'userid=', $r['userid'], "\n";
echo ' 0 mem=' . memory_get_usage(false), "\n";
unset($r);
}
The output now changes to the following:
php ./sanitize-data.php
Sanitize user
mem=1106816
userid=1
mem=1133408
userid=2
mem=1136608
userid=3
mem=1139808
userid=5
mem=1143008
userid=6
mem=1146208
userid=7
mem=1149408
userid=8
mem=1152656
userid=10
mem=1155856
userid=11
mem=1159056
userid=12
mem=1162256
This makes no sense to me, as it appears that even unsetting the variable is having no effect. Previously this workaround prevented the memory leak.
Any ideas?
Aaron Tomosky
I dot know if it's worth messing with but it could be a problem in the mysql connector function not to base php. I use adodb as a simple abstraction layer ( I don't let it build querries for me) and havnt had any memory leaks with it.
ASKER
How would I go about dealing with that?
ASKER
Actually, the full script has about 10 such loops. For the first nine, no memory is leaked. Only on the last one is there leakage.
Remember PHP is an interpreter and thus you have no real controll over the used memory.
It is very complicated to see why php does not free resources and has a special algorithm in php 5.3 to reduce the calls to the OS memory allocation functions and thus more more memory is used. This is a tradoff between memory and CPU costs and the decicion was to use a little bit more memory to speed up php.
See http://www.informit.com/articles/article.aspx?p=516587 for a explanation how the memory allocation works in php and why you don't get the expected results
It is very complicated to see why php does not free resources and has a special algorithm in php 5.3 to reduce the calls to the OS memory allocation functions and thus more more memory is used. This is a tradoff between memory and CPU costs and the decicion was to use a little bit more memory to speed up php.
See http://www.informit.com/articles/article.aspx?p=516587 for a explanation how the memory allocation works in php and why you don't get the expected results
If the script has 10 of these kind of loops, you would have to show us the whole as the variables may interrupt each other... or such thing.
ASKER
hernst42: an interesting article. However in my case there are no issues with bailouts, as you will see when I post the code.
Roads_Roads: regarding showing you the script: this is actually feasible in this case, as it is a simple script that is used to sanitize a database. There is a simple loop for each table that is being sanitized. I will create a suitably instrumented version and post the results.
Ray_Paseur: I am assuming that you are suggesting upgrading to 5.3.6. That may well be the real solution. However, this is frustrating, as we just did the 5.3.5 upgrade two weeks ago. This compares to previous intervals of several years between upgrades. Too bad for us!
Roads_Roads: regarding showing you the script: this is actually feasible in this case, as it is a simple script that is used to sanitize a database. There is a simple loop for each table that is being sanitized. I will create a suitably instrumented version and post the results.
Ray_Paseur: I am assuming that you are suggesting upgrading to 5.3.6. That may well be the real solution. However, this is frustrating, as we just did the 5.3.5 upgrade two weeks ago. This compares to previous intervals of several years between upgrades. Too bad for us!
Yeah, the upgrade might or might not be of any value. But I am curious - is this situation causing anything to blow up, or is it just a matter of trying to figure out an anomalous condition?
ASKER
I am zeroing in on where the problem comes from by working with a test version of the script. The code snippet shows the relevant part of the script.
You will note that there are 7 loops. The first six do not cause trouble. In the seventh loop ("Sanitize user"), you will note a commented out SELECT that selects a single column, along with the non-commented out version that does a SELECT *.
Here is the output from the version that does SELECT userid:
Here is the output from the version that does SELECT *
Observations
1. You will notice each loop does no work. I have replaced the actual work with the memory display, using a mod to reduce the output.
2. You will notice that each loop ends with $r = null. If this is not done then every loop shows a "memory leak".
3. All of the loops are prevented from leaking memory via the $r = null, except for the one with the SELECT *. Even though the $r array is set to null, something is still going on to leak memory.
My suspicion is that there is a php bug related to the way the mysql connector is transferring the data, and that perhaps there is an intermediate storage location that is not getting cleared.
At present I work around this issue by having the script not process the whole table. So I get the job done.
In another possible workaround, rather than SELECT * only the needed columns could be specified. But because this is most of the values anyway, that may or may not change the result. But worth a try, so I will report back on that.
$cnt = 0;
echo "Sanitize contactrequestcomment\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT contactrequestid FROM contactrequestcomment");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt % 5000 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize contactrequestmessage\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT contactrequestid FROM contactrequestmessage");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt %1000 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize admin-comment\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT id, userid FROM admin_comment");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt % 100 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize principal\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT userid FROM principal");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt % 1500 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize rep\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT userid FROM rep");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt % 1500 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize user\n";
echo 'mem=' . memory_get_usage(false), "\n";
//$rs = SQL("SELECT userid FROM user"); // sort to aid in debugging memory issue
$rs = SQL("SELECT * FROM user"); // sort to aid in debugging memory issue
while($r = mysqli_fetch_assoc($rs))
{
if (++$cnt % 3000 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}You will note that there are 7 loops. The first six do not cause trouble. In the seventh loop ("Sanitize user"), you will note a commented out SELECT that selects a single column, along with the non-commented out version that does a SELECT *.
Here is the output from the version that does SELECT userid:
$php ./test-sanitize.php
Sanitize contactrequestcomment
mem=1075480
id=179232 mem=2621440
id=218153 mem=3145728
id=256203 mem=3407872
id=293601 mem=3932160
id=332747 mem=4194304
id=372590 mem=4718592
id=417114 mem=4980736
id=455755 mem=5505024
id=488527 mem=6029312
Sanitize contactrequestmessage
mem=5474432
id=209474 mem=2359296
id=238713 mem=2359296
id=265884 mem=2359296
id=288243 mem=2359296
id=311222 mem=2359296
id=343457 mem=2359296
id=375255 mem=2359296
id=406325 mem=2359296
id=431230 mem=2359296
id=457217 mem=2359296
Sanitize admin-comment
mem=2022752
Sanitize principal
mem=1079312
id=6069 mem=2359296
id=10261 mem=2359296
id=14060 mem=2359296
id=18177 mem=2359296
id=22800 mem=2359296
id=26216 mem=2359296
id=29439 mem=2359296
id=32434 mem=2621440
id=35382 mem=2621440
id=38078 mem=2883584
id=40756 mem=2883584
id=43080 mem=2883584
id=45592 mem=3145728
Sanitize rep
mem=2953904
id=1440 mem=2883584
id=3123 mem=2883584
id=5590 mem=2883584
id=8103 mem=2883584
id=10425 mem=2883584
id=12949 mem=2883584
id=15424 mem=2883584
id=17702 mem=2883584
id=20091 mem=2883584
id=22027 mem=2883584
id=24725 mem=3145728
id=27426 mem=3145728
id=30285 mem=3407872
id=33546 mem=3407872
id=36470 mem=3407872
id=39876 mem=3670016
id=43987 mem=3670016
Sanitize user
mem=3597368
id= mem=3932160
id= mem=3932160
id= mem=3932160
id= mem=3932160
id= mem=3932160
id= mem=3932160
id= mem=3932160
id= mem=3932160
id= mem=4194304
id= mem=4456448
id= mem=4718592
id= mem=4980736
id= mem=5242880
id= mem=5505024
id= mem=5767168
id= mem=6029312
$Here is the output from the version that does SELECT *
$ php ./test-sanitize.php
Sanitize contactrequestcomment
mem=1075512
id=179232 mem=2621440
id=218153 mem=3145728
id=256203 mem=3407872
id=293601 mem=3932160
id=332747 mem=4194304
id=372590 mem=4718592
id=417114 mem=4980736
id=455755 mem=5505024
id=488527 mem=6029312
Sanitize contactrequestmessage
mem=5474464
id=209474 mem=2359296
id=238713 mem=2359296
id=265884 mem=2359296
id=288243 mem=2359296
id=311222 mem=2359296
id=343457 mem=2359296
id=375255 mem=2359296
id=406325 mem=2359296
id=431230 mem=2359296
id=457217 mem=2359296
Sanitize admin-comment
mem=2022784
Sanitize principal
mem=1079344
id=6069 mem=2359296
id=10261 mem=2359296
id=14060 mem=2359296
id=18177 mem=2359296
id=22800 mem=2359296
id=26216 mem=2359296
id=29439 mem=2359296
id=32434 mem=2621440
id=35382 mem=2621440
id=38078 mem=2883584
id=40756 mem=2883584
id=43080 mem=2883584
id=45592 mem=3145728
Sanitize rep
mem=2953992
id=1440 mem=2883584
id=3123 mem=2883584
id=5590 mem=2883584
id=8103 mem=2883584
id=10425 mem=2883584
id=12949 mem=2883584
id=15424 mem=2883584
id=17702 mem=2883584
id=20091 mem=2883584
id=22027 mem=2883584
id=24725 mem=3145728
id=27426 mem=3145728
id=30285 mem=3407872
id=33546 mem=3407872
id=36470 mem=3407872
id=39876 mem=3670016
id=43987 mem=3670016
Sanitize user
mem=3597376
id= mem=18612224
id= mem=28049408
id= mem=37748736
id= mem=47185920
id= mem=56885248
id= mem=66584576
id= mem=76021760
id= mem=85721088
id= mem=95420416
id= mem=104857600
id= mem=114556928
id= mem=123994112
id= mem=133693440
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 32 bytes) in /var/www/rephunter/dev/cron/test-sanitize.php on line 78
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 32 bytes) in /var/www/rephunter/dev/cron/test-sanitize.php on line 78
Bus error: 10 (core dumped)
$Observations
1. You will notice each loop does no work. I have replaced the actual work with the memory display, using a mod to reduce the output.
2. You will notice that each loop ends with $r = null. If this is not done then every loop shows a "memory leak".
3. All of the loops are prevented from leaking memory via the $r = null, except for the one with the SELECT *. Even though the $r array is set to null, something is still going on to leak memory.
My suspicion is that there is a php bug related to the way the mysql connector is transferring the data, and that perhaps there is an intermediate storage location that is not getting cleared.
At present I work around this issue by having the script not process the whole table. So I get the job done.
In another possible workaround, rather than SELECT * only the needed columns could be specified. But because this is most of the values anyway, that may or may not change the result. But worth a try, so I will report back on that.
ASKER
Ray_Paseur: it is a maintenance script, which as I indicated can be tweaked to do the user table in two passes. So there is a workaround to get the job done.
But in fact in other mission-critical production code, I have had to apply the same workaround of clearing the array before reusing it in order to keep the train running. Fortunately in that case, the workaround is effective.
But in fact in other mission-critical production code, I have had to apply the same workaround of clearing the array before reusing it in order to keep the train running. Fortunately in that case, the workaround is effective.
ASKER
Aha!
By avoiding the splat, it appears that memory is not leaked nearly as much. Here is the portion of the output with specified fields in the SELECT:
Sanitize user
mem=3598344
userid=634 mem=10223616
userid=3697 mem=13893632
userid=6693 mem=18087936
userid=9670 mem=22544384
userid=12662 mem=26738688
userid=15645 mem=31195136
userid=18616 mem=35389440
userid=21605 mem=39845888
userid=24643 mem=44040192
userid=27715 mem=48496640
userid=30717 mem=52690944
userid=33757 mem=57147392
userid=36735 mem=61341696
userid=39741 mem=65798144
userid=42756 mem=69992448
userid=45766 mem=74448896
mem=73958976
So while there still does appear to be a small memory leak, it does not blow up as when * is used. The memory readings are every 3000 rows.
Looking at the schema of the table to see the difference between the specified columns as compared to "all columns", the ones omitted are mostly timestamps and various integer types. Specifically, there are no other varchars or text fields.
This result shows another reason to avoid SELECT * (which should never be used in production code): there is "much larger than normal memory leak" in PHP associated with its use.
I am going to consider the issue resolved as far as I am concerned. However, I am wondering if it might not be worth opening a php bug report? Of course, I would only do this after confirming that the issue still occurs on 5.3.6, so it is somewhat academic at this time, as we just upgraded to 5.3.5 two weeks ago.
By avoiding the splat, it appears that memory is not leaked nearly as much. Here is the portion of the output with specified fields in the SELECT:
Sanitize user
mem=3598344
userid=634 mem=10223616
userid=3697 mem=13893632
userid=6693 mem=18087936
userid=9670 mem=22544384
userid=12662 mem=26738688
userid=15645 mem=31195136
userid=18616 mem=35389440
userid=21605 mem=39845888
userid=24643 mem=44040192
userid=27715 mem=48496640
userid=30717 mem=52690944
userid=33757 mem=57147392
userid=36735 mem=61341696
userid=39741 mem=65798144
userid=42756 mem=69992448
userid=45766 mem=74448896
mem=73958976
So while there still does appear to be a small memory leak, it does not blow up as when * is used. The memory readings are every 3000 rows.
Looking at the schema of the table to see the difference between the specified columns as compared to "all columns", the ones omitted are mostly timestamps and various integer types. Specifically, there are no other varchars or text fields.
This result shows another reason to avoid SELECT * (which should never be used in production code): there is "much larger than normal memory leak" in PHP associated with its use.
I am going to consider the issue resolved as far as I am concerned. However, I am wondering if it might not be worth opening a php bug report? Of course, I would only do this after confirming that the issue still occurs on 5.3.6, so it is somewhat academic at this time, as we just upgraded to 5.3.5 two weeks ago.
Amazing. As if we needed another reason not to use SELECT *!
If you have a test data set that you want to give me, I have 5.3.6 and would be glad to test on that version number. My guess - it will still happen. The 5.3.6 upgrade notice did not say anything about this, so I would expect that it is no different.
If you have a test data set that you want to give me, I have 5.3.6 and would be glad to test on that version number. My guess - it will still happen. The 5.3.6 upgrade notice did not say anything about this, so I would expect that it is no different.
ASKER
I can give you a mysqldump of the data, and the test script. Would that work?
Sure! If you want to email it to me use Ray.Paseur at Gmail.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I will need to package up a standalone version of the script, along with the database, which is already sanitized to send to Ray_Paseur.
When I do that it will be easy to see what happens if I pop in mysql_fetch_assoc in place of mysqli_fetch_assoc.
When I do that it will be easy to see what happens if I pop in mysql_fetch_assoc in place of mysqli_fetch_assoc.
ASKER
Ray_Paseur:
It took me a few days to get back to this. During the packaging up the standalone version of the script, I discovered and interesting discrepancy.
I wanted to make a standalone script that does not depend upon any of the standard includes for simplicity. When I did this, I ran from my local development environment, which is php 5.2.4, and no memory leak occurred. This was odd, because during my previous testing, there was a memory leak in that environment.
I was wondering if the simplification might have removed the issue. So next I ran the test script in our server development environment, which is php 5.3.5. This time the memory leak was still there.
Here is an extract of the output from php 5.2.4:
Sanitize user
mem=90192
userid=590 mem=262144
userid=3653 mem=262144
userid=6652 mem=262144
userid=9628 mem=262144
userid=12619 mem=262144
userid=15602 mem=262144
userid=18573 mem=262144
userid=22057 mem=262144
userid=24599 mem=262144
userid=27672 mem=262144
userid=30676 mem=262144
userid=33714 mem=262144
userid=36693 mem=262144
userid=39699 mem=262144
userid=42712 mem=262144
userid=45721 mem=262144
mem=91984
As you can see, memory does not change.
Here is the same extract of the same script using the same database, but in php 5.3.5:
Sanitize user
mem=3190360
userid=590 mem=9961472
userid=3653 mem=13369344
userid=6652 mem=17563648
userid=9628 mem=22020096
userid=12619 mem=26214400
userid=15602 mem=30670848
userid=18573 mem=34865152
userid=22057 mem=39321600
userid=24599 mem=43515904
userid=27672 mem=47972352
userid=30676 mem=52166656
userid=33714 mem=56623104
userid=36693 mem=60817408
userid=39699 mem=65273856
userid=42712 mem=69468160
userid=45721 mem=73924608
mem=73560488
[jas1@dev /var/www/rephunter/dev/web
Here is the test script:
It took me a few days to get back to this. During the packaging up the standalone version of the script, I discovered and interesting discrepancy.
I wanted to make a standalone script that does not depend upon any of the standard includes for simplicity. When I did this, I ran from my local development environment, which is php 5.2.4, and no memory leak occurred. This was odd, because during my previous testing, there was a memory leak in that environment.
I was wondering if the simplification might have removed the issue. So next I ran the test script in our server development environment, which is php 5.3.5. This time the memory leak was still there.
Here is an extract of the output from php 5.2.4:
Sanitize user
mem=90192
userid=590 mem=262144
userid=3653 mem=262144
userid=6652 mem=262144
userid=9628 mem=262144
userid=12619 mem=262144
userid=15602 mem=262144
userid=18573 mem=262144
userid=22057 mem=262144
userid=24599 mem=262144
userid=27672 mem=262144
userid=30676 mem=262144
userid=33714 mem=262144
userid=36693 mem=262144
userid=39699 mem=262144
userid=42712 mem=262144
userid=45721 mem=262144
mem=91984
As you can see, memory does not change.
Here is the same extract of the same script using the same database, but in php 5.3.5:
Sanitize user
mem=3190360
userid=590 mem=9961472
userid=3653 mem=13369344
userid=6652 mem=17563648
userid=9628 mem=22020096
userid=12619 mem=26214400
userid=15602 mem=30670848
userid=18573 mem=34865152
userid=22057 mem=39321600
userid=24599 mem=43515904
userid=27672 mem=47972352
userid=30676 mem=52166656
userid=33714 mem=56623104
userid=36693 mem=60817408
userid=39699 mem=65273856
userid=42712 mem=69468160
userid=45721 mem=73924608
mem=73560488
[jas1@dev /var/www/rephunter/dev/web
Here is the test script:
<?php
/**
* Title: Test Sanitize
* Author: JAS
* Date: 11-May-10
* Project: RepHunter
* Purpose: Testing memory leak
*
*/
$DBHOSTNAME = 'dev-mysql.rephunter.net';
$DBUSER = 'phpuser';
$DBPASSWORD = 'sanitize';
$DBNAME = 'sanitize';
// Connect To DB
$link = @mysqli_connect($DBHOSTNAME, $DBUSER, $DBPASSWORD, $DBNAME);
if(!$link)
{
trigger_error('Cannot connect to mysql: '.mysqli_connect_error($link),E_USER_ERROR);
}
set_time_limit(900);
$cnt = 0;
echo "Sanitize contactrequestcomment\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT contactrequestid FROM contactrequestcomment");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt % 5000 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize contactrequestmessage\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT contactrequestid FROM contactrequestmessage");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt %1000 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize admin-comment\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT id, userid FROM admin_comment");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt % 100 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize principal\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT userid FROM principal");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt % 1500 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize rep\n";
echo 'mem=' . memory_get_usage(false), "\n";
$rs = SQL("SELECT userid FROM rep");
while($r = mysqli_fetch_array($rs))
{
if (++$cnt % 1500 == 0)
echo ' id=', $r[0], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo "Sanitize user\n";
echo 'mem=' . memory_get_usage(false), "\n";
//$rs = SQL("SELECT userid FROM user");
// $rs = SQL("SELECT * FROM user");
$rs = SQL("SELECT userid, password, original_ip, email1, email2, companyname, positiontitle, fname, lname, address1, address2, city, postal, phone1, phone2, fax, cell, website FROM user");
while($r = mysqli_fetch_assoc($rs))
{
if (++$cnt % 3000 == 0)
echo ' userid=', $r['userid'], ' mem=' . memory_get_usage(true), "\n";
$r = null;
}
echo 'mem=' . memory_get_usage(false), "\n";
function SQL($query)
{
global $link;
$rs = mysqli_query($link, $query);
if ($rs)
{
return $rs;
}
$error_msg = "** SQL Error #$errno ** Description: $error SQL Text: $query";
trigger_error($error_msg,E_USER_ERROR);
return false;
}
?>ASKER
Ray_Paseur: I can email or otherwise deliver the database, but it is 37MB in bzip2.
ASKER
bportlock: I also converted the script to mysql_ functions instead of mysqli_ functions. That also does not have a leak. Here is the output extract:
Sanitize user
mem=659680
userid=590 mem=786432
userid=3653 mem=786432
userid=6652 mem=786432
userid=9628 mem=786432
userid=12619 mem=786432
userid=15602 mem=786432
userid=18573 mem=786432
userid=22057 mem=786432
userid=24599 mem=786432
userid=27672 mem=786432
userid=30676 mem=786432
userid=33714 mem=786432
userid=36693 mem=786432
userid=39699 mem=786432
userid=42712 mem=786432
userid=45721 mem=786432
mem=659680
My conclusion is that is actually a memory leak in 5.3.5 mysqli_ functionality, as the code is otherwise identical.
Sanitize user
mem=659680
userid=590 mem=786432
userid=3653 mem=786432
userid=6652 mem=786432
userid=9628 mem=786432
userid=12619 mem=786432
userid=15602 mem=786432
userid=18573 mem=786432
userid=22057 mem=786432
userid=24599 mem=786432
userid=27672 mem=786432
userid=30676 mem=786432
userid=33714 mem=786432
userid=36693 mem=786432
userid=39699 mem=786432
userid=42712 mem=786432
userid=45721 mem=786432
mem=659680
My conclusion is that is actually a memory leak in 5.3.5 mysqli_ functionality, as the code is otherwise identical.
If you still want me to try this, I'll be glad to. I think your conclusion relative to mysqlI is probably correct.
ASKER
So my next step is to consider whether I should report it as a PHP bug.
My experience in the past of reporting bugs to major products is that I search the bug tracker, usually cannot find the bug, open a ticket, only to have it immediately closed as a duplicate.
My experience in the past of reporting bugs to major products is that I search the bug tracker, usually cannot find the bug, open a ticket, only to have it immediately closed as a duplicate.
"My experience in the past of reporting bugs to major products is that I search the bug tracker, usually cannot find the bug, open a ticket, only to have it immediately closed as a duplicate."
Submit it anyway. They might not be aware of it and if they are it will remind them that people are having problems with this.
Submit it anyway. They might not be aware of it and if they are it will remind them that people are having problems with this.
ASKER
You are probably right. So I started to open the bug report. Unfortunately we are on 5.3.5 (just upgraded to that), and it is probably not right to report it unless we first went to 5.3.6.
I don't see why you need to worry about 5.3.6 because you are not on that version.
ASKER
I am not worried about it. It is just that they don't like or even accept bug reports for other than the current version.
"It is just that they don't like or even accept bug reports for other than the current version."
I learn something new every day.
What's your way forward from here then? Revert to mysql_ from mysqli_ or downgrade to an earlier version of PHP such as 5.3.2?
I learn something new every day.
What's your way forward from here then? Revert to mysql_ from mysqli_ or downgrade to an earlier version of PHP such as 5.3.2?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Best solution is the two parts of the workaround I gave in my final post. However, the suggestion to see what msyql_fetch_assoc did shed some light on where the problem might be (a bug the PHP connector), although is not applicable to our case. Rather the two workarounds are the best solution.