duncanb7
asked on
can not set php memory limit in .htaccess and ini_set()
Dear Experts,
I am using curl in php to grab one remote webpage site that is more than 36Meg byte
And I get the fatal error message which is "Out of memory(allocated 3407872( tried to
allocate 3147593 byte) in .../data/include/curl.php on line42" line42 is my $htm=curl_exec($url);
I have checked my php system 's max memory limit is 128M byte by php's phpinfo(). Why I will have such error ?
I go to google it about this at http://drupal.org/node/29268
and I changed my .htaccess file with php_value memory_limit 120M, I talked to my hosting domain support, they just replied it is right there is memory limit=128M byte.
So anyone have suggestion to solve the fatal error issue ?
I also use this in php at the top , ini_set("memory_limit", "120M");, but it still fail
I also use phpinfo() and host support confirm it is 128Meg in my php system.
Any variable use related to curl_exec() is just 2 so it should be less than 120Meg usage in my php ($htm, $file_url)
And attached my .htaccess file and my php sytem configuration.
Is it one variable such as $file_url can not store more than 32Meg byte ?
Please advise & Thanks
Duncan
my php script
=====================
<?php
$file_url=my_curl("wwww.example.com","" , 3000, TRUE);
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
// PREPARE THE ARGUMENT STRING IF NEEDED
// $get_string = '';
// foreach ($get_array as $key => $val)
// {
// $get_string
// = $get_string
// . urlencode($key)
// . '='
// . urlencode($val)
// . '&';
// }
// $get_string = rtrim($get_string, '&');
if (!empty($get_string)) $url .= '?' . $get_string;
$curl = curl_init();
// HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
$header[] = "Accept: text/xml,application/xml,a pplication /xhtml+xml ,text/html ;q=0.9,tex t/plain;q= 0.8,image/ png,*/*;q= 0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q =0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK
// SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6' );
curl_setopt( $curl, CURLOPT_HTTPHEADER, $header );
curl_setopt( $curl, CURLOPT_REFERER, 'http://www.google.com' );
curl_setopt( $curl, CURLOPT_ENCODING, 'gzip,deflate' );
curl_setopt( $curl, CURLOPT_AUTOREFERER, TRUE );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE );
// curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE );
curl_setopt( $curl, CURLOPT_TIMEOUT, $timeout );
// RUN THE CURL REQUEST AND GET THE RESULTS
$htm = curl_exec($curl);
// ON FAILURE HANDLE ERROR MESSAGE
//global $track; // declaring global variable
if ($htm === FALSE)
{
if ($error_report)
{
$err = curl_errno($curl);
$inf = curl_getinfo($curl);
// echo "FAILC: $url TIMEOUT=$timeout, CURL_ERRNOC=$err";
var_dump($inf);
}
curl_close($curl);
return FALSE;
}
// ON SUCCESS RETURN XML / HTML STRING
curl_close($curl);
return $htm;
}
?>
I am using curl in php to grab one remote webpage site that is more than 36Meg byte
And I get the fatal error message which is "Out of memory(allocated 3407872( tried to
allocate 3147593 byte) in .../data/include/curl.php on line42" line42 is my $htm=curl_exec($url);
I have checked my php system 's max memory limit is 128M byte by php's phpinfo(). Why I will have such error ?
I go to google it about this at http://drupal.org/node/29268
and I changed my .htaccess file with php_value memory_limit 120M, I talked to my hosting domain support, they just replied it is right there is memory limit=128M byte.
So anyone have suggestion to solve the fatal error issue ?
I also use this in php at the top , ini_set("memory_limit", "120M");, but it still fail
I also use phpinfo() and host support confirm it is 128Meg in my php system.
Any variable use related to curl_exec() is just 2 so it should be less than 120Meg usage in my php ($htm, $file_url)
And attached my .htaccess file and my php sytem configuration.
Is it one variable such as $file_url can not store more than 32Meg byte ?
Please advise & Thanks
Duncan
my php script
=====================
<?php
$file_url=my_curl("wwww.example.com","" , 3000, TRUE);
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
// PREPARE THE ARGUMENT STRING IF NEEDED
// $get_string = '';
// foreach ($get_array as $key => $val)
// {
// $get_string
// = $get_string
// . urlencode($key)
// . '='
// . urlencode($val)
// . '&';
// }
// $get_string = rtrim($get_string, '&');
if (!empty($get_string)) $url .= '?' . $get_string;
$curl = curl_init();
// HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
$header[] = "Accept: text/xml,application/xml,a
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK
// SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6' );
curl_setopt( $curl, CURLOPT_HTTPHEADER, $header );
curl_setopt( $curl, CURLOPT_REFERER, 'http://www.google.com' );
curl_setopt( $curl, CURLOPT_ENCODING, 'gzip,deflate' );
curl_setopt( $curl, CURLOPT_AUTOREFERER, TRUE );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE );
// curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE );
curl_setopt( $curl, CURLOPT_TIMEOUT, $timeout );
// RUN THE CURL REQUEST AND GET THE RESULTS
$htm = curl_exec($curl);
// ON FAILURE HANDLE ERROR MESSAGE
//global $track; // declaring global variable
if ($htm === FALSE)
{
if ($error_report)
{
$err = curl_errno($curl);
$inf = curl_getinfo($curl);
// echo "FAILC: $url TIMEOUT=$timeout, CURL_ERRNOC=$err";
var_dump($inf);
}
curl_close($curl);
return FALSE;
}
// ON SUCCESS RETURN XML / HTML STRING
curl_close($curl);
return $htm;
}
?>
my .htaccess file
====================
# -FrontPage-
<Limit GET POST>
order deny,allow
deny from all
allow from all
</Limit>
<Limit PUT DELETE>
order deny,allow
deny from all
</Limit>
php_value memory_limit 120M
AuthUserFile /home/aab/public_html/_vti_pvt/service.pwd
AuthGroupFile /home/aab/public_html/_vti_pvt/service.grp
AuthName aaa.com
IndexIgnore .htaccess */.??* *~ *# */HEADER* */README* */_vti* Accept and Award Points Accept as Solution
my Php Configuration
===================
PHP Core
Directive => Local Value => Master Value
allow_call_time_pass_reference => On => On
allow_url_fopen => On => On
allow_url_include => Off => Off
always_populate_raw_post_data => Off => Off
arg_separator.input => & => &
arg_separator.output => & => &
asp_tags => Off => Off
auto_append_file => no value => no value
auto_globals_jit => On => On
auto_prepend_file => /usr/lib/php/head.php => /usr/lib/php/head.php
browscap => no value => no value
default_charset => no value => no value
default_mimetype => text/html => text/html
define_syslog_variables => Off => Off
detect_unicode => On => On
disable_classes => no value => no value
disable_functions => symlink,shell_exec,exec,proc_close,proc_open,popen,system,dl,passthru,escapeshellarg,escapeshellcmd,show_source => symlink,shell_exec,exec,proc_close,proc_open,popen,system,dl,passthru,escapeshellarg,escapeshellcmd,show_source
display_errors => STDOUT => STDOUT
display_startup_errors => Off => Off
doc_root => no value => no value
docref_ext => no value => no value
docref_root => no value => no value
enable_dl => Off => Off
error_append_string => no value => no value
error_log => error_log => error_log
error_prepend_string => no value => no value
error_reporting => 6135 => 6135
expose_php => On => On
extension_dir => /usr/local/lib/php/extensions/no-debug-non-zts-20060613 => /usr/local/lib/php/extensions/no-debug-non-zts-20060613
file_uploads => On => On
highlight.bg => <font style="color: #FFFFFF">#FFFFFF</font> => <font style="color: #FFFFFF">#FFFFFF</font>
highlight.comment => <font style="color: #FF8000">#FF8000</font> => <font style="color: #FF8000">#FF8000</font>
highlight.default => <font style="color: #0000BB">#0000BB</font> => <font style="color: #0000BB">#0000BB</font>
highlight.html => <font style="color: #000000">#000000</font> => <font style="color: #000000">#000000</font>
highlight.keyword => <font style="color: #007700">#007700</font> => <font style="color: #007700">#007700</font>
highlight.string => <font style="color: #DD0000">#DD0000</font> => <font style="color: #DD0000">#DD0000</font>
html_errors => Off => Off
ignore_repeated_errors => On => On
ignore_repeated_source => On => On
ignore_user_abort => Off => Off
implicit_flush => On => On
include_path => .:/usr/lib/php:/usr/local/lib/php => .:/usr/lib/php:/usr/local/lib/php
log_errors => On => On
log_errors_max_len => 1024 => 1024
magic_quotes_gpc => On => On
magic_quotes_runtime => Off => Off
magic_quotes_sybase => Off => Off
mail.force_extra_parameters => no value => no value
max_execution_time => 0 => 0
max_file_uploads => 20 => 20
max_input_nesting_level => 64 => 64
max_input_time => -1 => -1
memory_limit => 128M => 128M
open_basedir => no value => no value
output_buffering => 0 => 0
output_handler => no value => no value
post_max_size => 128M => 128M
precision => 12 => 12
realpath_cache_size => 16K => 16K
realpath_cache_ttl => 120 => 120
register_argc_argv => On => On
register_globals => Off => Off
register_long_arrays => On => On
report_memleaks => On => On
report_zend_debug => Off => Off
safe_mode => Off => Off
safe_mode_exec_dir => /usr/local/php/bin => /usr/local/php/bin
safe_mode_gid => Off => Off
safe_mode_include_dir => no value => no value
sendmail_from => no value => no value
sendmail_path => /usr/sbin/sendmail -t -i => /usr/sbin/sendmail -t -i
serialize_precision => 100 => 100
short_open_tag => On => On
SMTP => localhost => localhost
smtp_port => 25 => 25
sql.safe_mode => Off => Off
track_errors => Off => Off
unserialize_callback_func => no value => no value
upload_max_filesize => 128M => 128M
upload_tmp_dir => /tmp => /tmp
user_dir => no value => no value
variables_order => EGPCS => EGPCS
xmlrpc_error_number => 0 => 0
xmlrpc_errors => Off => Off
y2k_compliance => On => On
zend.ze1_compatibility_mode => Off => Off Accept and Award Points Accept as Solution
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Finally, I give up to
use memory instead of file
and it works fine in term speed
just a little bit slow
use memory instead of file
and it works fine in term speed
just a little bit slow
ASKER
but finally I still need to open the saved file for data operation in memory so that
sitll have out of memory issue once I open it by fopen or file() in php.
Or Fopen and File won't open and put whole file data into memory ?
Please advise
Duncan