Solved

php not loading all of file with file_get_contents and very slow

Posted on 2007-03-19
16
860 Views
Last Modified: 2014-11-12
Well I am having a problem and I don't know if it is the server or php or what.

I have this code that restyles the google calendar with my own css for my webpage.  
The code used to work and now it just kind of stopped.  Basically now the page acts like it is loading for a minute then only displays part of the page.  

My code reads the html in and replaces the css lines and hyperlinks with ones to my page.  But the problem is that it stops reading after so many lines and just prints that to the screen.

But I tested this on my server at work and it works fine loads in 2 secs and displays everything correctly.  
But on my godaddy server it doesn't work so great anymore.

Here is my code.

$stylesheet = 'http://rlchome.org/googlecal/googlecal.css';



/* Default URL

 * This is the embeddable public address for the calendar.  If this value is an

 * empty string, then the script will return a blank page if a valid query

 * string is not submitted. This URL will typically be of the form

 *

 *   http://www.google.com/calendar/embed?src=user%40domain.tld

 *

 * where user@domain.tld is a valid Google Calendar account

 */



$url = "http://www.google.com/calendar/embed?src=brenda%40churchthatcares.org&chrome=NAVIGATION&mode=MONTH&height=900";
//print $url;

// Request the calendar

$buffer = file_get_contents($url);

// Fix hrefs, image sources, and stylesheet

$pattern = '/(href="render)/';

$replacement = 'href="http://www.google.com/calendar/render';

$buffer = preg_replace($pattern, $replacement, $buffer);



$pattern = '/(href="event)/';

$replacement = 'href="http://www.google.com/calendar/event';

$buffer = preg_replace($pattern, $replacement, $buffer);



$pattern = '/(http:\/\/www.google.com\/calendar\/embed)/';

$replacement = 'cal.php';

$buffer = preg_replace($pattern, $replacement, $buffer);



$pattern = '/(src="images)/';

$replacement = 'src="http://rlchome.org/googlecal/images';

$buffer = preg_replace($pattern, $replacement, $buffer);



$pattern = '/(<link.*>)/';

$replacement = '<link rel="stylesheet" type="text/css" href="' . $stylesheet . '" />';

$buffer = preg_replace($pattern, $replacement, $buffer);



// display the calendar

print $buffer;



Also I have tried replaceing the file get conents with this.  No difference.

$fd = fopen ($url, "r");

while (!feof ($fd))

    $buffer .= fread($fd, 8192);

fclose ($fd);



Here is the page. I have tried php4 and 5 both do the same thing.  
http://rlchome.org/googlecal/cal.php


So any help would be appreciated.  I am stuck on this one.  
0
Comment
Question by:fixitben
  • 6
  • 6
  • 3
  • +1
16 Comments
 
LVL 48

Expert Comment

by:hernst42
ID: 18754647
add timestamps around the file_get_contents to see if that function needs the time, or if only the fopen needs the time. In this case godaddy might be solwing down access to the internet from their servers to protect their server from acting for a DDoS-attack.
0
 
LVL 12

Expert Comment

by:adrian_brooks
ID: 18754813
I think I have to agree with hernst42 on the GoDaddy side of things. Our company is an IT/Hosting company and we're using GoDaddy for a lot of stuff, including having servers with them. There have been times that I have noticed, what appears to be, some form of bandwidth throttling going on. It never downs any of our hosted domains, but execution time definitely seems to be a problem.

I'm not sure that this is the source of your problem, but felt it to be worthy of note.
0
 
LVL 5

Expert Comment

by:sebastienbo
ID: 18754896
Hey Ben,

could you provide me the size of $buffer = file_get_contents($url);
and the time for getting content.

The size can help me know if this is size limit issue and the time can give away a time limit (connection time limit)

so give me the seconds that your file_get_content works and also the strlen($buffer)
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 
LVL 7

Author Comment

by:fixitben
ID: 18756285
Ok I am not sure what you mean by "add timestamps around the file_get_contents to see if that function needs the time"

So this is what I did.

print date("d:i:s");
$buffer = file_get_contents($url);
print date("d:i:s");
print strlen($buffer);

And on my work server I get this.  "DISPLAYS CORRECTLY"
20:48:57
20:49:08
79940

On godaddys server i get this.  "GETS TO A CERTIAN POINT AND PRINTS HALF OF THE PAGE"
20:58:31
20:59:33
3799

It looks like it is only reading in 3000 charatcters but my server reads in all 79,000.  It looks like it is stoping at 60 seconds, but it shouldn't take 60 seconds to read in that text.
Any Ideas.

Thanks
Fixitben

0
 
LVL 48

Expert Comment

by:hernst42
ID: 18757235
So i guess it's a problem at godaddy hosting service, which may block/filter/throttle outgoing connections to other servers. Open a ticket their with your problem. It's not a problem with php itself only with the network of godaddy.
0
 
LVL 5

Expert Comment

by:sebastienbo
ID: 18757606
Ben,

It's possible that the function  file_get_contents is not completely supported.
try instead this :

$buffer=implode('',file($url))

The 60 second limit is reached because the function had a problem, maybe because outgoing connections are not allowed via that function (php safe mode)

So try the implode read thingie
0
 
LVL 48

Expert Comment

by:hernst42
ID: 18757798
@sebastienbo:
php internaly uses the same c functions to open the file for file and file_get_contents, so all restrictions which apply to file_get_contents also apply to file. I bet the $buffer=implode('',file($url)) does also not return the whole page.
0
 
LVL 5

Expert Comment

by:sebastienbo
ID: 18757912
hernst42,

Speaking of restictions this is indeed true but php safe mode works on interpreter level, and thus can make difference between functions
0
 
LVL 7

Author Comment

by:fixitben
ID: 18759115
Ok Godaddy said that I need to run through a proxy server.

with code like this

curl_setopt ($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
curl_setopt ($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_setopt ($ch, CURLOPT_PROXY, http://proxy.shr.secureserver.net:3128);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);

I don't quite understand the code above.  Do you know how this would work in my example.


Thanks
Fixitben
0
 
LVL 5

Expert Comment

by:sebastienbo
ID: 18759505
<?php
$ch = curl_init(); // create cURL handle (ch)
if (!$ch)  die("Couldn't initialize a cURL handle");

// set some cURL options
curl_setopt ($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
curl_setopt ($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_setopt ($ch, CURLOPT_PROXY, http://proxy.shr.secureserver.net:3128);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);

$ret = curl_setopt($ch, CURLOPT_URL, "http://www.vbo.nl/");
$ret = curl_setopt($ch, CURLOPT_HEADER, 1);
$ret = curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$ret = curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
$ret = curl_setopt($ch, CURLOPT_TIMEOUT, 130);

// execute
$ret = curl_exec($ch);
if (empty($ret))
{
   die(curl_error($ch));
   curl_close($ch); // close cURL handler
} else {
   $info = curl_getinfo($ch);
   curl_close($ch); // close cURL handler
}

//$ret contains your page

?>


0
 
LVL 7

Author Comment

by:fixitben
ID: 18759576
Now I get this

Received error code 403 from proxy

The only thing I changed from your code was putting " " around the proxy URL.

Thanks
fixitben
0
 
LVL 7

Author Comment

by:fixitben
ID: 18759643
Well I moved the url out and set it to a variable but now I get this

http://rlchome.org/googlecal/cal5.php

Which it still stops at the same point and doesn't load any more.  But now it does it quick.

So I did this one with a different url and it loads the whole page.  
http://rlchome.org/googlecal/cal6.php

Do you guys have any ideas on why the partial load  is occuring for the calendar.

I think we are getting close. Thanks for all of your help so far.

Thanks
Fixitben
0
 
LVL 5

Expert Comment

by:sebastienbo
ID: 18762930
Ben,

curl is not one of my favourites because it is not portable in case you change of hosting plan, you will need curl again !!

instead you could bypass the proxy using standard internal functions from php

change your code like this :

$page="";
/*your proxy server address*/
$proxy = "192.168.10.1";
/*your proxy server port*/
$port = 8080;
/*the url you want to connect to*/
$url = "http://www.php.net/";

// The code to get your $page
$fp = fsockopen($proxy, $port);
fputs($fp, "GET $url HTTP/1.0\r\nHost: $proxy\r\n\r\n");
while(!feof($fp)) $page.= fgets($fp, 4000);
fclose($fp);

echo $page;
0
 
LVL 7

Author Comment

by:fixitben
ID: 18764722
Ok It will load all kinds of pages execpt the one that I need.

With this code it just times out and the browser quits.

<?php
$page="";
/*your proxy server address*/
$proxy = "64.202.165.130";
/*your proxy server port*/
$port = 3128;
/*the url you want to connect to*/
$url = "http://www.google.com/calendar/embed?src=brenda%40churchthatcares.org&mode=MONTH";

// The code to get your $page
$fp = fsockopen($proxy, $port);
fputs($fp, "GET $url HTTP/1.0\r\nHost: $proxy\r\n\r\n");
while(!feof($fp)) $page.= fgets($fp, 4000);
fclose($fp);

echo $page;
?>

But if you change the url to have a / at the end it prints the google invlaid url error on the page.

Here is a link to the one that times out
http://rlchome.org/googlecal/call7.php


Here is a link to the one that prints the error.  
http://rlchome.org/googlecal/call8.php


I don't know what this means but maybe you will.

Thanks
Fixitben
0
 
LVL 5

Accepted Solution

by:
sebastienbo earned 500 total points
ID: 18770147
Ok, I've seen that once.
"google.com" the server checks for headers that are sent by your browser, but fsockopen doesn't send those headers, you should mannually give them in.

Something like :
       $requestHeader = $method." ".$path."  HTTP/1.1\r\n";
       $requestHeader.= "Host: ".$host."\r\n";
       $requestHeader.= "User-Agent:      Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0\r\n";
       $requestHeader.= "Content-Type: application/x-www-form-urlencoded\r\n";

I get this from php.net : http://be2.php.net/fsockopen

Beware: as you need first to make a connection to the proxy in the fsockopen requests, so you should find out where to put those headers.

Hope that helps
0
 
LVL 7

Author Comment

by:fixitben
ID: 18777517
Well this seems a little over my head.  So I just got some cheap space on a server and used an iframe to display the calendar.

If you have an example that I can try feel free. Cause I would like to see it actually work.  But for now I have a work around.


Basically I got really lost on your last post.

Thanks
Fixitben
0

Featured Post

Ransomware-A Revenue Bonanza for Service Providers

Ransomware – malware that gets on your customers’ computers, encrypts their data, and extorts a hefty ransom for the decryption keys – is a surging new threat.  The purpose of this eBook is to educate the reader about ransomware attacks.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Introduction This article is intended for those who are new to PHP error handling (https://www.experts-exchange.com/articles/11769/And-by-the-way-I-am-New-to-PHP.html).  It addresses one of the most common problems that plague beginning PHP develop…
3 proven steps to speed up Magento powered sites. The article focus is on optimizing time to first byte (TTFB), full page caching and configuring server for optimal performance.
This Micro Tutorial will demonstrate the easy use of Gmail embedding images in your email so the recipient of your email can view them in context.
This Micro Tutorial will demonstrate using Google Doc how to import live data to another spreadsheet in Google Spreadsheets using the IMPORTRANGE function.

773 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question