Solved

php not loading all of file with file_get_contents and very slow

Posted on 2007-03-19
16
850 Views
Last Modified: 2014-11-12
Well I am having a problem and I don't know if it is the server or php or what.

I have this code that restyles the google calendar with my own css for my webpage.  
The code used to work and now it just kind of stopped.  Basically now the page acts like it is loading for a minute then only displays part of the page.  

My code reads the html in and replaces the css lines and hyperlinks with ones to my page.  But the problem is that it stops reading after so many lines and just prints that to the screen.

But I tested this on my server at work and it works fine loads in 2 secs and displays everything correctly.  
But on my godaddy server it doesn't work so great anymore.

Here is my code.

$stylesheet = 'http://rlchome.org/googlecal/googlecal.css';



/* Default URL

 * This is the embeddable public address for the calendar.  If this value is an

 * empty string, then the script will return a blank page if a valid query

 * string is not submitted. This URL will typically be of the form

 *

 *   http://www.google.com/calendar/embed?src=user%40domain.tld

 *

 * where user@domain.tld is a valid Google Calendar account

 */



$url = "http://www.google.com/calendar/embed?src=brenda%40churchthatcares.org&chrome=NAVIGATION&mode=MONTH&height=900";
//print $url;

// Request the calendar

$buffer = file_get_contents($url);

// Fix hrefs, image sources, and stylesheet

$pattern = '/(href="render)/';

$replacement = 'href="http://www.google.com/calendar/render';

$buffer = preg_replace($pattern, $replacement, $buffer);



$pattern = '/(href="event)/';

$replacement = 'href="http://www.google.com/calendar/event';

$buffer = preg_replace($pattern, $replacement, $buffer);



$pattern = '/(http:\/\/www.google.com\/calendar\/embed)/';

$replacement = 'cal.php';

$buffer = preg_replace($pattern, $replacement, $buffer);



$pattern = '/(src="images)/';

$replacement = 'src="http://rlchome.org/googlecal/images';

$buffer = preg_replace($pattern, $replacement, $buffer);



$pattern = '/(<link.*>)/';

$replacement = '<link rel="stylesheet" type="text/css" href="' . $stylesheet . '" />';

$buffer = preg_replace($pattern, $replacement, $buffer);



// display the calendar

print $buffer;



Also I have tried replaceing the file get conents with this.  No difference.

$fd = fopen ($url, "r");

while (!feof ($fd))

    $buffer .= fread($fd, 8192);

fclose ($fd);



Here is the page. I have tried php4 and 5 both do the same thing.  
http://rlchome.org/googlecal/cal.php


So any help would be appreciated.  I am stuck on this one.  
0
Comment
Question by:fixitben
  • 6
  • 6
  • 3
  • +1
16 Comments
 
LVL 48

Expert Comment

by:hernst42
Comment Utility
add timestamps around the file_get_contents to see if that function needs the time, or if only the fopen needs the time. In this case godaddy might be solwing down access to the internet from their servers to protect their server from acting for a DDoS-attack.
0
 
LVL 12

Expert Comment

by:adrian_brooks
Comment Utility
I think I have to agree with hernst42 on the GoDaddy side of things. Our company is an IT/Hosting company and we're using GoDaddy for a lot of stuff, including having servers with them. There have been times that I have noticed, what appears to be, some form of bandwidth throttling going on. It never downs any of our hosted domains, but execution time definitely seems to be a problem.

I'm not sure that this is the source of your problem, but felt it to be worthy of note.
0
 
LVL 5

Expert Comment

by:sebastienbo
Comment Utility
Hey Ben,

could you provide me the size of $buffer = file_get_contents($url);
and the time for getting content.

The size can help me know if this is size limit issue and the time can give away a time limit (connection time limit)

so give me the seconds that your file_get_content works and also the strlen($buffer)
0
 
LVL 7

Author Comment

by:fixitben
Comment Utility
Ok I am not sure what you mean by "add timestamps around the file_get_contents to see if that function needs the time"

So this is what I did.

print date("d:i:s");
$buffer = file_get_contents($url);
print date("d:i:s");
print strlen($buffer);

And on my work server I get this.  "DISPLAYS CORRECTLY"
20:48:57
20:49:08
79940

On godaddys server i get this.  "GETS TO A CERTIAN POINT AND PRINTS HALF OF THE PAGE"
20:58:31
20:59:33
3799

It looks like it is only reading in 3000 charatcters but my server reads in all 79,000.  It looks like it is stoping at 60 seconds, but it shouldn't take 60 seconds to read in that text.
Any Ideas.

Thanks
Fixitben

0
 
LVL 48

Expert Comment

by:hernst42
Comment Utility
So i guess it's a problem at godaddy hosting service, which may block/filter/throttle outgoing connections to other servers. Open a ticket their with your problem. It's not a problem with php itself only with the network of godaddy.
0
 
LVL 5

Expert Comment

by:sebastienbo
Comment Utility
Ben,

It's possible that the function  file_get_contents is not completely supported.
try instead this :

$buffer=implode('',file($url))

The 60 second limit is reached because the function had a problem, maybe because outgoing connections are not allowed via that function (php safe mode)

So try the implode read thingie
0
 
LVL 48

Expert Comment

by:hernst42
Comment Utility
@sebastienbo:
php internaly uses the same c functions to open the file for file and file_get_contents, so all restrictions which apply to file_get_contents also apply to file. I bet the $buffer=implode('',file($url)) does also not return the whole page.
0
 
LVL 5

Expert Comment

by:sebastienbo
Comment Utility
hernst42,

Speaking of restictions this is indeed true but php safe mode works on interpreter level, and thus can make difference between functions
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 7

Author Comment

by:fixitben
Comment Utility
Ok Godaddy said that I need to run through a proxy server.

with code like this

curl_setopt ($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
curl_setopt ($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_setopt ($ch, CURLOPT_PROXY, http://proxy.shr.secureserver.net:3128);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);

I don't quite understand the code above.  Do you know how this would work in my example.


Thanks
Fixitben
0
 
LVL 5

Expert Comment

by:sebastienbo
Comment Utility
<?php
$ch = curl_init(); // create cURL handle (ch)
if (!$ch)  die("Couldn't initialize a cURL handle");

// set some cURL options
curl_setopt ($ch, CURLOPT_HTTPPROXYTUNNEL, TRUE);
curl_setopt ($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_setopt ($ch, CURLOPT_PROXY, http://proxy.shr.secureserver.net:3128);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);

$ret = curl_setopt($ch, CURLOPT_URL, "http://www.vbo.nl/");
$ret = curl_setopt($ch, CURLOPT_HEADER, 1);
$ret = curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$ret = curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
$ret = curl_setopt($ch, CURLOPT_TIMEOUT, 130);

// execute
$ret = curl_exec($ch);
if (empty($ret))
{
   die(curl_error($ch));
   curl_close($ch); // close cURL handler
} else {
   $info = curl_getinfo($ch);
   curl_close($ch); // close cURL handler
}

//$ret contains your page

?>


0
 
LVL 7

Author Comment

by:fixitben
Comment Utility
Now I get this

Received error code 403 from proxy

The only thing I changed from your code was putting " " around the proxy URL.

Thanks
fixitben
0
 
LVL 7

Author Comment

by:fixitben
Comment Utility
Well I moved the url out and set it to a variable but now I get this

http://rlchome.org/googlecal/cal5.php

Which it still stops at the same point and doesn't load any more.  But now it does it quick.

So I did this one with a different url and it loads the whole page.  
http://rlchome.org/googlecal/cal6.php

Do you guys have any ideas on why the partial load  is occuring for the calendar.

I think we are getting close. Thanks for all of your help so far.

Thanks
Fixitben
0
 
LVL 5

Expert Comment

by:sebastienbo
Comment Utility
Ben,

curl is not one of my favourites because it is not portable in case you change of hosting plan, you will need curl again !!

instead you could bypass the proxy using standard internal functions from php

change your code like this :

$page="";
/*your proxy server address*/
$proxy = "192.168.10.1";
/*your proxy server port*/
$port = 8080;
/*the url you want to connect to*/
$url = "http://www.php.net/";

// The code to get your $page
$fp = fsockopen($proxy, $port);
fputs($fp, "GET $url HTTP/1.0\r\nHost: $proxy\r\n\r\n");
while(!feof($fp)) $page.= fgets($fp, 4000);
fclose($fp);

echo $page;
0
 
LVL 7

Author Comment

by:fixitben
Comment Utility
Ok It will load all kinds of pages execpt the one that I need.

With this code it just times out and the browser quits.

<?php
$page="";
/*your proxy server address*/
$proxy = "64.202.165.130";
/*your proxy server port*/
$port = 3128;
/*the url you want to connect to*/
$url = "http://www.google.com/calendar/embed?src=brenda%40churchthatcares.org&mode=MONTH";

// The code to get your $page
$fp = fsockopen($proxy, $port);
fputs($fp, "GET $url HTTP/1.0\r\nHost: $proxy\r\n\r\n");
while(!feof($fp)) $page.= fgets($fp, 4000);
fclose($fp);

echo $page;
?>

But if you change the url to have a / at the end it prints the google invlaid url error on the page.

Here is a link to the one that times out
http://rlchome.org/googlecal/call7.php


Here is a link to the one that prints the error.  
http://rlchome.org/googlecal/call8.php


I don't know what this means but maybe you will.

Thanks
Fixitben
0
 
LVL 5

Accepted Solution

by:
sebastienbo earned 500 total points
Comment Utility
Ok, I've seen that once.
"google.com" the server checks for headers that are sent by your browser, but fsockopen doesn't send those headers, you should mannually give them in.

Something like :
       $requestHeader = $method." ".$path."  HTTP/1.1\r\n";
       $requestHeader.= "Host: ".$host."\r\n";
       $requestHeader.= "User-Agent:      Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0\r\n";
       $requestHeader.= "Content-Type: application/x-www-form-urlencoded\r\n";

I get this from php.net : http://be2.php.net/fsockopen

Beware: as you need first to make a connection to the proxy in the fsockopen requests, so you should find out where to put those headers.

Hope that helps
0
 
LVL 7

Author Comment

by:fixitben
Comment Utility
Well this seems a little over my head.  So I just got some cheap space on a server and used an iframe to display the calendar.

If you have an example that I can try feel free. Cause I would like to see it actually work.  But for now I have a work around.


Basically I got really lost on your last post.

Thanks
Fixitben
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

You can provide a virtual interface for remote stakeholders in a SWOT analysis through a Google Drawing template. By making real time viewing and collaboration possible, your team can build a stronger product.
These days socially coordinated efforts have turned into a critical requirement for enterprises.
This Micro Tutorial demonstrates the importance of annotations in Google Analytics and how they should be used to document changes made to a site, Google updates (Ex: Panda & Penguin), marketing campaigns, and any other events that might have contri…
In this Experts Exchange video Micro Tutorial, I'm going to show how small business owners who use Google Apps can save money by setting up what is called a catch-all email address in their Gmail accounts. By using the catch-all feature, small busin…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now