PHP read web page contents

Posted on 2006-04-20
Last Modified: 2008-01-09
I've been trying to read the contents from a particular URL into a string in PHP, and can't get it to work.  I've tried the Request.PHP PEAR module, the cURL methods, and the standard file_get_contents() function, but no matter what I do, I can't seem to get anything.

I'm running PHP 5 w/ IIS 6 on a Win2k3 server.  The URL I'm trying to read responds with an XML fragment, based on some query data I send to it.  There URL changes based on the query data, but when I output the URL from my script into a link and click it, I get the XML fragment returned to my browser with no trouble.  So I know that the URL I'm trying to pass to the various methods is indeed valid and will indeed respond with an XML fragment.

Again, when I try to get the same fragment into a from within my PHP script (using the URL from a string variable) into a string, I get nothing.  I've tried code that apparently works on an Apache server (using the Request.PHP module), but nothing I try gets anything.  I've used all kinds of var_dumps to see what's going on, and everything looks like it checks out, but nothing ever comes through into the string.

Can anyone think of what the problem might be?  I'm really struggling here and cannot for the life of me figure out what the problem is.  I've got another question on here with a title like PHP / CAS question that has more information.  I've been screwing with this for over a week now and I need to figure it out.
Question by:Zeek0
    LVL 11

    Assisted Solution

    LVL 9

    Expert Comment

    whats the url your trying to get. If we can retrieve it ok, then we can offer a solution that should work for you. (win200, apache 2, mysql 4)

    p.s. I luv curl.
    LVL 2

    Accepted Solution

    well here is a piece of code that I have used to read parts of other pages.  maybe it will help

    <form action="harvester.php" method="post">
          Read from site:<input type="text" value="" name="address"><br>
          Start at word:<input type="text" value="" name="start"><br>
          End at word:<input type="text" value="" name="stop"><br>
          <input type="submit" value="submit" name="submit">

    if($submit) {       

          $handle = fopen($address, "r"); // point stream to site
          while (!feof($handle)) {
                $contents .= fread($handle, 8192); // read site into buffer
          echo "searching site: " . $address ."<br>";

          // pick word to start reading from
          $start_pos = strpos($contents, $start);  // find position of that word
          echo "Starting at: " . $start . "<br>found at location: " . $start_pos . "<br>";

          $first_trim = substr($contents, $start_pos);

          // pick word to stop reading at
          $stop_pos = strpos($first_trim, $stop);  // find position of that word
          echo "Stoping at: " . $stop . "<br>found at location: " . $stop_pos . "<br>";

          // cut the string before and after selected words
          $second_trim = substr($first_trim, 0, $stop_pos);

          echo "<p>" . $second_trim;
    LVL 6

    Assisted Solution

    I guess the problem in W2K3. You have to check security settings, because it can block attempts of your scripts to connect somewhere outside. You can try to read contents of web page located on the same server, for example trya to recieve with "http://localhost/getcontent.php" a content of "http://localhost/sample.php" and if so, then do something with security of OS :))

    Author Comment

    I figured out that the problem was, in fact, with IIS 6.  The application pooling doesn't allow sockets to be created, let alone opened and used to transfer data.  Lord only knows why, but I got the script I had to work when I set the application pooling to "IIS Isolation Mode" and chose "Low (IIS Process)" for the 'application protection' setting.  

    I don't know why I had to change those settings, but they made my script work.

    Featured Post

    What Security Threats Are You Missing?

    Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

    Join & Write a Comment

    Suggested Solutions

    Popularity Can Be Measured Sometimes we deal with questions of popularity, and we need a way to collect opinions from our clients.  This article shows a simple teaching example of how we might elect a favorite color by letting our clients vote for …
    These days socially coordinated efforts have turned into a critical requirement for enterprises.
    Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
    The viewer will learn how to count occurrences of each item in an array.

    733 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now