Solved

How do I get file_get_contents or cURL working behind a proxy?

Posted on 2012-03-20
3
3,388 Views
Last Modified: 2012-03-20
Hello.

I was wondering if anyone had any advice with using PHP behind a proxy. We have an internal server (running Windows Server 2003) that serves our internal PHP (5.3.3) sites, both internally and externally (via HTTPS). I am trying to get a component called Joomdle working that will connect our Joomla 2.5.1 installation with our Moodle 1.9.12.

Whilst trying to get Joomdle working (which uses either file_get_contents or cURL), I ran into an issue that I haven't experienced before. The functions don't work because of some kind of authentication/authorisation issue and I'm not sure what to do to fix the problem.

I have set up a very simple test.php page that uses file_get_contents to return the data from our internal site (http://portal/) and stores it into a variable that is then echoed. Unfortunately, when I use the following code I get an error message which leads me to believe it could be a proxy server issue:

PHP:
<?php
$data = file_get_contents('http://portal/');
echo $data;
?>

Open in new window

Error:
Warning: file_get_contents(http://portal/) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 401 Unauthorized in F:\Inetpub\wwwroot\portal2\test.php on line 2
I also tried forcing the script through our proxy server:

PHP:
<?php
$opts = array('http' => array('proxy' => 'tcp://192.168.0.39:801', 'request_fulluri' => true));
$context = stream_context_create($opts);
$data = file_get_contents('http://portal/', false, $context);
echo $data;
?>

Open in new window

Error:
Warning: file_get_contents(http://portal/) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.0 403 Forbidden in F:\Inetpub\wwwroot\portal2\test.php on line 4
I have also tried returning the data for http://www.google.com, http://www.bbc.co.uk and our corporate website http://www.landau-forte.org.uk, all of which fail with this error message:

Error:
Warning: file_get_contents(http://www.landau-forte.org.uk/) [function.file-get-contents]: failed to open stream: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. in F:\Inetpub\wwwroot\portal2\test.php on line 2
I have asked our admins to provide me with unfiltered access through the proxy, which they have, but it still returns the same error messages. I have been making the proxy server changes in Internet Options on the server, is this right? Also, just to point out, none of the websites I am trying to get the contents for are password protected and I can access them via IE on the server.

As I will be using third party scripts, I don't want to have to edit the code to include a proxy server when using file_get_contents or cURL.

Does anyone have any advice on how to get around this or do you need any more information?

Many thanks,
James
0
Comment
Question by:nikez2k4
  • 2
3 Comments
 
LVL 108

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 37741657
I think you can dispense with file_get_contents() and just focus your attentions on using CURL.  It has many options.
http://php.net/manual/en/function.curl-setopt.php

This script can make a GET method request and show you some of the diagnostic information when the request fails.
<?php // RAY_curl_get_example.php
error_reporting(E_ALL);


// DEMONSTRATE THE BASICS OF CURL


// CHANGE THIS LINE TO SUIT YOUR NEEDS
$url = 'http://twitter.com';

// TRY THE REMOTE WEB SERVICE
$htm = my_curl($url);

// SHOW THE WORK PRODUCT OR BARK OUT ERROR MESSAGES
echo "<pre>";
echo PHP_EOL . '<strong>' . $url . '</strong>';
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;


// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $timeout=3
, $error_report=TRUE
)
{
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 
LVL 1

Author Comment

by:nikez2k4
ID: 37741999
Hi Ray

Many thanks for your reply.

I have tested your code on our server and it seems as though cURL just won't work unless I explicitly add the proxy server details. As soon as I added the following line and it worked straight away:
curl_setopt( $curl, CURLOPT_PROXY, "http://192.168.0.39:8888");

Open in new window

The problem is, you can't specify any proxy exceptions like you can in Internet Options so I still can't access our internal website. Our proxy doesn't even see the request!

After some extensive research it appears as though there isn't an easy way of globally setting proxy server details for PHP. It would make sense for them to include the options in the php.ini file...but I guess that will have to wait :(

Many thanks for your code, very useful.

Cheers,
James
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 37742158
Thanks for the points.  I find CURL to be a lot easier to use than file_get_contents() especially for remote calls.  You can control timeout, you get better error messages, etc.

best of luck with it, ~Ray
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Part of the Global Positioning System A geocode (https://developers.google.com/maps/documentation/geocoding/) is the major subset of a GPS coordinate (http://en.wikipedia.org/wiki/Global_Positioning_System), the other parts being the altitude and t…
Password hashing is better than message digests or encryption, and you should be using it instead of message digests or encryption.  Find out why and how in this article, which supplements the original article on PHP Client Registration, Login, Logo…
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now