[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 3785
  • Last Modified:

How do I get file_get_contents or cURL working behind a proxy?

Hello.

I was wondering if anyone had any advice with using PHP behind a proxy. We have an internal server (running Windows Server 2003) that serves our internal PHP (5.3.3) sites, both internally and externally (via HTTPS). I am trying to get a component called Joomdle working that will connect our Joomla 2.5.1 installation with our Moodle 1.9.12.

Whilst trying to get Joomdle working (which uses either file_get_contents or cURL), I ran into an issue that I haven't experienced before. The functions don't work because of some kind of authentication/authorisation issue and I'm not sure what to do to fix the problem.

I have set up a very simple test.php page that uses file_get_contents to return the data from our internal site (http://portal/) and stores it into a variable that is then echoed. Unfortunately, when I use the following code I get an error message which leads me to believe it could be a proxy server issue:

PHP:
<?php
$data = file_get_contents('http://portal/');
echo $data;
?>

Open in new window

Error:
Warning: file_get_contents(http://portal/) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 401 Unauthorized in F:\Inetpub\wwwroot\portal2\test.php on line 2
I also tried forcing the script through our proxy server:

PHP:
<?php
$opts = array('http' => array('proxy' => 'tcp://192.168.0.39:801', 'request_fulluri' => true));
$context = stream_context_create($opts);
$data = file_get_contents('http://portal/', false, $context);
echo $data;
?>

Open in new window

Error:
Warning: file_get_contents(http://portal/) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.0 403 Forbidden in F:\Inetpub\wwwroot\portal2\test.php on line 4
I have also tried returning the data for http://www.google.com, http://www.bbc.co.uk and our corporate website http://www.landau-forte.org.uk, all of which fail with this error message:

Error:
Warning: file_get_contents(http://www.landau-forte.org.uk/) [function.file-get-contents]: failed to open stream: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. in F:\Inetpub\wwwroot\portal2\test.php on line 2
I have asked our admins to provide me with unfiltered access through the proxy, which they have, but it still returns the same error messages. I have been making the proxy server changes in Internet Options on the server, is this right? Also, just to point out, none of the websites I am trying to get the contents for are password protected and I can access them via IE on the server.

As I will be using third party scripts, I don't want to have to edit the code to include a proxy server when using file_get_contents or cURL.

Does anyone have any advice on how to get around this or do you need any more information?

Many thanks,
James
0
nikez2k4
Asked:
nikez2k4
  • 2
1 Solution
 
Ray PaseurCommented:
I think you can dispense with file_get_contents() and just focus your attentions on using CURL.  It has many options.
http://php.net/manual/en/function.curl-setopt.php

This script can make a GET method request and show you some of the diagnostic information when the request fails.
<?php // RAY_curl_get_example.php
error_reporting(E_ALL);


// DEMONSTRATE THE BASICS OF CURL


// CHANGE THIS LINE TO SUIT YOUR NEEDS
$url = 'http://twitter.com';

// TRY THE REMOTE WEB SERVICE
$htm = my_curl($url);

// SHOW THE WORK PRODUCT OR BARK OUT ERROR MESSAGES
echo "<pre>";
echo PHP_EOL . '<strong>' . $url . '</strong>';
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;


// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $timeout=3
, $error_report=TRUE
)
{
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 
nikez2k4Author Commented:
Hi Ray

Many thanks for your reply.

I have tested your code on our server and it seems as though cURL just won't work unless I explicitly add the proxy server details. As soon as I added the following line and it worked straight away:
curl_setopt( $curl, CURLOPT_PROXY, "http://192.168.0.39:8888");

Open in new window

The problem is, you can't specify any proxy exceptions like you can in Internet Options so I still can't access our internal website. Our proxy doesn't even see the request!

After some extensive research it appears as though there isn't an easy way of globally setting proxy server details for PHP. It would make sense for them to include the options in the php.ini file...but I guess that will have to wait :(

Many thanks for your code, very useful.

Cheers,
James
0
 
Ray PaseurCommented:
Thanks for the points.  I find CURL to be a lot easier to use than file_get_contents() especially for remote calls.  You can control timeout, you get better error messages, etc.

best of luck with it, ~Ray
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now