Solved

How do I get file_get_contents or cURL working behind a proxy?

Posted on 2012-03-20
3
3,510 Views
Last Modified: 2012-03-20
Hello.

I was wondering if anyone had any advice with using PHP behind a proxy. We have an internal server (running Windows Server 2003) that serves our internal PHP (5.3.3) sites, both internally and externally (via HTTPS). I am trying to get a component called Joomdle working that will connect our Joomla 2.5.1 installation with our Moodle 1.9.12.

Whilst trying to get Joomdle working (which uses either file_get_contents or cURL), I ran into an issue that I haven't experienced before. The functions don't work because of some kind of authentication/authorisation issue and I'm not sure what to do to fix the problem.

I have set up a very simple test.php page that uses file_get_contents to return the data from our internal site (http://portal/) and stores it into a variable that is then echoed. Unfortunately, when I use the following code I get an error message which leads me to believe it could be a proxy server issue:

PHP:
<?php
$data = file_get_contents('http://portal/');
echo $data;
?>

Open in new window

Error:
Warning: file_get_contents(http://portal/) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 401 Unauthorized in F:\Inetpub\wwwroot\portal2\test.php on line 2
I also tried forcing the script through our proxy server:

PHP:
<?php
$opts = array('http' => array('proxy' => 'tcp://192.168.0.39:801', 'request_fulluri' => true));
$context = stream_context_create($opts);
$data = file_get_contents('http://portal/', false, $context);
echo $data;
?>

Open in new window

Error:
Warning: file_get_contents(http://portal/) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.0 403 Forbidden in F:\Inetpub\wwwroot\portal2\test.php on line 4
I have also tried returning the data for http://www.google.com, http://www.bbc.co.uk and our corporate website http://www.landau-forte.org.uk, all of which fail with this error message:

Error:
Warning: file_get_contents(http://www.landau-forte.org.uk/) [function.file-get-contents]: failed to open stream: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. in F:\Inetpub\wwwroot\portal2\test.php on line 2
I have asked our admins to provide me with unfiltered access through the proxy, which they have, but it still returns the same error messages. I have been making the proxy server changes in Internet Options on the server, is this right? Also, just to point out, none of the websites I am trying to get the contents for are password protected and I can access them via IE on the server.

As I will be using third party scripts, I don't want to have to edit the code to include a proxy server when using file_get_contents or cURL.

Does anyone have any advice on how to get around this or do you need any more information?

Many thanks,
James
0
Comment
Question by:nikez2k4
  • 2
3 Comments
 
LVL 109

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 37741657
I think you can dispense with file_get_contents() and just focus your attentions on using CURL.  It has many options.
http://php.net/manual/en/function.curl-setopt.php

This script can make a GET method request and show you some of the diagnostic information when the request fails.
<?php // RAY_curl_get_example.php
error_reporting(E_ALL);


// DEMONSTRATE THE BASICS OF CURL


// CHANGE THIS LINE TO SUIT YOUR NEEDS
$url = 'http://twitter.com';

// TRY THE REMOTE WEB SERVICE
$htm = my_curl($url);

// SHOW THE WORK PRODUCT OR BARK OUT ERROR MESSAGES
echo "<pre>";
echo PHP_EOL . '<strong>' . $url . '</strong>';
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;


// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $timeout=3
, $error_report=TRUE
)
{
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 
LVL 1

Author Comment

by:nikez2k4
ID: 37741999
Hi Ray

Many thanks for your reply.

I have tested your code on our server and it seems as though cURL just won't work unless I explicitly add the proxy server details. As soon as I added the following line and it worked straight away:
curl_setopt( $curl, CURLOPT_PROXY, "http://192.168.0.39:8888");

Open in new window

The problem is, you can't specify any proxy exceptions like you can in Internet Options so I still can't access our internal website. Our proxy doesn't even see the request!

After some extensive research it appears as though there isn't an easy way of globally setting proxy server details for PHP. It would make sense for them to include the options in the php.ini file...but I guess that will have to wait :(

Many thanks for your code, very useful.

Cheers,
James
0
 
LVL 109

Expert Comment

by:Ray Paseur
ID: 37742158
Thanks for the points.  I find CURL to be a lot easier to use than file_get_contents() especially for remote calls.  You can control timeout, you get better error messages, etc.

best of luck with it, ~Ray
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

These days socially coordinated efforts have turned into a critical requirement for enterprises.
This article discusses how to create an extensible mechanism for linked drop downs.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to count occurrences of each item in an array.

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question