[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

How do I get file_get_contents or cURL working behind a proxy?

Posted on 2012-03-20
3
Medium Priority
?
3,729 Views
Last Modified: 2012-03-20
Hello.

I was wondering if anyone had any advice with using PHP behind a proxy. We have an internal server (running Windows Server 2003) that serves our internal PHP (5.3.3) sites, both internally and externally (via HTTPS). I am trying to get a component called Joomdle working that will connect our Joomla 2.5.1 installation with our Moodle 1.9.12.

Whilst trying to get Joomdle working (which uses either file_get_contents or cURL), I ran into an issue that I haven't experienced before. The functions don't work because of some kind of authentication/authorisation issue and I'm not sure what to do to fix the problem.

I have set up a very simple test.php page that uses file_get_contents to return the data from our internal site (http://portal/) and stores it into a variable that is then echoed. Unfortunately, when I use the following code I get an error message which leads me to believe it could be a proxy server issue:

PHP:
<?php
$data = file_get_contents('http://portal/');
echo $data;
?>

Open in new window

Error:
Warning: file_get_contents(http://portal/) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 401 Unauthorized in F:\Inetpub\wwwroot\portal2\test.php on line 2
I also tried forcing the script through our proxy server:

PHP:
<?php
$opts = array('http' => array('proxy' => 'tcp://192.168.0.39:801', 'request_fulluri' => true));
$context = stream_context_create($opts);
$data = file_get_contents('http://portal/', false, $context);
echo $data;
?>

Open in new window

Error:
Warning: file_get_contents(http://portal/) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.0 403 Forbidden in F:\Inetpub\wwwroot\portal2\test.php on line 4
I have also tried returning the data for http://www.google.com, http://www.bbc.co.uk and our corporate website http://www.landau-forte.org.uk, all of which fail with this error message:

Error:
Warning: file_get_contents(http://www.landau-forte.org.uk/) [function.file-get-contents]: failed to open stream: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. in F:\Inetpub\wwwroot\portal2\test.php on line 2
I have asked our admins to provide me with unfiltered access through the proxy, which they have, but it still returns the same error messages. I have been making the proxy server changes in Internet Options on the server, is this right? Also, just to point out, none of the websites I am trying to get the contents for are password protected and I can access them via IE on the server.

As I will be using third party scripts, I don't want to have to edit the code to include a proxy server when using file_get_contents or cURL.

Does anyone have any advice on how to get around this or do you need any more information?

Many thanks,
James
0
Comment
Question by:nikez2k4
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 111

Accepted Solution

by:
Ray Paseur earned 2000 total points
ID: 37741657
I think you can dispense with file_get_contents() and just focus your attentions on using CURL.  It has many options.
http://php.net/manual/en/function.curl-setopt.php

This script can make a GET method request and show you some of the diagnostic information when the request fails.
<?php // RAY_curl_get_example.php
error_reporting(E_ALL);


// DEMONSTRATE THE BASICS OF CURL


// CHANGE THIS LINE TO SUIT YOUR NEEDS
$url = 'http://twitter.com';

// TRY THE REMOTE WEB SERVICE
$htm = my_curl($url);

// SHOW THE WORK PRODUCT OR BARK OUT ERROR MESSAGES
echo "<pre>";
echo PHP_EOL . '<strong>' . $url . '</strong>';
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;


// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $timeout=3
, $error_report=TRUE
)
{
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 
LVL 1

Author Comment

by:nikez2k4
ID: 37741999
Hi Ray

Many thanks for your reply.

I have tested your code on our server and it seems as though cURL just won't work unless I explicitly add the proxy server details. As soon as I added the following line and it worked straight away:
curl_setopt( $curl, CURLOPT_PROXY, "http://192.168.0.39:8888");

Open in new window

The problem is, you can't specify any proxy exceptions like you can in Internet Options so I still can't access our internal website. Our proxy doesn't even see the request!

After some extensive research it appears as though there isn't an easy way of globally setting proxy server details for PHP. It would make sense for them to include the options in the php.ini file...but I guess that will have to wait :(

Many thanks for your code, very useful.

Cheers,
James
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 37742158
Thanks for the points.  I find CURL to be a lot easier to use than file_get_contents() especially for remote calls.  You can control timeout, you get better error messages, etc.

best of luck with it, ~Ray
0

Featured Post

Learn how to optimize MySQL for your business need

With the increasing importance of apps & networks in both business & personal interconnections, perfor. has become one of the key metrics of successful communication. This ebook is a hands-on business-case-driven guide to understanding MySQL query parameter tuning & database perf

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Build an array called $myWeek which will hold the array elements Today, Yesterday and then builds up the rest of the week by the name of the day going back 1 week.   (CODE) (CODE) Then you just need to pass your date to the function. If i…
Password hashing is better than message digests or encryption, and you should be using it instead of message digests or encryption.  Find out why and how in this article, which supplements the original article on PHP Client Registration, Login, Logo…
The viewer will learn how to count occurrences of each item in an array.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.
Suggested Courses

649 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question