Solved

Using CURL to get File Contents returns 404 File not found

Posted on 2014-03-03
27
5,496 Views
Last Modified: 2014-04-14
Here is the script

<?php 
function file_get_contents_curl($url) {
    $ch = curl_init();

    curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);       

    $data = curl_exec($ch);
    curl_close($ch);
	echo $data;
}
file_get_contents_curl('http://mediaridge.com/script/twitterapp.php');
?>

Open in new window


but yet it returns 404 file not found... here is a test page:

http://mediaridge.com/test1.php


PHP Info for reference http://mediaridge.com/test.php
0
Comment
Question by:jporter80
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 13
  • 10
  • 4
27 Comments
 
LVL 14

Expert Comment

by:shobinsun
ID: 39902439
Hi,

Your code is fine and I am able to fetch the json data. did you check the file: http://mediaridge.com/test1.php is there in the server?? It looks like the test1.php file is not there in the server.

Regards,
Shobin Markose.
0
 
LVL 14

Expert Comment

by:shobinsun
ID: 39902441
Hi,

Also check whether the test1.php having the file execute permission or not.
0
 

Author Comment

by:jporter80
ID: 39902448
i added an echo line to test1.php and yes it is reading the file correctly.. take a look:

echo 'is the file here?';

http://mediaridge.com/test1.php
0
Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

 
LVL 14

Expert Comment

by:shobinsun
ID: 39902482
Hi,

Sometimes a website will block crawlers(from remote servers) from getting to their pages.
What they do to work around this is spoof a browsers headers. Like pretend to be Mozilla Firefox instead of the sneaky PHP web scraper they are. You can use the line below with the code:

$userAgent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13';

curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);

Full code:

<?php
function file_get_contents_curl($url) {
      $userAgent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13';

    $ch = curl_init();
   
    curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
    curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);      

    $data = curl_exec($ch);
    curl_close($ch);
      echo $data;
}
file_get_contents_curl('http://mediaridge.com/script/twitterapp.php');
?>


Try this.

Regards,
Shobin Markose.
0
 

Author Comment

by:jporter80
ID: 39902489
tried that same thing....
0
 
LVL 14

Expert Comment

by:shobinsun
ID: 39902515
hmmmm. Might be some issue with the hosting! Please contact ur hosting provider.

try this also:

file_get_contents_curl('http://mediaridge.com/script/twitterapp.php', FILE_USE_INCLUDE_PATH);
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903098
This is what I find at the URL http://mediaridge.com/test1.php.  Is this what you expect to read with cURL?

is the file here?<HTML>
<HEAD>
<TITLE>404 Not Found</TITLE>
</HEAD>
<BODY>
<H1>Not Found</H1>
The requested document was not found on this server.
<P>
<HR>
<ADDRESS>
Web Server at default1.com
</ADDRESS>
</BODY>
</HTML>

<!--
   - Unfortunately, Microsoft has added a clever new
   - "feature" to Internet Explorer. If the text of
   - an error's message is "too small", specifically
   - less than 512 bytes, Internet Explorer returns
   - its own error message. You can turn that off,
   - but it's pretty tricky to find switch called
   - "smart error messages". That means, of course,
   - that short error messages are censored by default.
   - IIS always returns error messages that are long
   - enough to make Internet Explorer happy. The
   - workaround is pretty simple: pad the error
   - message with a big comment like this to push it
   - over the five hundred and twelve bytes minimum.
   - Of course, that's exactly what you're reading
   - right now.
   -->

Open in new window

0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903107
Please see http://www.iconoun.com/demo/temp_jporter80.php

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/test1.php';

// READ THE DOCUMENT
$htm = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($htm);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

HTH, ~Ray
0
 

Author Comment

by:jporter80
ID: 39903118
No It should be pulling in the Json data from http://mediaridge.com/script/twitterapp.php

From the function above
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903216
Just scratching my head... If what you want is from a URL with twitterapp.php, why would I be thinking I should use a different test page?  Must not have had enough coffee.  Try changing out the URL, maybe?

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/script/twitterapp.php';

// READ THE DOCUMENT
$jso = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($jso);

// MAKE AN OBJECT
$obj = json_decode($jso);
var_dump($obj);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 

Author Comment

by:jporter80
ID: 39903236
Are you able to pull the data using the same function above from your server? I'm wondering if I need to look at my server settings and tweak something (my pho info page is listed above) I run my own DV so I have some control to change things.
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903298
To see what I am able to get from the script I posted above, please see:
http://www.iconoun.com/demo/temp_jporter80.php

Before you start making any server changes, you might try copying that script from the code snippet and installing it on your server.  You may find that it runs and gets the JSON data.  If it does not, then we can look into the reasons why it worked on my server and failed in a different environment.
0
 

Author Comment

by:jporter80
ID: 39903299
looks like a server setting problem.. if i go to a json file outside of the server like this url: http://beerchow.com/craftbeer/data/tijson.php

it works. (beerchow.com is on a diferent DV)

Dont know which setting is causing the issue though... CURL is enabled
0
 

Author Comment

by:jporter80
ID: 39903305
here is your script running on http://mediaridge.com/test2.php .... no dice.
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903314
The beerchow URL works fine for me, too.

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/script/twitterapp.php';
$url = 'http://beerchow.com/craftbeer/data/tijson.php';

// READ THE DOCUMENT
$jso = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($jso);

// MAKE AN OBJECT
$obj = json_decode($jso);
var_dump($obj);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903318
Looks like the script at http://mediaridge.com/test2.php is reading from this URL:
http://mediaridge.com/test1.php

If that's not the URL you want, just change the script to point it to a different URL.
0
 

Author Comment

by:jporter80
ID: 39903325
im getting confused lol.... this is the code on test2.php

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/script/twitterapp.php';

// READ THE DOCUMENT
$jso = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($jso);

// MAKE AN OBJECT
$obj = json_decode($jso);
var_dump($obj);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}
                                            

Open in new window

0
 

Author Comment

by:jporter80
ID: 39903369
i dont know if i know what im talking about or not... but what Port does CURL use when running this script?

I know i have restricted some ports (if i remember correctly) and only allow SFTP access.  This was done a while back to get PCI compliance

Would this have anything? what should i test if this is the case?
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903410
Please see this link, which contains the code that I copied verbatim from the code snippet at ID: 39903325
http://www.iconoun.com/demo/temp_jporter80.php

My guess is that you may be testing the wrong script?
0
 

Author Comment

by:jporter80
ID: 39903426
i have copied the code verbatim also and placed it here:  http://mediaridge.com/test2.php  does not work
0
 

Accepted Solution

by:
jporter80 earned 0 total points
ID: 39903466
Curl outside domains.. works fine... curl using the local domain it doesnt work.  

I just tried curl using my http://IP address/script/twitterapp.php and it WORKED!!

Hmmmm why does it not like to use the domain name?
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903475
Let me give you a new code example.  Copy / paste, then post the link to your copy here.

Here is my copy of the script:
http://www.iconoun.com/demo/temp_jporter80.php

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/script/twitterapp.php';

// CAPTION THE PAGE TO SHOW WHAT URL WE ARE ADDRESSING
echo "<h1>$url</h1>";

// READ THE DOCUMENT
$jso = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($jso);

// MAKE AN OBJECT
$obj = json_decode($jso);
var_dump($obj);

// SHOW THE SCRIPT
highlight_file(__FILE__);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&'
        ;
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 

Author Comment

by:jporter80
ID: 39903495
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903543
Strange.  Have you tried this with a relative link to the page, not using the full URL?  How about file_get_contents() for the same URL?
0
 

Author Comment

by:jporter80
ID: 39903622
relative link doesnt work and cant use file_get_contents()
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 39903650
"Doesn't work" is not an error message, so we can't really help until we know what actually happens.  What happens when you use file_get_contents()?  I could see nothing in the phpinfo() that would indicate it should not work.  Do you have error_reporting(E_ALL) set?  Do you get anything in the error logs?
0
 

Author Closing Comment

by:jporter80
ID: 39998523
Curling using the IP was the only way it worked
0

Featured Post

Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When the s#!t hits the fan, you don’t have time to look up who’s on call, draft emails, call collaborators, or send text messages. An instant chat window is definitely the way to go, especially one like HipChat. HipChat is a true business app. An…
Q&A with Course Creator, Mark Lassoff, on the importance of HTML5 in the career of a modern-day developer.
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.
The is a quite short video tutorial. In this video, I'm going to show you how to create self-host WordPress blog with free hosting service.

628 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question