Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Using CURL to get File Contents returns 404 File not found

Posted on 2014-03-03
27
Medium Priority
?
5,821 Views
Last Modified: 2014-04-14
Here is the script

<?php 
function file_get_contents_curl($url) {
    $ch = curl_init();

    curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);       

    $data = curl_exec($ch);
    curl_close($ch);
	echo $data;
}
file_get_contents_curl('http://mediaridge.com/script/twitterapp.php');
?>

Open in new window


but yet it returns 404 file not found... here is a test page:

http://mediaridge.com/test1.php


PHP Info for reference http://mediaridge.com/test.php
0
Comment
Question by:jporter80
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 13
  • 10
  • 4
27 Comments
 
LVL 14

Expert Comment

by:shobinsun
ID: 39902439
Hi,

Your code is fine and I am able to fetch the json data. did you check the file: http://mediaridge.com/test1.php is there in the server?? It looks like the test1.php file is not there in the server.

Regards,
Shobin Markose.
0
 
LVL 14

Expert Comment

by:shobinsun
ID: 39902441
Hi,

Also check whether the test1.php having the file execute permission or not.
0
 

Author Comment

by:jporter80
ID: 39902448
i added an echo line to test1.php and yes it is reading the file correctly.. take a look:

echo 'is the file here?';

http://mediaridge.com/test1.php
0
Plesk WordPress Toolkit

Plesk's WordPress Toolkit allows server administrators, resellers and customers to manage their WordPress instances, enabling a variety of development workflows for WordPress admins of all skill levels, from beginners to pros.

See why 2/3 of Plesk servers use it.

 
LVL 14

Expert Comment

by:shobinsun
ID: 39902482
Hi,

Sometimes a website will block crawlers(from remote servers) from getting to their pages.
What they do to work around this is spoof a browsers headers. Like pretend to be Mozilla Firefox instead of the sneaky PHP web scraper they are. You can use the line below with the code:

$userAgent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13';

curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);

Full code:

<?php
function file_get_contents_curl($url) {
      $userAgent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13';

    $ch = curl_init();
   
    curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
    curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);      

    $data = curl_exec($ch);
    curl_close($ch);
      echo $data;
}
file_get_contents_curl('http://mediaridge.com/script/twitterapp.php');
?>


Try this.

Regards,
Shobin Markose.
0
 

Author Comment

by:jporter80
ID: 39902489
tried that same thing....
0
 
LVL 14

Expert Comment

by:shobinsun
ID: 39902515
hmmmm. Might be some issue with the hosting! Please contact ur hosting provider.

try this also:

file_get_contents_curl('http://mediaridge.com/script/twitterapp.php', FILE_USE_INCLUDE_PATH);
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903098
This is what I find at the URL http://mediaridge.com/test1.php.  Is this what you expect to read with cURL?

is the file here?<HTML>
<HEAD>
<TITLE>404 Not Found</TITLE>
</HEAD>
<BODY>
<H1>Not Found</H1>
The requested document was not found on this server.
<P>
<HR>
<ADDRESS>
Web Server at default1.com
</ADDRESS>
</BODY>
</HTML>

<!--
   - Unfortunately, Microsoft has added a clever new
   - "feature" to Internet Explorer. If the text of
   - an error's message is "too small", specifically
   - less than 512 bytes, Internet Explorer returns
   - its own error message. You can turn that off,
   - but it's pretty tricky to find switch called
   - "smart error messages". That means, of course,
   - that short error messages are censored by default.
   - IIS always returns error messages that are long
   - enough to make Internet Explorer happy. The
   - workaround is pretty simple: pad the error
   - message with a big comment like this to push it
   - over the five hundred and twelve bytes minimum.
   - Of course, that's exactly what you're reading
   - right now.
   -->

Open in new window

0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903107
Please see http://www.iconoun.com/demo/temp_jporter80.php

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/test1.php';

// READ THE DOCUMENT
$htm = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($htm);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

HTH, ~Ray
0
 

Author Comment

by:jporter80
ID: 39903118
No It should be pulling in the Json data from http://mediaridge.com/script/twitterapp.php

From the function above
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903216
Just scratching my head... If what you want is from a URL with twitterapp.php, why would I be thinking I should use a different test page?  Must not have had enough coffee.  Try changing out the URL, maybe?

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/script/twitterapp.php';

// READ THE DOCUMENT
$jso = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($jso);

// MAKE AN OBJECT
$obj = json_decode($jso);
var_dump($obj);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 

Author Comment

by:jporter80
ID: 39903236
Are you able to pull the data using the same function above from your server? I'm wondering if I need to look at my server settings and tweak something (my pho info page is listed above) I run my own DV so I have some control to change things.
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903298
To see what I am able to get from the script I posted above, please see:
http://www.iconoun.com/demo/temp_jporter80.php

Before you start making any server changes, you might try copying that script from the code snippet and installing it on your server.  You may find that it runs and gets the JSON data.  If it does not, then we can look into the reasons why it worked on my server and failed in a different environment.
0
 

Author Comment

by:jporter80
ID: 39903299
looks like a server setting problem.. if i go to a json file outside of the server like this url: http://beerchow.com/craftbeer/data/tijson.php

it works. (beerchow.com is on a diferent DV)

Dont know which setting is causing the issue though... CURL is enabled
0
 

Author Comment

by:jporter80
ID: 39903305
here is your script running on http://mediaridge.com/test2.php .... no dice.
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903314
The beerchow URL works fine for me, too.

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/script/twitterapp.php';
$url = 'http://beerchow.com/craftbeer/data/tijson.php';

// READ THE DOCUMENT
$jso = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($jso);

// MAKE AN OBJECT
$obj = json_decode($jso);
var_dump($obj);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903318
Looks like the script at http://mediaridge.com/test2.php is reading from this URL:
http://mediaridge.com/test1.php

If that's not the URL you want, just change the script to point it to a different URL.
0
 

Author Comment

by:jporter80
ID: 39903325
im getting confused lol.... this is the code on test2.php

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/script/twitterapp.php';

// READ THE DOCUMENT
$jso = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($jso);

// MAKE AN OBJECT
$obj = json_decode($jso);
var_dump($obj);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}
                                            

Open in new window

0
 

Author Comment

by:jporter80
ID: 39903369
i dont know if i know what im talking about or not... but what Port does CURL use when running this script?

I know i have restricted some ports (if i remember correctly) and only allow SFTP access.  This was done a while back to get PCI compliance

Would this have anything? what should i test if this is the case?
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903410
Please see this link, which contains the code that I copied verbatim from the code snippet at ID: 39903325
http://www.iconoun.com/demo/temp_jporter80.php

My guess is that you may be testing the wrong script?
0
 

Author Comment

by:jporter80
ID: 39903426
i have copied the code verbatim also and placed it here:  http://mediaridge.com/test2.php  does not work
0
 

Accepted Solution

by:
jporter80 earned 0 total points
ID: 39903466
Curl outside domains.. works fine... curl using the local domain it doesnt work.  

I just tried curl using my http://IP address/script/twitterapp.php and it WORKED!!

Hmmmm why does it not like to use the domain name?
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903475
Let me give you a new code example.  Copy / paste, then post the link to your copy here.

Here is my copy of the script:
http://www.iconoun.com/demo/temp_jporter80.php

<?php // demo/temp_jporter80.php
error_reporting(E_ALL);


// SEE http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28379490.html


// FILE LINK FROM THE POST AT EE
$url = 'http://mediaridge.com/script/twitterapp.php';

// CAPTION THE PAGE TO SHOW WHAT URL WE ARE ADDRESSING
echo "<h1>$url</h1>";

// READ THE DOCUMENT
$jso = my_curl($url);

// SHOW WHAT WE GOT
echo '<pre>';
echo PHP_EOL;
echo htmlentities($jso);

// MAKE AN OBJECT
$obj = json_decode($jso);
var_dump($obj);

// SHOW THE SCRIPT
highlight_file(__FILE__);


// A FUNCTION TO RUN CURL WITH ERROR HANDLING
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = NULL;
    foreach ($get_array as $key => $val)
    {
        $get_string
        = $get_string
        . $key
        . '='
        . urlencode($val)
        . '&'
        ;
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    // START CURL
    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE THIS BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // THIS SEEMS TO LET IT WORK WITH HTTPS SITES
    curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, FALSE );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE CREATION OF ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}

Open in new window

0
 

Author Comment

by:jporter80
ID: 39903495
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903543
Strange.  Have you tried this with a relative link to the page, not using the full URL?  How about file_get_contents() for the same URL?
0
 

Author Comment

by:jporter80
ID: 39903622
relative link doesnt work and cant use file_get_contents()
0
 
LVL 111

Expert Comment

by:Ray Paseur
ID: 39903650
"Doesn't work" is not an error message, so we can't really help until we know what actually happens.  What happens when you use file_get_contents()?  I could see nothing in the phpinfo() that would indicate it should not work.  Do you have error_reporting(E_ALL) set?  Do you get anything in the error logs?
0
 

Author Closing Comment

by:jporter80
ID: 39998523
Curling using the IP was the only way it worked
0

Featured Post

U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Dramatic changes are revolutionizing how we build and use technology. Every company is automating, digitizing, and modernizing operations. We need a better, more connected way to work together as teams so we can harness the insights from our system…
Q&A with Course Creator, Mark Lassoff, on the importance of HTML5 in the career of a modern-day developer.
The viewer will learn how to dynamically set the form action using jQuery.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question