Solved

Fetch page using curl with cookie

Posted on 2008-06-14
6
772 Views
Last Modified: 2010-04-21
I need to fetch a page with my server that requires a cookie.  If I understand things correctly, I can use curl for this (I have it on my server).
From the example on php.net, I am guessing that I need to use setopt for this?  Can you show me how? I think I use it like:

$ch = curl_init("http://www.example.com/");
$fp = fopen("example_homepage.txt", "r");

//here is where (I think) I would put the cookie in - it needs to say zipcode=23232

curl_setopt();

curl_exec($ch);
curl_close($ch);
fclose($fp);

//display the page (in real life I will be modifying the text/html with str_replace a bit before displaying it.  I'm not stealing anyone's content by the way, just reformatting it to make my daily research a bit easier).

PRINT $fp;


Thanks,  Chris
0
Comment
Question by:St_Aug_Beach_Bum
6 Comments
 
LVL 3

Expert Comment

by:Xorlev
ID: 21787415
This should work:
curl_setopt(CURLOPT_COOKIE, 'zipcode=23232');

Open in new window

0
 
LVL 3

Expert Comment

by:Xorlev
ID: 21787417
My mistake, forgot to add $ch, before CURLOPT_COOKIE
curl_setopt($ch, CURLOPT_COOKIE, 'zipcode=23232');

Open in new window

0
 
LVL 19

Expert Comment

by:bevhost
ID: 21787434
$cookiejar = "/tmp/cookies.txt";
if (!file_exists($cookiejar)) curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiejar);
else curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiejar);

See full example in code snippet
function web($host,$path,$method,$data="",$password="") {
 

        $cookiejar = "/tmp/cookies.txt";

        $query = ""; $sepCh = "?";

        while (list($k, $v) = each($data)) {

          $query .= $sepCh . urlencode($k) . "=" . urlencode($v);

          $sepCh = "&";

        }
 

        $url = "http://".$host.$path;
 

        $answer = "";
 

        switch ($method) {

              case "GET" :

                //GET METHOD

                if ($fp = fopen($url.$query,"r")) {

                        while (!feof($fp)) {

                                $line = fgets($fp,4096);

                                $answer .= $line;

                        }

                }

                break;
 

              case "POST" :

                // POST METHOD

                $port = 80;

                $data = substr($query,1);
 

                // if php version 4.3 or better $port=443; change host to "ssl://".$host;

                $fp = fsockopen($host, $port, $errno, $errstr, $timeout = 30);
 

                if(!$fp){

                  echo "Error: $errstr ($errno)\n";

                }else{

                  fputs($fp, "POST $path HTTP/1.1\r\n");

                  fputs($fp, "Host: $host\r\n");

                  if ($password) fputs($fp, "Authorization: Basic ".base64_encode($password)."\r\n");

                  fputs($fp, "Content-type: application/x-www-form-urlencoded\r\n");

                  fputs($fp, "Content-length: ".strlen($data)."\r\n");

                  fputs($fp, "Connection: close\r\n\r\n");

                  fputs($fp, $data . "\r\n\r\n");
 

                  while (!feof($fp)) {

                        $line = fgets($fp,4096);

                        $answer .= $line;

                  }

                  fclose($fp);

                }

                // END POST METHOD

                break;

              case "CURLPOST" :

                // CURL METHOD

                $url = "https://".$host.$path;

                $data = substr($query,1);

                if (!$ch = curl_init()) {

                    echo "Could not initialize cURL session.\n";

                    exit;

                }

                curl_setopt($ch, CURLOPT_URL, $url);

                //if (!file_exists($cookiejar)) curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiejar);

                //else curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiejar);

                curl_setopt($ch, CURLOPT_POST, 0);

                curl_setopt($ch, CURLOPT_POSTFIELDSIZE, 0);

                curl_setopt($ch, CURLOPT_POSTFIELDS, $data);

                curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

                curl_setopt($ch, CURLOPT_HEADER, 0);

                curl_setopt($ch, CURLOPT_TIMEOUT, 60);

                //curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)');

                //if ($previousUrl) curl_setopt($ch, CURLOPT_REFERER, $previousUrl);

                $output = curl_exec($ch);

                curl_close($ch);

                if($output == ''){

                   echo "cURL did not receive a response back.\n";

                   mail('hosting@bevhost.com','IPN-CURL','cURL did not receive a response back.');

                   exit;

                }

                $answer = preg_replace("'Content-type: text/plain'si","",$output);

                $error_lines = split("\n", $error_message);

                $i=0;

                while($i <= sizeof($error_lines)) {            

                  $error_message_html .= "<p>" .$error_lines[$i];

                  $i++;

                }

                // END CURL METHOD

                break;

             case "CURLGET" :

                // CURL METHOD

                $url = "http://".$host.$path.$query;

                if (!$ch = curl_init()) {

                    echo "Could not initialize cURL session.\n";

                    exit;

                }

                curl_setopt($ch, CURLOPT_URL, $url);

                if (!file_exists($cookiejar)) curl_setopt($ch, CURLOPT_COOKIEJAR, $cookiejar);

                else curl_setopt($ch, CURLOPT_COOKIEFILE, $cookiejar);

                curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

                curl_setopt($ch, CURLOPT_HEADER, 1);

                curl_setopt($ch, CURLOPT_TIMEOUT, 60);

                curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)');

                if ($previousUrl) curl_setopt($ch, CURLOPT_REFERER, $previousUrl);

                $output = curl_exec($ch);

                curl_close($ch);

                if($output == ''){

                   echo "cURL did not receive a response back.\n";

                   exit;

                }

                $answer = preg_replace("'Content-type: text/plain'si","",$output);

                $error_lines = split("\n", $error_message);

                $i=0;

                while($i <= sizeof($error_lines)) {

                  $error_message_html .= "<p>" .$error_lines[$i];

                  $i++;

                }

                // END CURL METHOD

                break;

        } //switch method
 

        $previousUrl = $url;
 

        return $answer;

}

Open in new window

0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 82

Accepted Solution

by:
hielo earned 500 total points
ID: 21787437
I'm not sure what's that fopen for. If you are trying to retrieve http://www.example.com/example_hompage.txt, then you do not need fopen. To retrieve the page you need either curl OR fopen, but not both. However, since you need to pass cookies, you cannot use fopen. Try:

<?
$string = 'cookie=atestcookie'; 
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.example.com/example_homepage.txt");
curl_setopt($ch, CURLOPT_COOKIE, $string);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$text = curl_exec($ch);
curl_close($ch);
echo($text);
?> 

Open in new window

0
 

Author Closing Comment

by:St_Aug_Beach_Bum
ID: 31467293
Thank you all.  Overwhelmed a bit by several answers, but the most through understandable, straightening me out answer was from Helio. Appreciate the help from everyone though!   Chris
0
 

Author Comment

by:St_Aug_Beach_Bum
ID: 21787486
opps, sorry, that should be Hielo  !
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

Introduction HTML checkboxes provide the perfect way for a web developer to receive client input when the client's options might be none, one or many.  But the PHP code for processing the checkboxes can be confusing at first.  What if a checkbox is…
Foreword (July, 2015) Since I first wrote this article, years ago, a great many more people have begun using the internet.  They are coming online from every part of the globe, learning, reading, shopping and spending money at an ever-increasing ra…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now