Solved

PHP CURL and ASPX page

Posted on 2011-04-26
6
1,452 Views
Last Modified: 2012-05-11
Hi,

I've spent quite a bit of time trying to get this to work, but so far to no avail.

I'm attempting to request information from a specific .aspx page.

Using firefox and firebug, I am able to determine the actual url and parameters and if I paste into one of several browsers, the result I am looking is returned satisfactorily:

http://widgets.ghin.com/HandicapLookupResults.aspx?entry=1&dynamic=&small=0&css=default&ghinno=9675575&hidename=0&showmsg=0&showheader=1&showtabheader=0&combinehieff=0&showheadertext=0&showfootertext=0&tab=2

I found I can even shorten this (I assume the other parameters are defaulted) and it works just as well:

http://widgets.ghin.com/HandicapLookupResults.aspx?ghinno=9675575

Now I have read something about the requirement to use VIEWSTATE, which I as understand would be the state from the previous form.  However, as you can see by using the above url's it seems to work fine without going through the form.  However, if it helps anyone understand this better this would be the url to the form:

http://ghin.com/lookup.aspx

and my inserting the number 9675575 in the field you can achieve the same results as above.

I have tried coding this up in a variety if ways without success.  see attached.

I would appreciate any help plus how the solution was thought out.

Thanks


 
<?php
header("Content-type: text/html");


$doc_root_x = $_SERVER['DOCUMENT_ROOT'];



$url = "http://widgets.ghin.com/HandicapLookupResults.aspx";

$entry          = 1;
$dynamic        = "";
$small          = 0;
$css            = "default";
$ghinno         = 9675575;
$hidename       = 0;
$showmsg        = 0;
$showheader     = 1;
$showtabheader  = 0;
$combinehieff   = 0;
$showheadertext = 1;
$showfootertext = 1;
$tab            = 2;

$fields = array(
                        'entry'         =>urlencode($entry),
                        'dynamic'       =>urlencode($dynamic),
                        'small'         =>urlencode($small),
                        'css'           =>urlencode($css),
                        'ghinno'        =>urlencode($ghinno),
                        'hidename'      =>urlencode($hidename),
                        'showmsg'       =>urlencode($showmsg),
                        'showheader'    =>urlencode($showheader),
                        'showtabheader' =>urlencode($showtabheader),
                        'combinehieff'  =>urlencode($combinehieff),
                        'showheadertext'=>urlencode($showheadertext),
                        'showfootertext'=>urlencode($showfootertext),
                        'tab'           =>urlencode($tab)
                );

$fields = array(
                        'ghinno'        =>urlencode($ghinno)
                );                
                
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
$fields_string = rtrim($fields_string,"&");


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);

//$useragent = $_SERVER["HTTP_USER_AGENT"];
$useragent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2";
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

curl_setopt($ch, CURLOPT_POST, count($fields));
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields_string);

$cookie = $doc_root_x . "/joomla/tmp/ghinCookie.txt";
$fp = fopen("$cookie","w") or die("<BR />Unable to open cookie file $cookie for write!<BR />");
fclose($fp);

curl_setopt($ch, CURLOPT_COOKIEJAR,  $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);

$result = trim(curl_exec($ch));
$status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
         
curl_close($ch);

echo($result) ;

// possible resutls:
// GHIN Number Invalid


?>

Open in new window

0
Comment
Question by:BeerFizz
  • 3
  • 2
6 Comments
 
LVL 1

Expert Comment

by:AnuarJamlus
ID: 35468216
If the url doesn't require a post and the page can be retrieved correctly by calling up the full url with parameters, I would think that would be the best way to do it rather than doing a CURL post

 
0
 

Author Comment

by:BeerFizz
ID: 35468331

Perhaps, but I need to be able to scrape the page and using curl will allow me to have the result returned to a variable, where I may process the contents.
0
 

Author Comment

by:BeerFizz
ID: 35468887

Any other ideas??
0
DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

 
LVL 108

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 35471844
See http://www.laprbass.com/RAY_temp_beerfizz.php

HTH, ~Ray
<?php // RAY_temp_beerfizz.php
error_reporting(E_ALL);

// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = '';
    foreach ($get_array as $key => $val)
    {
        $get_string .= urlencode($key) . '=' . urlencode($val) . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}




// USAGE EXAMPLE - PUT YOUR FAVORITE URL HERE
$url = "http://widgets.ghin.com/HandicapLookupResults.aspx";

// PUT YOUR ARRAY OF KEY=>VALUE PAIRS HERE
$arg = array( 'ghinno' => '9675575' );

// MAKE THE CALL
$htm = my_curl($url, $arg, 2, TRUE);
if (!$htm) die("NO $url");

// SHOW WHAT WE GOT
echo "<pre>";
var_dump($arg);
echo PHP_EOL . $url;
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;

Open in new window

0
 

Author Closing Comment

by:BeerFizz
ID: 35471902
Ray,

thank you so much.  It works perfectly.

Phil
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 35471927
Thanks for the points - it's a great question, and having several URLs to choose from made it easier to get good results.  Cheers, ~Ray
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
Developer portfolios can be a bit of an enigma—how do you present yourself to employers without burying them in lines of code?  A modern portfolio is more than just work samples, it’s also a statement of how you work.
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
Any person in technology especially those working for big companies should at least know about the basics of web accessibility. Believe it or not there are even laws in place that require businesses to provide such means for the disabled and aging p…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now