PHP CURL and ASPX page

Hi,

I've spent quite a bit of time trying to get this to work, but so far to no avail.

I'm attempting to request information from a specific .aspx page.

Using firefox and firebug, I am able to determine the actual url and parameters and if I paste into one of several browsers, the result I am looking is returned satisfactorily:

http://widgets.ghin.com/HandicapLookupResults.aspx?entry=1&dynamic=&small=0&css=default&ghinno=9675575&hidename=0&showmsg=0&showheader=1&showtabheader=0&combinehieff=0&showheadertext=0&showfootertext=0&tab=2

I found I can even shorten this (I assume the other parameters are defaulted) and it works just as well:

http://widgets.ghin.com/HandicapLookupResults.aspx?ghinno=9675575

Now I have read something about the requirement to use VIEWSTATE, which I as understand would be the state from the previous form.  However, as you can see by using the above url's it seems to work fine without going through the form.  However, if it helps anyone understand this better this would be the url to the form:

http://ghin.com/lookup.aspx

and my inserting the number 9675575 in the field you can achieve the same results as above.

I have tried coding this up in a variety if ways without success.  see attached.

I would appreciate any help plus how the solution was thought out.

Thanks


 
<?php
header("Content-type: text/html");


$doc_root_x = $_SERVER['DOCUMENT_ROOT'];



$url = "http://widgets.ghin.com/HandicapLookupResults.aspx";

$entry          = 1;
$dynamic        = "";
$small          = 0;
$css            = "default";
$ghinno         = 9675575;
$hidename       = 0;
$showmsg        = 0;
$showheader     = 1;
$showtabheader  = 0;
$combinehieff   = 0;
$showheadertext = 1;
$showfootertext = 1;
$tab            = 2;

$fields = array(
                        'entry'         =>urlencode($entry),
                        'dynamic'       =>urlencode($dynamic),
                        'small'         =>urlencode($small),
                        'css'           =>urlencode($css),
                        'ghinno'        =>urlencode($ghinno),
                        'hidename'      =>urlencode($hidename),
                        'showmsg'       =>urlencode($showmsg),
                        'showheader'    =>urlencode($showheader),
                        'showtabheader' =>urlencode($showtabheader),
                        'combinehieff'  =>urlencode($combinehieff),
                        'showheadertext'=>urlencode($showheadertext),
                        'showfootertext'=>urlencode($showfootertext),
                        'tab'           =>urlencode($tab)
                );

$fields = array(
                        'ghinno'        =>urlencode($ghinno)
                );                
                
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
$fields_string = rtrim($fields_string,"&");


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);

//$useragent = $_SERVER["HTTP_USER_AGENT"];
$useragent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2";
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

curl_setopt($ch, CURLOPT_POST, count($fields));
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields_string);

$cookie = $doc_root_x . "/joomla/tmp/ghinCookie.txt";
$fp = fopen("$cookie","w") or die("<BR />Unable to open cookie file $cookie for write!<BR />");
fclose($fp);

curl_setopt($ch, CURLOPT_COOKIEJAR,  $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);

$result = trim(curl_exec($ch));
$status = curl_getinfo($ch, CURLINFO_HTTP_CODE);
         
curl_close($ch);

echo($result) ;

// possible resutls:
// GHIN Number Invalid


?>

Open in new window

BeerFizzAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

AnuarJamlusCommented:
If the url doesn't require a post and the page can be retrieved correctly by calling up the full url with parameters, I would think that would be the best way to do it rather than doing a CURL post

 
BeerFizzAuthor Commented:

Perhaps, but I need to be able to scrape the page and using curl will allow me to have the result returned to a variable, where I may process the contents.
BeerFizzAuthor Commented:

Any other ideas??
Exploring SQL Server 2016: Fundamentals

Learn the fundamentals of Microsoft SQL Server, a relational database management system that stores and retrieves data when requested by other software applications.

Ray PaseurCommented:
See http://www.laprbass.com/RAY_temp_beerfizz.php

HTH, ~Ray
<?php // RAY_temp_beerfizz.php
error_reporting(E_ALL);

// A FUNCTION TO RUN A CURL-GET CLIENT CALL TO A FOREIGN SERVER
function my_curl
( $url
, $get_array=array()
, $timeout=3
, $error_report=TRUE
)
{
    // PREPARE THE ARGUMENT STRING IF NEEDED
    $get_string = '';
    foreach ($get_array as $key => $val)
    {
        $get_string .= urlencode($key) . '=' . urlencode($val) . '&';
    }
    $get_string = rtrim($get_string, '&');
    if (!empty($get_string)) $url .= '?' . $get_string;

    $curl = curl_init();

    // HEADERS AND OPTIONS APPEAR TO BE A FIREFOX BROWSER REFERRED BY GOOGLE
    $header[] = "Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
    $header[] = "Cache-Control: max-age=0";
    $header[] = "Connection: keep-alive";
    $header[] = "Keep-Alive: 300";
    $header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
    $header[] = "Accept-Language: en-us,en;q=0.5";
    $header[] = "Pragma: "; // BROWSERS USUALLY LEAVE BLANK

    // SET THE CURL OPTIONS - SEE http://php.net/manual/en/function.curl-setopt.php
    curl_setopt( $curl, CURLOPT_URL,            $url  );
    curl_setopt( $curl, CURLOPT_USERAGENT,      'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'  );
    curl_setopt( $curl, CURLOPT_HTTPHEADER,     $header  );
    curl_setopt( $curl, CURLOPT_REFERER,        'http://www.google.com'  );
    curl_setopt( $curl, CURLOPT_ENCODING,       'gzip,deflate'  );
    curl_setopt( $curl, CURLOPT_AUTOREFERER,    TRUE  );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE  );
    curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, TRUE  );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout  );

    // RUN THE CURL REQUEST AND GET THE RESULTS
    $htm = curl_exec($curl);

    // ON FAILURE HANDLE ERROR MESSAGE
    if ($htm === FALSE)
    {
        if ($error_report)
        {
            $err = curl_errno($curl);
            $inf = curl_getinfo($curl);
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            var_dump($inf);
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS RETURN XML / HTML STRING
    curl_close($curl);
    return $htm;
}




// USAGE EXAMPLE - PUT YOUR FAVORITE URL HERE
$url = "http://widgets.ghin.com/HandicapLookupResults.aspx";

// PUT YOUR ARRAY OF KEY=>VALUE PAIRS HERE
$arg = array( 'ghinno' => '9675575' );

// MAKE THE CALL
$htm = my_curl($url, $arg, 2, TRUE);
if (!$htm) die("NO $url");

// SHOW WHAT WE GOT
echo "<pre>";
var_dump($arg);
echo PHP_EOL . $url;
echo PHP_EOL . htmlentities($htm);
echo PHP_EOL;

Open in new window

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
BeerFizzAuthor Commented:
Ray,

thank you so much.  It works perfectly.

Phil
Ray PaseurCommented:
Thanks for the points - it's a great question, and having several URLs to choose from made it easier to get good results.  Cheers, ~Ray
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
PHP

From novice to tech pro — start learning today.