Solved

POSTing data to CGI using PHP

Posted on 2004-09-08
29
389 Views
Last Modified: 2007-12-19
Hi all, Ok, I've looked through the available answers and noone seems to answer this question nicely...

I want my PHP script to access data served by a 2nd server. Normally to access this data on server B (which I don't control) I have to fill in a form. What I want is for my script to simulate the form and POST the data to the CGI on server B, then parse the returned page for the data I want. I think I'll also have to change the http-referrer so the CGI on "B" thinks I'm using the form.

A brief explanation:

I just signed p for Barclays on-line banking and want my script to be able to read my current account balance and report it on my homepage. I'm aware that I need to be ultra-careful security-wise. All communications take place over SSL secured connections.


The data used to show the form in the 1st place:

https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,logon,00.html?newMember=true

<form method="POST" action="/fp/1_2c/online/1,26806,,00.html">
<input type=hidden name="action" value="Submit Membership Number">
<input type=hidden name="servlet" value="startlogin">
<input type=hidden name="screenName" value="logonMember1i">
<input type="text" size=25  maxlength=24 name="surname" title="Surname">
<input type="text" size=20 maxlength=24 name="membershipNo" title="Membership Number (last 8 digits)">
<input type=image src="newgreennext.gif" border=0 width=25 height=42 alt="Next" name="Next">
</form>

so I need to post the above data to https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,,00.html

Any suggestions greatly appreciated.

I did wonder about the right & wrong issue of accessing the server this way, but as I can still only log-in with the correct information, I don't see this as being a problem
0
Comment
Question by:basiclife
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 16
  • 13
29 Comments
 
LVL 36

Accepted Solution

by:
Zyloch earned 500 total points
ID: 12012522
Hi

First, you'd have to download PHP cURL, see this page: http://be2.php.net/curl

Then this script should be able to work: (I only know the first Step)

<?php

//Set page to needed page first.
$ch=curl_init();

curl_setopt ($ch, CURLOPT_URL, "https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,logon,00.html?newMember=true");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_exec($ch);
curl_close($ch);

$ch=curl_init();
/*rememberDetails could be false*/
curl_setopt($ch, CURLOPT_URL, "https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,,00.html");
curl_setopt($ch, CURLOPT_REFERER, "https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,logon,00.html?newMember=true");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,"surname=yourSurname&membershipNo=selfExplanatory&rememberDetails=true");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$res=curl_exec($ch);
curl_close($ch);

echo($res);

?>

Regards,
Zyloch
0
 
LVL 5

Author Comment

by:basiclife
ID: 12012568
Will DL and play as you suggested. Is there not a simpler way?

Also, would you mind giving me a V. brief explanation of what CURL is / What it's doing here ? Thanks
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12012584
I don't think so. I know in Perl you can use LWP::UserAgent but that's Perl and this is PHP. PHP is primarily used to design webpages and not to do stuff to them, so cURL you can say is actually kind of a blessing. cURL is just a bunch of functions that extend PHP, for instance, being able to modify the Referal header.

This provides quite a basic overview of cURL but should show what it is--you can also visit the cURL site on there. (since you need to download the cURL package anyways)
0
Why Off-Site Backups Are The Only Way To Go

You are probably backing up your data—but how and where? Ransomware is on the rise and there are variants that specifically target backups. Read on to discover why off-site is the way to go.

 
LVL 5

Author Comment

by:basiclife
ID: 12012672
Thanks. Am installing curl on the server now (This is going to be fun said the linux noob) shouldn't be that much of a problem really. As I said, I'll get back to you. Thanks for your quick response
0
 
LVL 5

Author Comment

by:basiclife
ID: 12022770
OK, hiccup 1:

I have installed CURL etc... and tested the code you posted. Problem is, The barclays server returns the main page rather than the 3rd login page (1: Main page, 2: enter details  your specified above, 3rd: prompt for passwords, etc...)

Suggestions? Do you think this counts as a seperate question?
0
 
LVL 5

Author Comment

by:basiclife
ID: 12022785
Meh. Ok, I'm going to award you the marks and open a new Q. I think

Thanks for the feedback, much appreciated
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12022789
Nah, if it's all related, it's all one question, and it's all good. Let's see...

The thing is, I can't login so I can't see the HTML code for the second part, the security thing. I'm not totally sure what you mean by returning the main page instead of the 3rd, so you'll have to be a bit more elaborative, if you don't mind.
0
 
LVL 5

Author Comment

by:basiclife
ID: 12022826
Ok, If you don't min playing a bit more,


When you go to http://ibank.barclays.co.uk you see the "Welcome to online banking........blah" page. Unfortunately, your code returns that page instead of being further along the login process. Your code has been uploaded to:

http://bits.bris.ac.uk/basiclife/test.php

so you can see the output (obv. the graphics die as the URLs are relative)

http://bits.bris.ac.uk/basiclife/test2.php shows the code for test.php which will also show you a member number / surname without me posting it here. Needless to say, I'm not going to poste my passwords / pin in any way but the membership number is public so shouldn't be a problem

If you can solve this, I'll throw some more points at you :D Thanks for your help
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12022875
The Login Process seems to be a two step process. Can you show me the HTML source code of the second login step?
0
 
LVL 5

Author Comment

by:basiclife
ID: 12022940
Ok, Pages as they go:

1) Click "login"

2) complete form:
https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,logon,00.html?newMember=true

<form method="POST" action="/fp/1_2c/online/1,26806,,00.html">
<input type=hidden name="action" value="Submit Membership Number">
<input type=hidden name="servlet" value="startlogin">
<input type=hidden name="screenName" value="logonMember1i">
<input type="text" size=25  maxlength=24 name="surname" class="formFont" title="Surname">
<input type="text" size=20 maxlength=24 name="membershipNo" class="formFont" title="Membership Number (last 8 digits)">
<input type=image src="/fp/1_2c/images/buttons/newgreennext.gif" border=0 width=25 height=42 alt="Next" name="Next">


3) Page complete form:
https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,,00.html


"<td class="bodytext" width="200">Letter" //Code to search for to find out which letters the server requires
"<td class="bodytext" width="200">Letter" //from my password to login


<form method="POST" action="/fp/1_2c/online/1,26806,,00.html" enctype="multipart/form-data">
<input type=hidden name=startTime value="1094692591638"> ************* I'll need to update this
<input type=hidden name=colourType value="">
<input type=hidden name=issued value="1094692591644"> ************* I'll need to update this
<input type=hidden name=sequence value="0">
<input type=hidden name="rememberDetails" value="false">
<input type=hidden name="membershipDetails" value="">
<input type=hidden name=servlet value="login">
<input type=hidden name=usec value="true">
<input type=hidden name="action" value="Submit Passcode">
<select name="firstMDC" class="bodytext">
<select name="secondMDC" class="bodytext">
<input type=image src="/fp/1_2c/images/buttons/newgreenlogin.gif" border=0 width=35 height=39 alt="Log-in" name="Log-in">


By looking at the code I listed in test2.php You'll see my membership number/surname so you can get to the 2nd page yourself. Needless to say, there's lots of superflous HTML. Main body of step 3 looks like:

Log-in Step 2 of 2  
         Security Check
 
Please enter your security details below  
 
 
Five-digit passcode  
     <PIN here>

Please use the drop-down menus to input:
Letter 1 of your memorable word    < a b c d e f g h i j k l m n o p q r s t u v w x y z > <--Choose
Letter 2 of your memorable word    < a b c d e f g h i j k l m n o p q r s t u v w x y z > <--Choose

Forgotten your log-in details?  
 
You can change your passcode and memorable word once you've logged in by selecting 'Customise My Site'.
 
Select the green 'log-in' button to continue.
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12022973
Wait, I'm a little confused about the security code. What is the name of the form field for that and of which form is it of?
0
 
LVL 5

Author Comment

by:basiclife
ID: 12022998
Nice split screen at: http://bits.bris.ac.uk/basiclife/test.html

Damn. Sorry. Missed out the important bit:
<input type="password" size=10 maxlength=5 name="passCode" class="formFont" title="Please enter your 5 digit passcode">

The 2 random letters from password are the select statements

Unedited they look like:

                                    <select name="firstMDC" class="bodytext">
                                          <option value="">&nbsp;
                                          <option value="a">&nbsp;a
                                          <option value="b">&nbsp;b
                                          <option value="c">&nbsp;c
                                          <option value="d">&nbsp;d
                                          <option value="e">&nbsp;e
                                          <option value="f">&nbsp;f
                                          <option value="g">&nbsp;g
                                          <option value="h">&nbsp;h
                                          <option value="i">&nbsp;i
                                          <option value="j">&nbsp;j
                                          <option value="k">&nbsp;k
                                          <option value="l">&nbsp;l
                                          <option value="m">&nbsp;m
                                          <option value="n">&nbsp;n
                                          <option value="o">&nbsp;o
                                          <option value="p">&nbsp;p
                                          <option value="q">&nbsp;q
                                          <option value="r">&nbsp;r
                                          <option value="s">&nbsp;s
                                          <option value="t">&nbsp;t
                                          <option value="u">&nbsp;u
                                          <option value="v">&nbsp;v
                                          <option value="w">&nbsp;w
                                          <option value="x">&nbsp;x
                                          <option value="y">&nbsp;y
                                          <option value="z">&nbsp;z
                                    </select>
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12023043
And all of this is under one form, the one above? What are the two random letters for? Do you know? (Sorry about these questions, but you know, I have to know what's going on)
0
 
LVL 5

Author Comment

by:basiclife
ID: 12023154
Ok  the way it works ( as far as I can tell:

Form 1: Get someone's surname and membership number

Form 2: prompt for 5 Digit PIN and 2 letters from password chosen at random.

Also, a LOT of hidden fields.

If you read through my post "Comment from basiclife Date: 09/10/2004 02:14AM BST" you'll see the form data stripped from the HTML and the URL at which the page can be found (with the exception that the 2nd form, listed as step 3, has the following field which I missed:

<input type="password" size=10 maxlength=5 name="passCode" class="formFont" title="Please enter your 5 digit passcode">
)
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12023350
I would modify this, but I keep thinking I'm missing something. Try this and see if anything changes (not full login, but partial):

<?php
$surname="yourSurname";
$membershipNo="selfExplanatory";
$rememberDetails="true"; //As a string ;)

$cpath="auto\cookies\cook";  //SET YOUR COOKIE PATH

//Set page to needed page first.
$ch=curl_init();

curl_setopt ($ch, CURLOPT_URL, "https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,logon,00.html?newMember=true");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cpath);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cpath);
curl_exec($ch);
curl_close($ch);

$ch=curl_init();
/*rememberDetails could be false*/
curl_setopt($ch, CURLOPT_URL, "https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,,00.html");
curl_setopt($ch, CURLOPT_REFERER, "https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,logon,00.html?newMember=true");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,"surname=".$surname."&membershipNo=".$membershipNo."&rememberDetails=".$rememberDetails);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cpath);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cpath);
$res=curl_exec($ch);
curl_close($ch);

echo($res);

?>
0
 
LVL 5

Author Comment

by:basiclife
ID: 12032204
Unfortunately, same result:

See http://bits.bris.ac.uk/basiclife/test.html
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12033181
Hm... Well try this for debugging:

Delete echo($res);

Place this right before each curl_close($ch) [there's two in code above]
print_r(curl_getinfo($ch));

Post what it prints out.
0
 
LVL 5

Author Comment

by:basiclife
ID: 12033846
Array
(
    [url] => https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,logon,00.html?newMember=true
    [content_type] => text/html
    [http_code] => 200
    [header_size] => 397
    [request_size] => 128
    [filetime] => -1
    [ssl_verify_result] => 0
    [redirect_count] => 0
    [total_time] => 3.602312
    [namelookup_time] => 0.004944
    [connect_time] => 0.024542
    [pretransfer_time] => 0.119753
    [size_upload] => 0
    [size_download] => 9922
    [speed_download] => 2754
    [speed_upload] => 0
    [download_content_length] => 0
    [upload_content_length] => 0
    [starttransfer_time] => 0.395667
    [redirect_time] => 0
)
Array
(
    [url] => https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,Welcome,00.html
    [content_type] => text/html
    [http_code] => 200
    [header_size] => 721
    [request_size] => 654
    [filetime] => -1
    [ssl_verify_result] => 0
    [redirect_count] => 1
    [total_time] => 2.391814
    [namelookup_time] => 0.002702
    [connect_time] => 0.019309
    [pretransfer_time] => 0.120552
    [size_upload] => 0
    [size_download] => 11966
    [speed_download] => 5002
    [speed_upload] => 0
    [download_content_length] => 0
    [upload_content_length] => 0
    [starttransfer_time] => 2.152247
    [redirect_time] => 0.416667
)


Seems the 1st link isn't SSL verified. Does that mean it's trying without SSL and getting rebuffed?
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12034514
The first one shouldn't matter, it's just connecting to the login site. You might actually not even need it, but I don't think it's causing any problems.

In the second one, comment out the line with the CURLOPT_RETURNTRANSFER and run it. See what page it lands on.

0
 
LVL 5

Author Comment

by:basiclife
ID: 12034870
Unfortunately, still the welcome screen

I had to comment out the print_r statements, otherwise it screwed up the output

see http://bits.bris.ac.uk/basiclife/test.html
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12034936
Ah, that's my bad. I meant to comment out the FOLLOWLOCATION, not RETURNTRANSER. Sorry!
0
 
LVL 5

Author Comment

by:basiclife
ID: 12036433
s'ok. Let me try


"Method Not Allowed
An error has occurred. "

http://bits.bris.ac.uk/basiclife/test.html
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12036851
I don't get the Method Not Allowed error, but I do get a 1. It looks like the PHP doesn't output any code but just checks whether you login or not. Ok. Try this:

MAKE SURE YOU REMOVE YOUR ACCOUNT NAME AND NUMBER!!! from the site!!!

Ok, now try something like this:

<?php

//header("Content-type: text/plain");

$surname="surname";
$membershipNo="membership_number";
$rememberDetails="true"; //As a string ;)

$cpath="auto\cookies\cook";  //SET YOUR COOKIE PATH

//Set page to needed page first.
$ch=curl_init();

curl_setopt ($ch, CURLOPT_URL, "https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,logon,00.html?newMember=true");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cpath);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cpath);
curl_exec($ch);
//print_r(curl_getinfo($ch));
curl_close($ch);

$ch=curl_init();
/*rememberDetails could be false*/
curl_setopt($ch, CURLOPT_URL, "https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,,00.html");
curl_setopt($ch, CURLOPT_REFERER, "https://ibank.barclays.co.uk/fp/1_2c/online/1,26806,logon,00.html?newMember=true");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,"surname=".$surname."&membershipNo=".$membershipNo."&rememberDetails=".$rememberDetails);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cpath);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cpath);
$res=curl_exec($ch);
//print_r(curl_getinfo($ch));
curl_close($ch);

echo($res);
?>

0
 
LVL 5

Author Comment

by:basiclife
ID: 12036870
Straight back to the login page. And I've removed the code preview
0
 
LVL 5

Author Comment

by:basiclife
ID: 12036872
*welcome page not login page
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12036879
That's odd. I think it's time for me to experiment by myself ;)
0
 
LVL 5

Author Comment

by:basiclife
ID: 12036909
lol ok. Get back to me when you have some ideas.
0
 
LVL 36

Expert Comment

by:Zyloch
ID: 12099896
Sorry for the delay, I should have it in a day or so. Was configuring my new laptop
0
 
LVL 5

Author Comment

by:basiclife
ID: 12100653
No worries.
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Author Note: Since this E-E article was originally written, years ago, formal testing has come into common use in the world of PHP.  PHPUnit (http://en.wikipedia.org/wiki/PHPUnit) and similar technologies have enjoyed wide adoption, making it possib…
Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

718 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question