Solved

Login to Jsp Page with Curl

Posted on 2008-06-16
15
3,121 Views
Last Modified: 2014-01-24
Hi,

0 am trying to loging to a Jsp page with curl. in the page source i see two inputs like username and password and i post the values in the code.  The jsp page may be written using struts, i don't know. What else do i have to post to the login site or should i take another way ?

Thanks.

$user="<user>";
$pass="<password>";
$ch = curl_init();
curl_setopt($ch3, CURLOPT_URL, 'https://website.com/login.do');
curl_setopt($ch3, CURLOPT_POSTFIELDS,'username='.$user.'&password='.$pass);
curl_setopt($ch3, CURLOPT_POST, 1);
curl_setopt($ch3, CURLOPT_HEADER, 1);
curl_setopt($ch3, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch3, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch3, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_setopt($ch3, CURLOPT_COOKIEJAR, "/tmp/getcookies.txt");
curl_setopt($ch3, CURLOPT_REFERER, "https://website.com/login.do");
curl_setopt($ch3, CURLOPT_COOKIEFILE, "/tmp/getcookies.txt");
curl_setopt($ch3, CURLOPT_RETURNTRANSFER, 1);
 
$data = curl_exec($ch3);
curl_close($ch3);
echo $data;

Open in new window

0
Comment
Question by:kenanerdey
  • 7
  • 5
  • 2
  • +1
15 Comments
 
LVL 48

Expert Comment

by:hernst42
ID: 21793180
Doesn't you code work or in this a typo in you posted example:
$ch = curl_init();
then all acces in done via $ch3 ??
Try usign an array to postfields like

curl_setopt($ch3, CURLOPT_POSTFIELDS, array('username' => $user, 'password' =>$pass));
0
 

Author Comment

by:kenanerdey
ID: 21793370
Hi,

it's my typo pardon. i changed the code so that post fields in array but still login page comes.
if i try to send data with username,password and JSESSIONID with the value i get from cookie file from adress bar like https://website.com/login.do?username=<username>&password=<password>&JSESSIONID=<id_in_cookie_file>  same login page comes again.

Thanks.
0
 
LVL 27

Expert Comment

by:mrcoffee365
ID: 21795910
It is likely that the site requires cookies returned to it.  Use a network monitoring tool, or one of the many browser Web developer tools, to watch the request/response interaction for a successful login to the site from a browser.  Then write your code to follow that sequence.

If you have control over the Web server at website.com, then you could change it to allow a no-cookie login, where, for example, all of the user and session info is in the request parameters.
0
Webinar: Aligning, Automating, Winning

Join Dan Russo, Senior Manager of Operations Intelligence, for an in-depth discussion on how Dealertrack, leading provider of integrated digital solutions for the automotive industry, transformed their DevOps processes to increase collaboration and move with greater velocity.

 

Author Comment

by:kenanerdey
ID: 21801581
Hi,

If it's a ssl connection i can't see any clear text information when i look at with ethereal. ACK messages and after than TLS connection begins. some hand shaking and afterwards application data is received.
0
 
LVL 27

Expert Comment

by:mrcoffee365
ID: 21802452
Then use a browser tool.  You'll see the correct sequence of HTTP request/response headers, which you can use in your login code.

For Firefox, LiveHTTPHeaders:
https://addons.mozilla.org/en-US/firefox/addon/3829

For IE, the IE Developer Toolbar will show you cookies.  Fiddler is good for debugging, as well:
http://www.fiddlertool.com/fiddler/

The page above says it works with Firefox, now, so that's good.
0
 

Author Comment

by:kenanerdey
ID: 21805569
Hi,

Thanks for your help. when i try to login from login page i noted headers as below:

POST /some_path /login.do HTTP/1.1
Host: website.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://website.com/login.do
Cookie: JSESSIONID=2D92DD1126FF254510B56B7263002151E
Content-Type: application/x-www-form-urlencoded
Content-Length: 38
userName=<my_user_name>&password=<my_password>
HTTP/1.x 200 OK
Date: Tue, 17 Jun 2008 15:14:14 GMT
Server: Apache-Coyote/1.1
Content-Type: text/html;charset=UTF-8
Connection: close
Transfer-Encoding: chunked

I saw it sends JSESSIONID in header. And i rewrote the php script as if i firsty enter login page, get the cookie then send the post data with that cookie.  when i try to run the code from command line in verbose mode, connection begins and waits on the line "Expect: 100".  i googled and removed that header with curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:')). Now nothing says when i run the code. But still waits there. When i tried in browser i's being seen as loading. i attach my code. thanks for your ideas.

$id = "<user_name>";
$pw = "<password>";
$ch = curl_init();
$header[]="Host:website.com";
$header[]="Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0";
$header[]="Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$header[]="Accept-Language: en-us,en;q=0.5";
$header[]="Accept-Encoding: gzip,deflate";
$header[]="Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[]="Keep-Alive: 300";
$header[]="Connection: keep-alive";
$header[]="Referer: http://website.com/login.do";
 
//ilk ekran bolumu
 
curl_setopt ($ch, CURLOPT_URL,"https://website.com/login.do");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ($ch, CURLOPT_COOKIEJAR, "/tmp/cookie");
curl_setopt ($ch, CURLOPT_COOKIEFILE, "/tmp/cookie");
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);
curl_exec ($ch);
curl_close($ch);
 
$cmd="cat /tmp/cookie | tail -1 | awk '{print $7;}'";
$sid=exec($cmd);
 
$header[]="Cookie: JSESSIONID=$sid";
$header[]="Content-Type: application/x-www-form-urlencoded";
$header[]="Content-Length: 38";
 
 
 
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt ($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_POSTFIELDS, array('userName' => $id, 'password' =>$pw));
//curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'))
curl_setopt ($ch, CURLOPT_COOKIEJAR, "/tmp/cookie");
curl_setopt ($ch, CURLOPT_COOKIEFILE, "/tmp/cookie");
curl_setopt ($ch, CURLOPT_URL,"https://website.com/login.do");
curl_setopt($ch, CURLOPT_POST, 1);
$data=curl_exec ($ch); 
echo $data;

Open in new window

0
 
LVL 27

Expert Comment

by:mrcoffee365
ID: 21807915
This looks pretty good -- I think you're on the right track.  I don't use curl, so I can't help with that, but I've written automatic login code.

There are a couple of things which give me pause:
* It looks as if you have your username and password in the middle of the post.  The cookie should be part of the header, then the username and password get posted as form fields.
However, maybe curl will handle this, since the  curl_setopt commands for the cookie are to special variables.  It just stands out, in looking at the code.
* your formatting for header fields is not exact.  For example, you have
$header[]="Host:website.com";
and the value should be
$header[]="Host: website.com";
Note the space before the website.com domain.  HTTP uses spaces for parsing, as well as colons.
However -- since you are using a special curl call CURLOPT_URL, you probably shouldn't include this in your header at all.
* you don't have all header fields identified.  For example:
$header[]="Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0";
does not say "UserAgent: " before the Mozilla string.
* it looks as if you are attaching the cookie using CURLOPT_COOKIEFILE, so I don't think it's a good idea to add the cookie explicitly in the header.  Make sure that the $sid for the JSESSIONID value matches the one handed to you in your GET to the site.

You should print out what you're posting to the site from your PHP code, so you can see how much it looks like the HTTP interaction from the browser.

I don't see where the Submit button name is in the headers from the site that you posted, or in the code where you're posting to the site.  The Post should have a Submit button, maybe called "Login" or "Submit" and the login program on the site might be looking for it.

Try printing out everything you get from the site in your PHP get to it, to see what it's sending you.
0
 

Author Comment

by:kenanerdey
ID: 21819908
Hi,

i disabled sending cookie manually and posting form values as string. Because if i send formfields as array it waits as i said in my previous post. in, i run php from cli and  the output is as below:

* About to connect() to website.com port 443
*   Trying xxx.xxx.xx.x... * connected
* Connected to website.com (xxx.xxx.xx.x) port 443
* successfully set certificate verify locations:
*   CAfile: /usr/share/curl/curl-ca-bundle.crt
  CApath: none
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
*        subject: <certificite information>
*        start date: <start_date>
*        expire date: <expire_Date>
*        common name: *.website.com (matched)
*        issuer: <certificate company>
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> POST /some_path/login.do HTTP/1.1


Cookie: JSESSIONID=8BEE15E90F331C3B55390D4378DF5769
Host: website.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://website.com/some_path/login.do
Content-Type: application/x-www-form-urlencoded
Content-Length: 38

userName=<user_name>&password=<password>< HTTP/1.1 200 OK
< Date: Thu, 19 Jun 2008 07:09:21 GMT
< Server: Apache-Coyote/1.1
< Content-Type: text/html;charset=UTF-8
< Connection: close
< Transfer-Encoding: chunked
* Closing connection #0
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Expires" content="-1">

And agains it fails and just login page comes again without any error or messages that will help us.
0
 
LVL 27

Expert Comment

by:mrcoffee365
ID: 21823118
From what you've posted, it looks as if you're doing the right post to the login page.  So that's good -- it's best to imitate the working example from your browser as much as possible.

So I googled for PHP and curl https requests, and found this page (from google cache, because the normal page doesn't have the useful info any more):
http://64.233.169.104/search?q=cache:iV-MXcvuN-YJ:www.php.net/curl+php+https+connect+curl&hl=en&ct=clnk&cd=6&gl=us

In it is a discussion thread of many people using PHP and curl to connect to HTTP and HTTPS servers.

If you are in a situation where you have to have an SSL certificate to send to the https server, then I imagine that the PHP code has to get a lot more complicated.  I know that Java programs have to go through extra hoops, to accept the SSL server, then send the right certificate.

Is this a connection where you have a certificate which authorizes you to the HTTPS server?  I assume that's not the case, but it's good to check.

I would look through the thread I posted above, and check your code against the suggestions given.

The next step is for you to create an https server to test locally, so you can see why the server is rejecting your login.
0
 
LVL 27

Accepted Solution

by:
mrcoffee365 earned 500 total points
ID: 21823559
Another suggestion:  Try your code logging in to another site using https.  Maybe Google?  If you have a gmail account, you can go to https to log in to Google with your account, and see if you get different behavior that helps you figure out this problem.
0
 

Author Comment

by:kenanerdey
ID: 21828838
Hi,

changed order of curl_setopts. i sent the postfields as text not arrray. And somehow i succeeded. i hope My boss may awards me.

Thanks for your all help.
0
 
LVL 27

Expert Comment

by:mrcoffee365
ID: 21835209
Congrats on that!  And you're welcome.

 I didn't realize that the username and password were in an array -- it is often better to be as simple as possible with HTTP communications.  If you changed the order of the setopts to more closely imitate the order sent by a browser, that's always a good thing to do.

0
 

Expert Comment

by:VAMSICA
ID: 39805728
Hi kenanerdey,

This is great post, I'm working on the  same requirement curl to jsp page, I'm almost close to the solution with your help but not yet there.

Could you please post your final solution here with change in order of curl_setopts you mentioned.

Thank you for your time.
0
 
LVL 27

Expert Comment

by:mrcoffee365
ID: 39808221
It would be better to ask your question in a new question rather than adding to this 6-year-old post.  The asker hasn't been back for 6 years, so the likelihood of them posting something is low.

When you ask your new question, try posting some code to show how far you've gotten, and tell us what isn't working.
0
 

Expert Comment

by:VAMSICA
ID: 39808258
Thank you for looking into it. As you suggested I've posted the new question, here it is

http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_28347137.html

I'd be glad if you can take a look at it.

Thanks,
Vamsi.
0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
XML extra information 8 29
PHP alternative to file_get_contents('php://input') 4 70
ajax to record click 3 16
JQuery Search Filter 2 33
Things That Drive Us Nuts Have you noticed the use of the reCaptcha feature at EE and other web sites?  It wants you to read and retype something that looks like this.Insanity!  It's not EE's fault - that's just the way reCaptcha works.  But it is …
This article discusses how to create an extensible mechanism for linked drop downs.
Learn the basics of strings in Python: declaration, operations, indices, and slicing. Strings are declared with quotations; for example: s = "string": Strings are immutable.: Strings may be concatenated or multiplied using the addition and multiplic…
The viewer will learn the basics of jQuery, including how to invoke it on a web page. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery.: (CODE)

831 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question