Link to home
Create AccountLog in
Avatar of jacksmind

asked on

Using curl to log into a post data website

I would like to design a script that will find out when a webpage is updated.  The problem is that this website is secure.  The password/username is submitted through php post data, cookies are used for authentication.  I would like to use curl from the command line to login,  update the cookie, and fetch the page, then I'm sure I an do a diff on a saved page to determine if there has been any changes.

Here is the POST submit code:
 <div class="content"><form action="/frontpage?destination=frontpage"  accept-charset="UTF-8" method="post" id="user-login-form">
<div><div class="form-item" id="edit-name-wrapper">

 <label for="edit-name">Username: <span class="form-required" title="This field is required.">*</span></label>
 <input type="text" maxlength="60" name="name" id="edit-name" size="15" value="" class="form-text required" />
<div class="form-item" id="edit-pass-wrapper">
 <label for="edit-pass">Password: <span class="form-required" title="This field is required.">*</span></label>
 <input type="password" name="pass" id="edit-pass"  maxlength="60"  size="15"  class="form-text required" />
<input type="submit" name="op" id="edit-submit" value="Log in"  class="form-submit" />
<input type="hidden" name="form_build_id" id="form-49b0af464716f15aadd2e85b14f546d8" value="form-49b0af464716f15aadd2e85b14f546d8"  />

<input type="hidden" name="form_id" id="edit-user-login-block" value="user_login_block"  />

Open in new window

I tried to use:

curl -d "name=USERNAME&pass=PASSWORD&submit=\"Log in\"" http:\\

and other variations, but I can't seem to grab the logged in version of the page.

Avatar of te-edu
Flag of Serbia image

No need command line you can use php curl, there are options for that. cookie and cookie jar. For checking difference you can use caching. Nothing to difficult just read php curl manual there  are all you need there.
Avatar of jacksmind


I'd rather not use php curl if I can avoid it as I don't have php installed.
You need to use the following keys/parameters:
-L make curl follow the HTTP redirect headers
-c <filename> set a file to store cookies

curl -L -d "name=USERNAME&pass=PASSWORD&submit=\"Log in\"" -c cookie.txt http:\\

Open in new window

Just curious... If you do not have PHP installed, why did you post this question in the PHP zone?

If you decide you want a PHP solution and (this is important) you can show us that the terms of service allow you to make automated logins, then please post back with that information and I will try to help you get it working.  But be forewarned that every CURL login script is a research project.  It may take a long time to debug.
Because someone who knows about php curl is more likely to know curl (and there wasn't a 'curl' zone ;) ).

There are no 'terms of service' per se, this is on a personal server associated with a research project, but I can't have direct mysql access.  The email from the admin:

"Hey Jack,
I wouldn't be comfortable giving you direct access to the mysql database, but if you want wget and curl have the ability to automate logins.  Ask Mathias if you need help with this, he's done something similar before."

Incidentally, Mathais wasn't much help ;).
Just curious, didn't my comment above help to get what you need?
Well, I can show you the PHP version of this, but that may not be very helpful if you don't have PHP.  It looks as if the form is protected by a token, so you would want to get the token and submit it, along with the other hidden form fields, and of course the credentials.
@sudarka It didn't seem to work

@Ray  Well I suppose I could install php if I have to, I was just hoping I could do it from the command line.
Avatar of Ray Paseur
Ray Paseur
Flag of United States of America image

Link to home
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
Link to home
Create an account to see this answer
Signing up is free. No credit card required.
Create Account
I think this info may help others, but unfortunately I couldn't get it to work.  I decided to just to use the cookie for authentication and then use wget every 15 min or so to check for an update. If the script detects a password prompt in the page then we know the cookie has expired.   Not the most elegant way, but probably the best use of my time. Thanks everyone for their help.
Sorry to hear that it didn't workout for you the way you wanted. Thanks for the points.