Link to home
Start Free TrialLog in
Avatar of gunjankhatri
gunjankhatri

asked on

Automatic Login to websites using HTTPUnit (To open the result in browser)

I am working on the automatic login to websites usingin java program.
I have written a code and getting the successfull response but I want open the response in browser.
Please find the code mentioned below which successfully logins to the rediff site and the response content has the  next page (i.e the used doesnt have to manullay login to the rediff site).but when i try to open the response in browser i am getting the blank page.I tried  working on the cookeis part but didnt get the proper response.
Please help me in this part.
(PLEASE REPLACE LOGIN AND PASSWORD WITH THE CORRECT ONE)
 
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.URL;
 
import com.meterware.httpunit.GetMethodWebRequest;
import com.meterware.httpunit.HttpUnitOptions;
import com.meterware.httpunit.WebConversation;
import com.meterware.httpunit.WebForm;
import com.meterware.httpunit.WebRequest;
import com.meterware.httpunit.WebResponse;
import com.meterware.httpunit.cookies.CookieListener;
import com.meterware.httpunit.cookies.CookieProperties;
 
public class Test2 {
	public static void main(String[] args) {
		try {
 
			WebConversation wc = new WebConversation();
			CookieProperties.setDomainMatchingStrict(false);
			CookieProperties.setPathMatchingStrict(false);
			HttpUnitOptions.setExceptionsThrownOnScriptError(false);
 
			CookieProperties.addCookieListener(new CookieListener() {
				public void cookieRejected(String s, int i, String s1) {
					System.out
							.println("CookieProperties.isDomainMatchingStrict() = "
									+ CookieProperties.isDomainMatchingStrict());
					System.out
							.println("CookieProperties.isPathMatchingStrict() = "
									+ CookieProperties.isPathMatchingStrict());
					System.out.println("s = " + s);
					System.out.println("s1 = " + s1);
					System.out.println("Reason: " + i);
				}
			});
			WebRequest req = new GetMethodWebRequest(
					"http://www.rediff.com/index.html");
			WebResponse resp = wc.getResponse(req);
 
			WebForm form = resp.getFormWithName("loginform");
           
          //  System.out.println(wc.getCookieDetails("JSESSIONID"));
			String[] cookies = wc.getCookieNames();
			for (int i = 0; i < cookies.length; i++) {
				String cooky = cookies[i];
				System.out.println(cooky + " = " + wc.getCookieValue(cooky));
				wc.putCookie(cooky, wc.getCookieValue(cooky));
			
	
				
			}
 
			form.setParameter("login", "myloginid");
			form.setParameter("passwd", "mypassowrd");
 
			form.submit();
			for (int i = 0; i < cookies.length; i++) {
				String cooky = cookies[i];
				System.out.println(cooky + " = " + wc.getCookieValue(cooky));
				wc.putCookie(cooky, wc.getCookieValue(cooky));
				
			}
			resp = wc.getCurrentPage();
              
			openURL(resp.getURL());
			//System.out.println(resp.getText());
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
 
	public static void openURL(URL url) throws ClassNotFoundException,
			NoSuchMethodException, IllegalAccessException,
			InvocationTargetException, IOException, InterruptedException {
		String osName = System.getProperty("os.name");
 
		if (osName.startsWith("Mac OS")) {
			Class fileMgr = Class.forName("com.apple.eio.FileManager");
			Method openURL = fileMgr.getDeclaredMethod("openURL",
					new Class[] { String.class });
			openURL.invoke(null, new Object[] { url.toString() });
		} else if (osName.startsWith("Windows")) {
			String cmdLine = "rundll32 url.dll,FileProtocolHandler "
					+ url.toString();
			Process exec = Runtime.getRuntime().exec(cmdLine);
			exec.waitFor();
		} else { // assume Unix or Linux
			String[] browsers = { "firefox", "opera", "konqueror", "epiphany",
					"mozilla", "netscape" };
			String browser = null;
			for (int count = 0; count < browsers.length && browser == null; count++) {
				if (Runtime.getRuntime().exec(
						new String[] { "which", browsers[count] }).waitFor() == 0) {
					browser = browsers[count];
				}
			}
			if (browser == null) {
				throw new IllegalStateException("Could not find web browser");
			} else {
				Runtime.getRuntime().exec(
						new String[] { browser, url.toString() });
			}
		}
	}
 
}

Open in new window

Avatar of Mick Barry
Mick Barry
Flag of Australia image

thats because when you open it in a browser you will be creating a new session.
The browser knows nothing about the session create by httpclient

Avatar of gunjankhatri
gunjankhatri

ASKER

but the session id which is generated from the code and the one which is shown when the browser is open is same
shown where?

httpclient is a lot better at this sort of thing btw

i tried httpclient also but there i was not even getting the next page  as output here atleast i ma getting next page
Yes u are correct.Browser is creating a new session
How can i resolve this?
not sure you can. you could try setting an appropriate cookie in the browser before loading the page

how can this be done??
no idea, it would be different for every browser.

have you tried using an applet?  I don't see how you're going to implement logging in the client by running code on the server.  You'd have to setup your server as a full blown proxy handling *all* requests for that to work.
 
the other thing to try would be url rewriting but that may not be supported by the server.

May be worth having a look at the variety of methods used for managing logins and sessions so you get a better understanding of what is needed.

I tried the same code with one of our internal site
The JSessionId generated from the code and teh one displyaed in the url is same.
but still the page is messy.
i have looked clearly in the rediff output also
The session id created is the same as the one in the url
Please find the output and the url .
Search for the session id  "5L23PK1KDKM0ukb7Eiivq2EWUXnOaCX1" in the Output content mentioned below.They are exactly same but still i m not getting the page

GENERATED URL:
http://f5plus.rediff.com/utilities/mailcontest/index.php?do=index&login=gunjankhatri&session_id=5L23PK1KDKM0ukb7Eiivq2EWUXnOaCX1



OUTPUT CONTENT:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
<HEAD>
<TITLE>Rediffmail</TITLE>
<script language="javascript">
var imgpath = 'http://jdelivery.rediff.com/utilities/mailcontest/pix';

var cc=window;
var u=true;
var reqnewmails;
function checkNewMails(login, session_id)
{
      var rkey = Math.floor(Math.random() * 1000000);
      folderurl="/iris/Main?do=folderservice&service=checknewmails&login="+login+"&session_id="+session_id+"&rkey="+rkey;
      if(cc.XMLHttpRequest)
      {
            reqnewmails=new XMLHttpRequest();
            reqnewmails.onreadystatechange=processReqNewMails;
            reqnewmails.open("GET",folderurl,u);
            reqnewmails.send(null);
      }
      else if(cc.ActiveXObject)
      {
            reqnewmails=new ActiveXObject("Microsoft.XMLHTTP");
            if(reqnewmails)
            {
                  reqnewmails.onreadystatechange=processReqNewMails;
                  reqnewmails.open("GET",folderurl,u);
                  reqnewmails.send();
            }
      }
}

function processReqNewMails()
{
      var data1, total, inboxtotal, bulktotal;
      total=0;bulktotal=0;inboxtotal=0;
      if(reqnewmails.readyState==4)
      {
            if(reqnewmails.status==200)
            {
                  if(reqnewmails.responseXML)
                  {
                         folders=reqnewmails.responseXML.getElementsByTagName("Name");
                        unreadcounts=reqnewmails.responseXML.getElementsByTagName("UnreadCount");
                        
                        for(var i=0;i<folders.length;i++)
                        {
                              if(folders[i].firstChild.nodeValue=="Inbox")
                              {
                                    inboxtotal = parseInt(unreadcounts[i].firstChild.nodeValue);
                              }
                              else if(folders[i].firstChild.nodeValue=="Bulk")
                              {
                                    bulktotal = parseInt(unreadcounts[i].firstChild.nodeValue);
                              }
                              else if(folders[i].firstChild.nodeValue!="Trash")
                              {
                                    total += parseInt(unreadcounts[i].firstChild.nodeValue);
                              }
                        }
                        document.getElementById("mailcountspan").innerHTML = "You have "+(inboxtotal==0?"No":inboxtotal)+" unread mail(s) in your Inbox";
                  }
            }
      }
}
function do_logout()
{
      window.open("/offerclose.htm","popunder","top=2, left=2,toolbars=no,maximize=no,resize=no,width=800,height=430,location=no,directories=no,scrollbars=yes");
      parent.window.focus();
      var logout_redr = "http://login.rediff.com/bn/logout.cgi?formname=general&login=gunjankhatri&session_id=5L23PK1KDKM0ukb7Eiivq2EWUXnOaCX1&function_name=logout";
      location.href = logout_redr;
}

var mydate=new Date();
var year=mydate.getYear();
if (year < 1000)
year+=1900;
var day=mydate.getDay();
var month=mydate.getMonth();
var daym=mydate.getDate();
if (daym<10)
daym="0"+daym;

var hours = mydate.getHours();
var minutes = mydate.getMinutes();
var timeValue = "" + ((hours >12) ? hours -12 :hours);
if (timeValue == "0") timeValue = 12;
timeValue += ((minutes < 10) ? ":0" : ":") + minutes;
timeValue += (hours >= 12) ? " PM" : " AM";

var dayarray=new Array("Sun","Mon","Tue","Wed","Thu","Fri","Sat");
var tmp_str_dt_tm = dayarray[day]+", "+daym+"/"+(++month)+"/"+year+", " +timeValue;
</script>



<style type="text/css">
.fntAr{FONT-FAMILY: Arial ;}
.fntTr{FONT-FAMILY: 'Trebuchet MS' ;}
.fv10{font-family:verdana;font-size:10px;}

.f12{font-size:12px;}
.f13{font-size:13px;}
.f14{font-size:14px;}
.f16{font-size:16px;}
.f17{font-size:17px;}
.f18{font-size:24px;}
.fntOrage{color:#e36300;}
.gerywhitegrad{background-color:#CFE4F7;
   filter:progid:DXImageTransform.Microsoft.Gradient
   (GradientType:0,StartColorStr:'#CFE4F7',EndColorStr:'#FFFFFF');}

.whitegerygrad{background-color:#FFFFFF;
   filter:progid:DXImageTransform.Microsoft.Gradient
   (GradientType:0,StartColorStr:'#ffffff',EndColorStr:'#f4f4f4');}

.sb6{font-family:verdana;font-size:10px;text-decoration:none}
.sb4{font-family:arial;font-size:18px;text-decoration:none}

.tbl_splash {background-color:#ffffff;
   filter:progid:DXImageTransform.Microsoft.Gradient
   (GradientType:0,StartColorStr:'#ffffff',EndColorStr:'#ededed');}

.tbl_splash_bottom {background-color:#ededed;}

.tbl_gotoinbox {background:#ffffff;border:1px solid #d3d3d3;}
</style>

</HEAD>

<BODY topmargin=0 leftmargin=0 rightmargin=0 bottommargin=0 link="#003bcd" >

<table width="100%" cellspacing=0 cellpadding=0 border=0>
<tr><td height="10"></td></tr>
<tr><td valign="bottom"><img src="http://jdelivery.rediff.com/iris/pix/Rediffmail_logo.gif" border=0 hspace="20"></td><td align="right" valign="bottom" class="fntTr f13"><a href="javascript:do_logout()">Logout</a>&nbsp;&nbsp;&nbsp;&nbsp;</td></tr>
<tr><td height="10"></td></tr>
<tr><td height="1" colspan="2" bgcolor="#8b8eb1"></td></tr>
</table>


<CENTER>
<table width=730 cellspacing=0 cellpadding=0 border=0>
<tr><td height=30></td></tr>
<tr>
<td width=383 valign="top">

<!--Go To Inbox block begins-->
<table cellspacing=0 cellpadding=0 border=0 width=100%>
<tr>
<td><img src="http://jdelivery.rediff.com/iris/pix/lefttop_container.gif" width="3" height="3" border="0" alt=""></td>
<td width=100% style="border-top:1px solid #cfcfcf;"><table cellspacing=0 cellpadding=0 border=0 width=100% height=1><tr><td height=1></td></tr></table></td>
<td><img src="http://jdelivery.rediff.com/iris/pix/righttop_container.gif" width="3" height="3" border="0" alt=""></td>
</table>

<table cellspacing=0 cellpadding=0 border=0 width=100% class=tbl_splash style="border-left:1px solid #cfcfcf;border-right:1px solid #cfcfcf;">
<tr>
<td width=8><table cellspacing=0 cellpadding=0 border=0 width=8><tr><td width=8></td></tr></table></td>
<td width=55% height=30 class="fntAr f14"><B>Welcome gunjankhatri,</B></td>
<td width=45% height=30 align="right" class=sb6><script>document.write(tmp_str_dt_tm);</script></td>
<td width=8><table cellspacing=0 cellpadding=0 border=0 width=8><tr><td width=8></td></tr></table></td>
</tr>
<tr>
<td></td>
<td colspan="2">
<table cellspacing=0 cellpadding=0 border=0 width=100% class=tbl_gotoinbox>
<tr>
<td height=36 style="padding-left:9px;" class="sb4"><A HREF="http://f5mail.rediff.com/prism/maillist?user_size=1&els=4781d5b9578bfd81693d3057b065b967" title="Go to Inbox"><img src="http://jdelivery.rediff.com/iris/pix/mailicon.gif" width="20" height="20" border="0" style="vertical-align:middle;"></A>&nbsp;&nbsp;<A HREF="http://f5mail.rediff.com/prism/maillist?user_size=1&els=4781d5b9578bfd81693d3057b065b967"><U>Go to Inbox</U></A>&nbsp;&nbsp;<FONT style="COLOR:#cecece;">|</FONT>&nbsp;&nbsp;<A HREF="http://f5plus.rediff.com/prism/writemail?els=4781d5b9578bfd81693d3057b065b967" title="Write Mail"><IMG SRC="http://jdelivery.rediff.com/iris/pix/write_mail.gif" WIDTH="29" HEIGHT="23" BORDER="0" style="vertical-align:middle;"></A>&nbsp;<A HREF="http://f5plus.rediff.com/prism/writemail?els=4781d5b9578bfd81693d3057b065b967"><U>Write Mail</U></A></td>
</tr>
</table>
</td>
<td></td>
</tr>
<tr><td height=5></td></tr>
</table>


<table cellspacing=0 cellpadding=0 border=0 width=100% class=tbl_splash_bottom>
<tr>
<td><img src="http://jdelivery.rediff.com/iris/pix/leftbottom_container.gif" width="3" height="3" border="0" alt=""></td>
<td width=100% style="border-bottom:1px solid #cfcfcf;"><table cellspacing=0 cellpadding=0 border=0 width=100% height=1><tr><td height=1></td></tr></table></td>
<td><img src="http://jdelivery.rediff.com/iris/pix/rightbottom_container.gif" width="3" height="3" border="0" alt=""></td>
</table>
<!--Go To Inbox block ends-->

<table cellspacing=0 cellpadding=0 border=0 height=8><tr><td height=8></td></tr></table>

<iframe src="http://www.rediff.com/uim/content/new/leftpromo.htm" name="leftpromo" marginwidth="0" marginheight="0" frameborder="0" height="300" scrolling="no" width="383"></iframe>
</td>
<td width=9></td>
<td width=338 valign="top">
<!--Right side Adv. begins-->
      <table cellspacing=0 cellpadding=0 border=0 width=338>
      <tr>
      <td width=338 align=center class=sb6>
      Advertisement<br/>
      <table height=1><tr><td height=1></td></tr></table>
            <SCRIPT LANGUAGE="JavaScript" TYPE="text/javascript">
            <!--
            document.write("<IFRAME SRC=\"http://im.rediff.com/uim/ads/Rediffmail_Welcomepage_336x280_Middle3.htm\" NAME=\"banner\" WIDTH=\"336\" HEIGHT=\"280\" MARGINWIDTH=\"0\" MARGINHEIGHT=\"0\" FRAMEBORDER=\"0\" SCROLLING=\"no\"><\/IFRAME>");
            //-->
            </SCRIPT>
      </td>
      <td width=14></td>
      </tr>
      <tr><td height=4></td></tr>
      </table>

<iframe src="http://www.rediff.com/uim/content/new/bottomads.htm" name="banner" marginwidth="0" marginheight="0" frameborder="0" height="100" scrolling="no" width="338"></iframe>
</td>
</tr>
</table>

</CENTER>


<!--footer begins -->            
<table cellspacing=0 cellpadding=0 border=0 width="100%">
<tr><td height=1 bgcolor=#8e8cb1></td></tr>
<tr><td align=center height=65><font class="fntTr f13"><SCRIPT LANGUAGE="JavaScript" src="http://im.rediff.com/uim/footer/rediffmail_footer.htm"></SCRIPT></font></td></tr>
</table>
<!--footer ends -->
</BODY>
</HTML>
Can someone please look into the code?
test on an internal server and check the server to see why its losing the session.

Can this be done by sharing session between the java program(WebResponse.getText) output and the output shown in the browser (which is nothing but WebResponse.getURL which i have opened in the browser using
      resp = wc.getCurrentPage();
             Desktop desktop = Desktop.getDesktop();
             desktop.browse(resp.getURL().toURI());

)

Please reply
I believe that would start a new session.

i have debugged the httpunit code and its doing exactly what i want and also teh response is exactly correct but the problem is displaying it on the browser.Here also the session ids created are exactly same. I have debugged the complete application and there is nowhere where its losing session.I have been hitting head against wall to resolve this issue
Please if someone can help or have faces similar kind of issue.
You can't do this without an interface between your Java app and the browser as it would require not only the session to be propagated to the browser, but also all cookies.

You could probably do this (in theory) with an open source browser by analysing its storage of the above
I am using Mozilla and its storing cookies for the url which i have opened but still not working
>>I am using Mozilla and its storing cookies for the url which i have opened but still not working

How are you moving the cookies from Java to Mozilla?
Have you checked the server yet?  You can'y say the session is ok unless you check the server as that is where the session is managed.

>>I am using Mozilla and its storing cookies for the url which i have opened but still not working

>>How are you moving the cookies from Java to Mozilla?

HTTPUnit does all this and generates the response. i am just taking that response and creating the url and opening it in browser.I have checked the the place where cookies are stored in mozzia.Its creating the cookie


>>Have you checked the server yet?  You can'y say the session is ok unless you check the server as that is where the session is managed.

As of now mine is a standalone application and we are trying to get teh response for the rediff page.
Please fine the code posted in the above  comments.
Also please let me know how to check the server as i m a newbee in this field of java
code looks fine to me

i have also posted the output html.It is exaclty the page which we require but when save the response generated as url and open it in browser , it gives me a blank page.
>>HTTPUnit does all this

It doesn't i'm afraid. It has no interface with your browser whatsoever. You'd have to make one
how do u save the response?

>>It doesn't i'm afraid. It has no interface with your browser whatsoever. You'd have to make one

I know it doesnot  have any interface with browser but  because of my requirement  i have send the response to the browser.I am developing a password manager which will be used by users to store thier passowords and automatically user can login to their websites without manual typing just by clicking teh link for this part i am using httpunit.

>>how do u save the response?

HttpUnit has a method to show the response url

I have written the following code to show the response in the browser
   WebConversation wc = new WebConversation();
   WebRequest req = new GetMethodWebRequest(
                              "http://rediffmail.com/");
  WebResponse resp = wc.getResponse(req);
  WebForm form = resp.getFormWithName("loginform");
  form.setParameter("userLogin", "mylogin");
  form.setParameter("userPassword", "mypassword");
  form.submit();
  resp = wc.getCurrentPage();
  Desktop desktop = Desktop.getDesktop();
  desktop.browse(resp.getURL().toURI());
>>but  because of my requirement  i have send the response to the browser

Yes, i'm aware of that. You want to pretend that the browser logged in to save the user the trouble of logging in. The fact is it didn't, but that's not the important thing. The important thing is that the server 'knows' it didn't because the browser doesn't have the cookies and session details that it should have. HttpUnit might have them, but that's no good to your browser
tats just the url of the page, it doesn't include response from server. the browser is going to send that request to the server to get a response.

>>ats just the url of the page, it doesn't include response from server. the browser
is going to send that request to the server to get a response.

Any idea how this can be done?
thats what you are already doing.

what you need to do is use a proxy to view what is sent and received when using a browser
then compare that to what happens when you attempt to use the httpunit generated url.

that should show you what the is going on differently.

Incidentally - if it's any consolation, you're part way there. When i open your page in Mozilla, it shows your name, so you certainly logged in OK.
You also need to handle opening the browser programatically properly, or that will cause problems too. Study the following carefully:

http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html
>> Incidentally - if it's any consolation, you're part way there. When i open your page in Mozilla, it shows your name, so you certainly logged in OK.

i didnt not understand the comment please elaborate
how to use proxy to view what is being sent and received?
>>i didnt not understand the comment please elaborate

I opened the html you posted above in Mozilla. You managed to log in OK, as it shows your name
ok  :)
Thanks but my main problem still persist
how to use proxy to view what is being sent and received?
use charles as I suggested earlier

I have tested rediff application.According to my testing when i open the rediff url through my program in browser, the cookies are not getting formed (Checked in "Show cookies" in mozzila. whereas while manullay login from rediff site cookies of first and second page are created.
How do i manage it using java program and to send the cookie infromation to the browser
as I already mentioned you can't

ASKER CERTIFIED SOLUTION
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
that has already been discussed, it would be a lot more helpful if you actually read the thread before posting :)

gunjankhatri, how did the marked comment answer your question? If that one didn't then please mark the one that did