• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 3668
  • Last Modified:

Login to mail.yahoo.com using CFHTTP (Yahoo Mail Scraper)

I'm trying to login to yahoo mail using CFHTTP. I read and pass the cookies, yet i'm still not succesful. Any ideas?

Feel free to plug this code into a blank cfm page. It will produces a useable output.

<CFHTTP Method="Get"
 URL="http://mail.yahoo.com/?.intl=us"
 UserAgent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
 Redirect="No">

<CFSET ChallengeStart=#Find(".challenge value=",CFHTTP.FileContent)#>
<CFSET ChallengeValue=#Mid(CFHTTP.FileContent,(ChallengeStart+18),28)#>
<CFSET uStart=#Find(".u value=",CFHTTP.FileContent)#>
<CFSET uValue=#Mid(CFHTTP.FileContent,(uStart+10),13)#>

<CFOUTPUT>
 <b>GET LOGIN PAGE</b><br>
 #CFHTTP.Header#<br><br>
 <CFSET ThisCookie=#Left(CFHTTP.ResponseHeader['Set-Cookie'],(FindOneOF("; ",CFHTTP.ResponseHeader['Set-Cookie'])-1))#>
 <CFSET FirstCookieName=#Left(ThisCookie,(Find("=",ThisCookie))-1)#>
 <CFSET FirstCookieValue=#Mid(ThisCookie,(Find("=",ThisCookie)+1),Len(ThisCookie))#>
 Cookie Name=#FirstCookieName#<br>
 Cooki Value=#FirstCookieValue#<br>
</CFOUTPUT>

<CFHTTP Method="Post"
 URL="https://login.yahoo.com/config/login?"
 UserAgent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
 Port="443"
 Redirect="No">
 <CFHTTPPARAM Type="Cookie" Name="#FirstCookieName#" Value="#FirstCookieValue#">
 <CFHTTPparam Type="FormField" name="login" Value="morgansowden">
 <CFHTTPparam Type="FormField" name="passwd" Value="12rifko12">
 <CFHTTPparam Type="FormField" name=".src" value="ym">
 <CFHTTPparam Type="FormField" name=".md5" value="">
 <CFHTTPparam Type="FormField" name=".hash" value="">
 <CFHTTPparam Type="FormField" name=".js" value="">
 <CFHTTPparam Type="FormField" name=".last" value="">
 <CFHTTPparam Type="FormField" name="promo" value="">
 <CFHTTPparam Type="FormField" name=".intl" value="us">
 <CFHTTPparam Type="FormField" name=".bypass" value="">
 <CFHTTPparam Type="FormField" name=".partner" value="">
 <CFHTTPparam Type="FormField" name=".u" value="#uValue#">
 <CFHTTPparam Type="FormField" name=".v" value="0">
 <CFHTTPparam Type="FormField" name=".challenge" value="#ChallengeValue#">
 <CFHTTPparam Type="FormField" name=".yplus" value="">
 <CFHTTPparam Type="FormField" name=".emailCode" value="">
 <CFHTTPparam Type="FormField" name="pkg" value="">
 <CFHTTPparam Type="FormField" name="stepid" value="">
 <CFHTTPparam Type="FormField" name=".ev" value="">
 <CFHTTPparam Type="FormField" name="hasMsgr" value="0">
 <CFHTTPparam Type="FormField" name=".chkP" value="Y">
 <CFHTTPparam Type="FormField" name=".done" value="http://mail.yahoo.com">
</CFHTTP>

<CFOUTPUT>
 <b>POST LOGIN INFO</b><br>
 #CFHTTP.Header#<br><br>
</CFOUTPUT>

 <CFLOOP From="1" To="4" Index="Count">
  <CFIF FindOneOF("; ",CFHTTP.ResponseHeader['Set-Cookie'][Count]) IS 0>
   <CFSET ThisCookie=#CFHTTP.ResponseHeader['Set-Cookie'][Count]#>
  <CFELSE>
   <CFSET ThisCookie=#Left(CFHTTP.ResponseHeader['Set-Cookie'][Count],(FindOneOF("; ",CFHTTP.ResponseHeader['Set-Cookie'][Count])-1))#>
  </CFIF>
  <CFSET CookieName=#Left(ThisCookie,(Find("=",ThisCookie))-1)#>
  <CFSET CookieValue=#Mid(ThisCookie,(Find("=",ThisCookie)+1),Len(ThisCookie))#>
 <CFOUTPUT>
 Cookie Name=#CookieName#<br>
 Cookie Value=#CookieValue#<br><br>
 </CFOUTPUT>
 </CFLOOP>

<CFHTTP Method="Get"
 URL="#CFHTTP.ResponseHeader['Location']#"
 UserAgent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
 Port="443"
 Redirect="No">
 <CFLOOP From="1" To="4" Index="Count">
  <CFIF FindOneOF("; ",CFHTTP.ResponseHeader['Set-Cookie'][Count]) IS 0>
   <CFSET ThisCookie=#CFHTTP.ResponseHeader['Set-Cookie'][Count]#>
  <CFELSE>
   <CFSET ThisCookie=#Left(CFHTTP.ResponseHeader['Set-Cookie'][Count],(FindOneOF("; ",CFHTTP.ResponseHeader['Set-Cookie'][Count])-1))#>
  </CFIF>
  <CFSET CookieName=#Left(ThisCookie,(Find("=",ThisCookie))-1)#>
  <CFSET CookieValue=#Mid(ThisCookie,(Find("=",ThisCookie)+1),Len(ThisCookie))#>
  <CFHTTPPARAM Type="Cookie" Name="#CookieName#" Value="#CookieValue#">
 </CFLOOP>
 <CFHTTPPARAM Type="Cookie" Name="#FirstCookieName#" Value="#FirstCookieValue#">
</CFHTTP>

<CFOUTPUT>
 <b>RESULT OF LOGIN</b><br>
 #CFHTTP.Header#<br><br>
 #CFHTTP.ErrorDetail#<br><br>
 #CFHTTP.FileContent#<br><br>
</CFOUTPUT>
0
drkolosgold
Asked:
drkolosgold
  • 2
1 Solution
 
INSDivision6Commented:
I was able successully pass login up to the main mail page using CFX_HTTP5 tag instead of CFHTTP.  However, the mail page is a frameset.  Some heavy JS business starts there.  The final page looks like:


<!--web31405-->
<html>
<title> Yahoo! Mail - morgansowden@yahoo.com </title>
<noscript>
<META HTTP-EQUIV=Refresh CONTENT="0; URL=/ym/login?nojs=1">
</noscript>
 
<script type="text/javascript" src="http://us.js1.yimg.com/us.yimg.com/lib/pim/r/medici/9_11/mail/mailcommonlib.js"></script>

<script>

function showCompose(body, to, from, title, cc, kind, urlExtras )
{
    //if(body != null)  
    if( arguments.length >= 5 )
    {
        gBody = body;
        gTo = to;
            gFrom = from;
        gTitle = title;
        gCc = cc;
        gKind = kind;
            gUrlExtras = urlExtras;
    }
    else
    {
        gBody = gTo = gFrom = gTitle = gCc = gKind = gUrlExtras = "";
    }

    ymailmain.document.location.href = currentURL();

}

function currentURL()
{
    return '/ym/instacompose?'+ gEmail+getComposeIter() ;
}

var gBody = "";
var gTo = "";
var gFrom = "";
var gTitle = "";
var gCc = "";
var gKind = "";
var gUrlExtras = "";

function getReplyForwardTo(){ return gTo; }
function getReplyForwardFrom(){ return gFrom; }
function getReplyForwardTitle(){ return gTitle; }
function getReplyForwardBody(){ return gBody; }
function getReplyForwardCc(){ return gCc; }
function getReplyForwardKind(){ return gKind; }
function getReplyForwardUrlExtras(){ return gUrlExtras; }

function setReplyForwardTo(toval){ gTo = toval; }
function setReplyForwardFrom(fromval){ gFrom = fromval; }
function setReplyForwardTitle(titleval){ gTitle = titleval; }
function setReplyForwardBody(bodyval){ gBody = bodyval; }
function setReplyForwardCc(ccval){ gCc = ccval; }
function setReplyForwardKind(kindval){ gKind = kindval; }
function setReplyForwardUrlExtras(ue){ gUrlExtras = ue; }


function refreshCompose()
{
      incrementComposeIter();
      loadCompose();
}

function loadCompose()
{
    ymailcompose.document.location.replace( currentURL() );
}

function loadMain(url)
{
    ymailmain.document.location.replace( url );
}

function asynchLoadCompose(mill)
{
    window.setTimeout("loadCompose();", mill);
}

function asynchRefreshCompose(mill)
{
    window.setTimeout("refreshCompose();", mill);
}


function getComposeIter()
{

      var composeIter = null;
      composeIter = getICCookie();
      return composeIter;
}

function seedComposeIter()
{
      var newDate = null;
      newDate = new Date();
      var timeStampInMillis = newDate.getTime();
      composeIter = timeStampInMillis;
      setICCookie(composeIter);
}

function incrementComposeIter()
{
      var composeIter = getComposeIter();
      composeIter++;
      setICCookie(composeIter);
}


var gWelcomePage = "http://us.f314.mail.yahoo.com/ym/welcome?YY=79670";
var gLoginUrl =  "http://us.f314.mail.yahoo.com/ym/login?YY=79670";

var gTargetPage = null;

var gEmail = "morgansowden@yahoo.com".split("@", 1)[0] ;
var gTheFrameset = true;

function initFrameset()
{
      if(top != self)
      {
            safeReloadFrameset(gLoginUrl);
            return;
      }
      removeCookie(gReloadCookieName);

      var composeIter = getComposeIter();
      if( composeIter == null )
            seedComposeIter();

      if( ymailmain.document.location.href != "about:blank" )
      {
            return;
      }

      var framesetCookie = getFramesetCookie();
      var redirectUrl = null;

      if(framesetCookie != "http://mail.yahoo.com")
            redirectUrl = framesetCookie ;

      if( redirectUrl )
      {

            removeCookie(gFramesetCookieName);
            loadMain(redirectUrl);
      }
      else if( gTargetPage )
      {

            loadMain(gTargetPage);
      }
      else
      {
            var newDate = new Date();
            loadMain(gWelcomePage + "&rand=" + newDate.getTime());
      }

      asynchLoadCompose(0);
}
</script>


<frameset ROWS="100%,*,*" border=0 onload="initFrameset();" >

<frame name="ymailmain" id="ymailmain" > 
<frame name="ymailcompose" id="ymailcompose" frameborder=0 noresize>

</frameset>
</html>
<!-- v3.301.10.16.2 1115757615 -->
<!-- 0.05786 -->
<!-- web31405.mail.mud.yahoo.com uncompressed/chunked Sun May 15 16:49:08 PDT 2005 -->


CFX_HTTP5 is available here: http://www.cftagstore.com/tags/cfxhttp5.cfm.  My code:

<cfoutput>

<CFX_HTTP5 URL="http://mail.yahoo.com/" SESSION="START" OUT="OUT">
<xmp>#OUT#</xmp>

<CFSET ChallengeStart=Find(".challenge value=", OUT)>
<CFSET ChallengeValue=Mid(OUT, (ChallengeStart+18), 28)>
<CFSET uStart=Find(".u value=", OUT)>
<CFSET uValue=Mid(OUT, (uStart+10), 13)>

<CFSET BODY="login=morgansowden&passwd=12rifko12">
<CFSET BODY="#BODY#&.src=ym&.md5=&.hash=&.js=&.last=&promo=&.intl=us&.bypass=">
<CFSET BODY="#BODY#&.partner=&.u=#uValue#&.v=0&.challenge=#ChallengeValue#&.yplus=&.emailCode=">
<CFSET BODY="#BODY#&pkg=&stepid=&.ev=&hasMsgr=0&.chkP=Y&.done=http://mail.yahoo.com">

<CFX_HTTP5 URL="https://login.yahoo.com/config/login?" METHOD="POST" SESSION=#HTTPSESSION# BODY=#BODY# OUT="OUT">
<xmp>#OUT#</xmp>

<cfset out=Left(OUT, Find('">click here', OUT)-1)>
<CFSET urlStart=Find('a href="', OUT)>
<CFSET urlValue=Mid(OUT, (urlStart+8), Len(OUT))>

<CFX_HTTP5 URL=#urlValue# SESSION=#HTTPSESSION# OUT="OUT" SESSIONEND="Y">
<xmp>#OUT#</xmp>
</cfoutput>
0
 
woodendudeCommented:
Interesting!
0
 
drkolosgoldAuthor Commented:
I've downloaded and tested CFX_HTTP5. It works much better than cfhttp as it seems to automatically handle cookies and redirects without the need for extra programming. That tip itself is worth the answer, so thank you.

I'm quite poor with JS, so was wondering how much of that is essential? Once I have the returned html page in some kind of variable, I'm quite happy parsing the string myself.

What would the above code look like without the JS?
0
 
INSDivision6Commented:
It is difficult to say what happens next (you can login normally an to see using Vies Source), but on the page above you have to construct next URL (main frame of mail app) using expression:

loadMain(gWelcomePage + "&rand=" + newDate.getTime());

This is what browser suppose to execute.  SInce there is no browser, you have to cook your next URL on your own.  Same with the other frame, if it is needed.  So, you should seriously consider whether it worth it.
0

Featured Post

How to Use the Help Bell

Need to boost the visibility of your question for solutions? Use the Experts Exchange Help Bell to confirm priority levels and contact subject-matter experts for question attention.  Check out this how-to article for more information.

  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now