Solved

httpwebrequest not getting all cookies

Posted on 2009-06-29
5
1,324 Views
Last Modified: 2013-11-27
I'm trying to log into Gmail using httpwebrequest
Login Page: https://www.google.com/accounts/ServiceLogin?service=mail

Cookies Example:

Name      NID
Value      24=qt-xxhhTTzY6EsgcHEsEJLbV4g0Ak3EYcVs-zvIkaLK8J-0vw3PdfHIP9sovL7V-ayQ_AlUgSo_5TG9_clYq0_zZrLC9Nmbo_jkyu645czYrs03eenI6PfA8IFt9EKba
Host      .google.com
Path      /
Secure      No
Expires      Wed, 30 Dec 2009 04:25:17 GMT

Name      PREF
Value      ID=7a29976d0736ce96:U=db98138f09065e00:TM=1245168329:LM=1245174280:GM=1:S=Ga8mZhXB65Ap0ram
Host      .google.com
Path      /
Secure      No
Expires      Thu, 16 Jun 2011 17:44:39 GMT

Name      __utma
Value      173272373.928349011.1245168415.1246069676.1246206632.33
Host      .google.com
Path      /accounts/
Secure      No
Expires      Tue, 28 Jun 2011 16:30:31 GMT

Name      __utmz
Value      173272373.1245168415.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none)
Host      .google.com
Path      /accounts/
Secure      No
Expires      Wed, 16 Dec 2009 04:06:54 GMT

Name      rememberme
Value      false
Host      .google.com
Path      /
Secure      No
Expires      Mon, 24 Jun 2019 14:13:21 GMT

Name      GALX
Value      bdTHhQZCu54
Host      www.google.com
Path      /accounts
Secure      Yes
Expires      At End Of Session

I run the code below and the only cookie it gets is the GALX
Dim CookieJar = New CookieContainer

 Dim httpRequest As HttpWebRequest

 Dim httpResponse As HttpWebResponse

 Dim responseStream As Stream

 Dim responseEncoding As Encoding

 Dim responseStreamReader As StreamReader

 Dim strResponse1 As String = ""

 

CookieJar = New CookieContainer

httpRequest = CType(WebRequest.Create("https://www.google.com/accounts/ServiceLogin?service=mail"), HttpWebRequest)

httpRequest.Timeout = 5 * 10000

httpRequest.AllowAutoRedirect = True

httpRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"

httpRequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729)"

httpRequest.KeepAlive = True

httpRequest.CookieContainer = CookieJar

httpResponse = CType(httpRequest.GetResponse(), HttpWebResponse)

responseStream = httpResponse.GetResponseStream()

responseEncoding = System.Text.Encoding.GetEncoding("utf-8")

responseStreamReader = New StreamReader(responseStream, responseEncoding)

If Not responseStreamReader Is Nothing Then

    strResponse1 = responseStreamReader.ReadToEnd

End If

Open in new window

0
Comment
Question by:neta1o
  • 3
  • 2
5 Comments
 
LVL 39

Expert Comment

by:abel
ID: 24743968
It may be a lot easier if you use the .NET GMail API for your task: http://sourceforge.net/projects/gmail-api

The problem with the request is probably the domain name. Are the above cookies inside your CookieJar? Can you change the hosts to reflect "www.google.com" instead of ".google.com"?

Note that the rememberme cookie is expired.

The server is not required to bounce the cookies back. The server is, in general, in control of the cookies. As an example, it could be that if they consider you're not from Internet Explorer, that certain cookies are not valid. In which case it may simply ignore some of them. In this case you mimic Firefox, but I'm not sure if all headers are set correctly to fool the server.

All these and other problems have already been resolved in the API I mentioned above.To save yourself a lot of problems, I can strongly suggest giving it a shot.

-- Abel --
0
 
LVL 39

Expert Comment

by:abel
ID: 24743984
> Are the above cookies inside your CookieJar?
hmm, I can answer that myself. Looking at your code again, I see that there are not cookies at all in the cookiejar, or perhaps you left out some of the code? Getting a meager response is then quite likely. If you want to send the cookies to the server to mimic a login of something, you'll have to send the cookies along, they're not picked up from somewhere automatically...
0
 

Author Comment

by:neta1o
ID: 24744404
The code above is a snippet, I have the code for the POST part that uses the same cookiejar but to keep the code simple I wanted to use that example.  It's also not a Gmail issue.  If you try to do a GET on https://accounts.craigslist.org/

You will also not get the cookie.

The only commonality I see is that it is a secure site (https) and the web request only seems to find the cookies marked as secure: yes as indicated by web developer toolbar (pasted above from the gmail example)
0
 
LVL 39

Expert Comment

by:abel
ID: 24746046
If I do a GET with my browser I don't see any cookies either for that site so that's not too surprising.

> and the web request only seems to find the cookies marked as secure
I noticed the "s" but didn't make that connection. But you are right of course. Can't believe I missed that, must be tired over here or something. You just answered the question ;-)
0
 

Accepted Solution

by:
neta1o earned 0 total points
ID: 24747087
My gut feeling is it has to do with the secure site and the non secure cookie.

For Example: https://accounts.craigslist.org/
If I do a httpwebrequest on that page with a cookie container I don't get any cookies.

Although if I go to that site with IE or FIrefox, with or without Javascript enabled, the cookie is passed to my computer.  There has to be a way to capture this cookie.

0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
egit plugin on eclipse 8 37
Get id from json Data with NewtonSoft 3 34
Class object 2 26
Angular JS Route 3 50
Whether you've completed a degree in computer sciences or you're a self-taught programmer, writing your first lines of code in the real world is always a challenge. Here are some of the most common pitfalls for new programmers.
Real-time is more about the business, not the technology. In day-to-day life, to make real-time decisions like buying or investing, business needs the latest information(e.g. Gold Rate/Stock Rate). Unlike traditional days, you need not wait for a fe…
An introduction to basic programming syntax in Java by creating a simple program. Viewers can follow the tutorial as they create their first class in Java. Definitions and explanations about each element are given to help prepare viewers for future …
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …

910 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now