using two httpWebRequest with the same cookies

Hi,

I’ve seen a lots of posting on the web but no real solution to my particular problem.

Here's what I have:

I have an application that logs onto a web site that returns me a HTTP page containing files that I need to download. This works.

My code for the first URL
   
 Dim CookieCol As CookieCollection
    Dim encoding As New System.Text.ASCIIEncoding
    Dim LoginRes As HttpWebResponse
    Dim LoginReq As HttpWebRequest
       
        Dim PostData As String = "UserID=mikec&password=xmltest&username=XML%20%Retrieval"
        Dim Data() As Byte = encoding.GetBytes(PostData)

        LoginReq = HttpWebRequest.Create(“HTTPS://mysite/logon”)

        'Initialise the request
        LoginReq.KeepAlive = True
        LoginReq.Method = "POST"
        LoginReq.ContentType = "application/x-www-form-urlencoded"
        LoginReq.ContentLength = Data.Length
        LoginReq.CookieContainer = New CookieContainer

        Dim SendReq As IO.Stream = LoginReq.GetRequestStream
        SendReq.Write(Data, 0, Data.Length)
         SendReq.Close

          LoginRes = LoginReq.GetResponse()
        'Add any returned cookies to the cookie collection
          CookieCol = LoginReq.CookieContainer.GetCookies(LoginReq.RequestUri)
     

        Dim sReader As IO.StreamReader = New IO.StreamReader(LoginRes.GetResponseStream)
        Dim HTML As String = sReader.ReadToEnd

       
I parse the HTML page that I’m redirect too and get the name of the files and then I have to download them
However  I need  to go to a different web page using the  “GET” method to have the web server send me the files.

My code for second URL:

  Dim results As MatchCollection = Regex.Matches(html, "(?<=<a.*>).*(?=</a> )", RegexOptions.IgnoreCase)

        For i As Short = 0 To results.Count - 1
            Console.WriteLine(results.Item(i).ToString)

            LoginReq = HttpWebRequest.Create("(“HTTPS://mysite/logon/getads?AdID=" & results.Item(i).ToString)

            LoginReq.CookieContainer = New CookieContainer
            LoginReq.CookieContainer.Add(CookieCol)
            Dim response As HttpWebResponse = CType(LoginReq.GetResponse(), HttpWebResponse)

            'code to save the file
        Next


Now, I know I must send the session cookie that I received back from the first URL to the second URL and I thought I was doing it correctly However I’m getting this error:  

“The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.”

Any help would be great.
Thanks
Mike Colonna

colonna_mikeAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Bob LearnedCommented:
Mike,

Why is this format diffferent than the first call:

          LoginReq = HttpWebRequest.Create("(“HTTPS://mysite/logon/getads?AdID=" & results.Item(i).ToString)

Bob
0
colonna_mikeAuthor Commented:
the frist call is a POST which gets a session cookie and redirects me to a web which has the list of ads I need to download.


I hope I answer this right

MIke C.
0
colonna_mikeAuthor Commented:
soory I now see what your saying the first web site should be

"https://www.adtransit.com/login/checklogin.asp"
0
Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

Bob LearnedCommented:
Yeah, the first one was like this:

  LoginReq = HttpWebRequest.Create("HTTPS://mysite/logon")

So, the second one should be like this:

  LoginReq = HttpWebRequest.Create("HTTPS://mysite/logon/getads?AdID=" & results.Item(i).ToString)

Bob
0
colonna_mikeAuthor Commented:
the first one should be
  LoginReq = HttpWebRequest.Create"https://www.adtransit.com/login/checklogin.asp")

the second one should be

  LoginReq = HttpWebRequest.Create("HTTPS://mysite/logon/getads?AdID=" & results.Item(i).ToString)

again sorry.


Mike C.
0
colonna_mikeAuthor Commented:
OK I double check the url and if I get my head out of  “you know what” I might get it right below
Is the correct URL’s

the first one should be
  LoginReq = HttpWebRequest.Create"https://www.adtransit.com/login/checklogin.asp")

the second one should be

  LoginReq = HttpWebRequest.Create("HTTPS:// www.adtransit.com /logon/getads?AdID=" & results.Item(i).ToString)

again sorry.
0
Bob LearnedCommented:
Why are there spaces in the URL for #2?

Shouldn't it be:

LoginReq = HttpWebRequest.Create("HTTPS://www.adtransit.com/logon/getads?AdID=" & results.Item(i).ToString)

Bob
0
colonna_mikeAuthor Commented:
just a type "O"
0
Bob LearnedCommented:
Where are you now?  Are you still getting the same exception?

Bob
0
colonna_mikeAuthor Commented:
Yes bob,


the sample i supplied was incorrect but my code is right.

I check the cookie information and contacted the site i'm trying to download the files and they tell me the the information is correct so I'm loss. now i'm looking into the request header does the cookie information need to there ?
i don't know

any help would be great.



Thanks for your time.


0
Bob LearnedCommented:
Mike,

Does the first request that gets the cookies work?

  CookieCol = LoginReq.CookieContainer.GetCookies(LoginReq.RequestUri)

Bob
0
colonna_mikeAuthor Commented:
yes, I get the cookie and I'm then redirect to a web page were I can get a list of files that need to be downloaded


everything is good on the first call

thanks again
Mikee C
0
Bob LearnedCommented:
Thoughts:

1) http://www.dotnet247.com/247reference/msgs/48/243115.aspx

the webservice still works fine when using a non-secured channel.

2) Consuming Webservices over HTTPS (SSL)
http://weblogs.asp.net/jan/archive/2003/12/04/41154.aspx

3) Credentials

4) FIPS compliant mode on Windows 2003

http://www.codecomments.com/archive321-2005-2-388746.html

Bob


0
colonna_mikeAuthor Commented:

I try example 2 because I believe that the site I’m trying to use is using a cert server.

However I now get a different message ‘The remote server returned an error: (404) Not Found.”

Which is at lease different.

I added the code I try to use on the bottom thanks again maybe I just doing it wrong?

thansk again.


Sub Main()

        Try

            System.Net.ServicePointManager.CertificatePolicy = New TrustAllCertificatePolicy()


            logfile.WriteLine(vbTab & "logging on to : https://www.adtransit.com/login/checklogin.asp")
            logfile.WriteLine(vbTab & "")
            Dim domain As String = "https://www.adtransit.com/login/checklogin.asp"
            Post(domain)
            GetadList()


        Catch webEx As System.Net.WebException
            'handle exception...
            logfile.WriteLine("")
            logfile.WriteLine("Error : " & webEx.Message)
        End Try



    End Sub


Public Class TrustAllCertificatePolicy
        Implements System.Net.ICertificatePolicy

        Public Sub TrustAllCertificatePolicy()
        End Sub

        Public Function CheckValidationResult(ByVal srvPoint As System.Net.ServicePoint, ByVal certificate As X509Certificate, ByVal request As System.Net.WebRequest, ByVal certificateProblem As Integer) As Boolean Implements System.Net.ICertificatePolicy.CheckValidationResult
            Return True
        End Function


    End Class

0
Bob LearnedCommented:
You applied that to the URL that was working.  Did you try that with the one that wasn't (step 2)?

Bob
0
colonna_mikeAuthor Commented:
I try it for both the first one still worked and the second one is now giving me a error The remote server returned an error: (404) Not Found instead of the "Could not establish trust relationship for the SSL/TLS secure channel.”

MIke c
0
Bob LearnedCommented:
Mike,

Sometimes there comes along a question that starts out good, and then slowly I feel myself sinking slowly below the water.  I don't have too many ideas yet on how to deal with this, but I would be very interested to find a solution, since we use https:// alot here.

Bob
0
colonna_mikeAuthor Commented:
no probelm I am making some head way I'm pretty sure it has to do with cert.


thansk for your time.
0
Bob LearnedCommented:
Since our certificates on our servers have just expired yesterday, I am starting to get that exact same error message from my web services, so you might be right.

Bob
0
colonna_mikeAuthor Commented:
Ok I figure it out.

what I had to to is force application to accept the certificate.
below is what I did to make it work, maybe it will help you

here what you need to do :

paste this function into your program (vb.net) you will have to convert it to C# or what ever program lang. you are using.

<code>

 Function ValidateCertificate(ByVal sender As Object, ByVal certificate As X509Certificate, ByVal chain As X509Chain, ByVal sslPolicyErrors As SslPolicyErrors) As Boolean
        'Return True to force the certificate to be accepted.
        Return True
    End Function

<end code>


next you need to override the  ServerCertificateValidation,
to do that use the code below.

NOTE: you only need to make the call once for the life of your program


<code>
ServicePointManager.ServerCertificateValidationCallback = New RemoteCertificateValidationCallback(AddressOf ValidateCertificate)
<end code>






Mike C
.
0
Bob LearnedCommented:
Cool b-)  =:)

Glad you got it figured out.

Bob
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Visual Basic.NET

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.