Solved

WinHTTP, Frames, and SID - problems collecting data from site.

Posted on 2004-03-31
3
761 Views
Last Modified: 2007-12-19
I need help with WinHTTP.  I am trying to pull tote board data from UsOfftrack.com.  from the url http://65.116.75.195/link2tote/main.aspx you can navigate to relevant data for tracks in the us.

I can get to programs.aspx fine, but when I try to go to data.aspx. it says my session has expired.  Using IE web Browser, I noticed a funny SID inserted in the path.  I checked the source, but nada.  As a test I tried typing in the url for track data that would normally go in the Data Frame (Center Screen).  It would always revert back to main.aspx.  I tried right clicking a link from the Programs Frame to open in a new window.  It worked.  Great, but why can I not do this in WinHTTP.  Opps!  Spoke too soon.  The page self refreshed to an error page.  I did the previous steps again to get to a data page and quickly right clicked to view source, No refresh tag.  How is that possible?

Anyway, can someone look at this site and tell me how to use the WinHTTP object to get data from each frame, Programs and Data Frames.  Also, where is that SID coming from and how do I capture to us it.

I would like to build a Server COM Component to collect this data routinely.
0
Comment
Question by:manganet2
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 14

Accepted Solution

by:
CyrexCore2k earned 500 total points
ID: 10728520
here:

Private Sub GetPages()
Dim myHTTPObj As New WinHttp.WinHttpRequest
Dim Location$
Dim i As Integer
With myHTTPObj
    .Option(WinHttpRequestOption_EnableRedirects) = False
    .Open "GET", "http://65.116.75.195/link2tote/main.aspx", True
    .SetRequestHeader "Accept", "*/*"
    .SetRequestHeader "Accept-Language", "en-us"
    .SetRequestHeader "Accept-Encoding", "gzip, deflate"
    .SetRequestHeader "User-Accept", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)"
    .SetRequestHeader "Host", "65.116.75.195"
    .SetRequestHeader "Connection", "Keep-Alive"
    .Send
End With
Do While Not getHeaders(myHTTPObj)
    DoEvents
Loop
Location = myHTTPObj.GetResponseHeader("Location")
Debug.Print Location
For i = Len(Location) To 1 Step -1
    If Mid(Location, i, 1) = "/" Then
        Location = Mid(Location, 1, i)
        Exit For
    End If
Next
With myHTTPObj
    .Option(WinHttpRequestOption_EnableRedirects) = False
    .Open "GET", "http://65.116.75.195" & Location & "main.aspx", True
    .SetRequestHeader "Accept", "*/*"
    .SetRequestHeader "Accept-Language", "en-us"
    .SetRequestHeader "Accept-Encoding", "gzip, deflate"
    .SetRequestHeader "User-Accept", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)"
    .SetRequestHeader "Host", "65.116.75.195"
    .SetRequestHeader "Connection", "Keep-Alive"
    .Send
End With
Do While Not getResponse(myHTTPObj)
    DoEvents
Loop
'You now have the main page souce in the myhttpobj.responsetext property
Debug.Print myHTTPObj.ResponseText
With myHTTPObj
    .Option(WinHttpRequestOption_EnableRedirects) = False
    .Open "GET", "http://65.116.75.195" & Location & "data.aspx", True
    .SetRequestHeader "Accept", "*/*"
    .SetRequestHeader "Referer", "http://65.116.75.195/" & Location & "main.aspx"
    .SetRequestHeader "Accept-Language", "en-us"
    .SetRequestHeader "Accept-Encoding", "gzip, deflate"
    .SetRequestHeader "User-Accept", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)"
    .SetRequestHeader "Host", "65.116.75.195"
    .SetRequestHeader "Connection", "Keep-Alive"
    .Send
End With
Do While Not getResponse(myHTTPObj)
    DoEvents
Loop
'You now have the data page souce in the myhttpobj.responsetext property
Debug.Print myHTTPObj.ResponseText

End Sub

Function getHeaders(tmpHTTPObj As WinHttp.WinHttpRequest) As Boolean
On Error Resume Next
    getHeaders = CBool(Len(tmpHTTPObj.GetAllResponseHeaders))
End Function

Function getResponse(tmpHTTPObj As WinHttp.WinHttpRequest) As Boolean
On Error Resume Next
    getResponse = CBool(Len(tmpHTTPObj.ResponseText))
End Function
0
 

Author Comment

by:manganet2
ID: 10735854
Thanks for you post, but I think you need to look a little deeper.  The challange I am having is when you grab a link from the Programs.aspx, ie

.Open "GET", "http://65.116.75.195" & Location & "data.aspx?ProgramKey=20332", True

This would normally update data.aspx to point to the track indicated, but we are not using frames.

The response is

First Get Using - .Open "GET", "http://65.116.75.195" & Location & "main.aspx", True

Location:  /link2tote/(agxdti2irjsjzrewv5elk2rc)/main.aspx

Second Get using - .Open "GET", "http://65.116.75.195" & Location & "data.aspx?ProgramKey=20332", True

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href='/link2tote/(agxdti2irjsjzrewv5elk2rc)/reload.htm'>here</a>.</h2>
</body></html>

Third Get using - .Open "GET", "http://65.116.75.195" & Location & "reload.htm", True

<html>
    <body onload="top.window.location='../main.aspx'">
    <h5>Your session has expired.  Reloading main page...</h5>
    </body>
</html>

If I were in the IE Browser UI, it would just update the Data Frame and you could see the data I am after.  It is not as simple programatically for some reason.
0
 

Author Comment

by:manganet2
ID: 10735923
Found this in the Programs.aspx.

<form name="frmMain" method="post" action="programs.aspx?Sport=2" id="frmMain"><input type="hidden" name="__VIEWSTATE"
value="dDw0NDI3MzU5MzA7Oz79KTDBOOr8l6zAiHVqTvyFR1M/yg==" />

This may be needed to maintain the session.  Don't Know!!
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When trying to find the cause of a problem in VBA or VB6 it's often valuable to know what procedures were executed prior to the error. You can use the Call Stack for that but it is often inadequate because it may show procedures you aren't intereste…
You can of course define an array to hold data that is of a particular type like an array of Strings to hold customer names or an array of Doubles to hold customer sales, but what do you do if you want to coordinate that data? This article describes…
Get people started with the process of using Access VBA to control Outlook using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Microsoft Outlook. Using automation, an Access applic…
Get people started with the process of using Access VBA to control Excel using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Excel. Using automation, an Access application can laun…

735 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question