?
Solved

WinHTTP, Frames, and SID - problems collecting data from site.

Posted on 2004-03-31
3
Medium Priority
?
850 Views
Last Modified: 2007-12-19
I need help with WinHTTP.  I am trying to pull tote board data from UsOfftrack.com.  from the url http://65.116.75.195/link2tote/main.aspx you can navigate to relevant data for tracks in the us.

I can get to programs.aspx fine, but when I try to go to data.aspx. it says my session has expired.  Using IE web Browser, I noticed a funny SID inserted in the path.  I checked the source, but nada.  As a test I tried typing in the url for track data that would normally go in the Data Frame (Center Screen).  It would always revert back to main.aspx.  I tried right clicking a link from the Programs Frame to open in a new window.  It worked.  Great, but why can I not do this in WinHTTP.  Opps!  Spoke too soon.  The page self refreshed to an error page.  I did the previous steps again to get to a data page and quickly right clicked to view source, No refresh tag.  How is that possible?

Anyway, can someone look at this site and tell me how to use the WinHTTP object to get data from each frame, Programs and Data Frames.  Also, where is that SID coming from and how do I capture to us it.

I would like to build a Server COM Component to collect this data routinely.
0
Comment
Question by:manganet2
  • 2
3 Comments
 
LVL 14

Accepted Solution

by:
CyrexCore2k earned 2000 total points
ID: 10728520
here:

Private Sub GetPages()
Dim myHTTPObj As New WinHttp.WinHttpRequest
Dim Location$
Dim i As Integer
With myHTTPObj
    .Option(WinHttpRequestOption_EnableRedirects) = False
    .Open "GET", "http://65.116.75.195/link2tote/main.aspx", True
    .SetRequestHeader "Accept", "*/*"
    .SetRequestHeader "Accept-Language", "en-us"
    .SetRequestHeader "Accept-Encoding", "gzip, deflate"
    .SetRequestHeader "User-Accept", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)"
    .SetRequestHeader "Host", "65.116.75.195"
    .SetRequestHeader "Connection", "Keep-Alive"
    .Send
End With
Do While Not getHeaders(myHTTPObj)
    DoEvents
Loop
Location = myHTTPObj.GetResponseHeader("Location")
Debug.Print Location
For i = Len(Location) To 1 Step -1
    If Mid(Location, i, 1) = "/" Then
        Location = Mid(Location, 1, i)
        Exit For
    End If
Next
With myHTTPObj
    .Option(WinHttpRequestOption_EnableRedirects) = False
    .Open "GET", "http://65.116.75.195" & Location & "main.aspx", True
    .SetRequestHeader "Accept", "*/*"
    .SetRequestHeader "Accept-Language", "en-us"
    .SetRequestHeader "Accept-Encoding", "gzip, deflate"
    .SetRequestHeader "User-Accept", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)"
    .SetRequestHeader "Host", "65.116.75.195"
    .SetRequestHeader "Connection", "Keep-Alive"
    .Send
End With
Do While Not getResponse(myHTTPObj)
    DoEvents
Loop
'You now have the main page souce in the myhttpobj.responsetext property
Debug.Print myHTTPObj.ResponseText
With myHTTPObj
    .Option(WinHttpRequestOption_EnableRedirects) = False
    .Open "GET", "http://65.116.75.195" & Location & "data.aspx", True
    .SetRequestHeader "Accept", "*/*"
    .SetRequestHeader "Referer", "http://65.116.75.195/" & Location & "main.aspx"
    .SetRequestHeader "Accept-Language", "en-us"
    .SetRequestHeader "Accept-Encoding", "gzip, deflate"
    .SetRequestHeader "User-Accept", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705)"
    .SetRequestHeader "Host", "65.116.75.195"
    .SetRequestHeader "Connection", "Keep-Alive"
    .Send
End With
Do While Not getResponse(myHTTPObj)
    DoEvents
Loop
'You now have the data page souce in the myhttpobj.responsetext property
Debug.Print myHTTPObj.ResponseText

End Sub

Function getHeaders(tmpHTTPObj As WinHttp.WinHttpRequest) As Boolean
On Error Resume Next
    getHeaders = CBool(Len(tmpHTTPObj.GetAllResponseHeaders))
End Function

Function getResponse(tmpHTTPObj As WinHttp.WinHttpRequest) As Boolean
On Error Resume Next
    getResponse = CBool(Len(tmpHTTPObj.ResponseText))
End Function
0
 

Author Comment

by:manganet2
ID: 10735854
Thanks for you post, but I think you need to look a little deeper.  The challange I am having is when you grab a link from the Programs.aspx, ie

.Open "GET", "http://65.116.75.195" & Location & "data.aspx?ProgramKey=20332", True

This would normally update data.aspx to point to the track indicated, but we are not using frames.

The response is

First Get Using - .Open "GET", "http://65.116.75.195" & Location & "main.aspx", True

Location:  /link2tote/(agxdti2irjsjzrewv5elk2rc)/main.aspx

Second Get using - .Open "GET", "http://65.116.75.195" & Location & "data.aspx?ProgramKey=20332", True

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href='/link2tote/(agxdti2irjsjzrewv5elk2rc)/reload.htm'>here</a>.</h2>
</body></html>

Third Get using - .Open "GET", "http://65.116.75.195" & Location & "reload.htm", True

<html>
    <body onload="top.window.location='../main.aspx'">
    <h5>Your session has expired.  Reloading main page...</h5>
    </body>
</html>

If I were in the IE Browser UI, it would just update the Data Frame and you could see the data I am after.  It is not as simple programatically for some reason.
0
 

Author Comment

by:manganet2
ID: 10735923
Found this in the Programs.aspx.

<form name="frmMain" method="post" action="programs.aspx?Sport=2" id="frmMain"><input type="hidden" name="__VIEWSTATE"
value="dDw0NDI3MzU5MzA7Oz79KTDBOOr8l6zAiHVqTvyFR1M/yg==" />

This may be needed to maintain the session.  Don't Know!!
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction In a recent article (http://www.experts-exchange.com/A_7811-A-Better-Concatenate-Function.html) for the Excel community, I showed an improved version of the Excel Concatenate() function.  While writing that article I realized that no o…
Most everyone who has done any programming in VB6 knows that you can do something in code like Debug.Print MyVar and that when the program runs from the IDE, the value of MyVar will be displayed in the Immediate Window. Less well known is Debug.Asse…
As developers, we are not limited to the functions provided by the VBA language. In addition, we can call the functions that are part of the Windows operating system. These functions are part of the Windows API (Application Programming Interface). U…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…
Suggested Courses

839 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question