How to retrieve data from online form using winhttp when the onaction of the form is javascript?
I am attempting to use VB with a WinHTTP object to pull data from the following page:
http://www.usballoon.net/webstore/Search_Product.asp
When you do tihs manually, searching for "star", the resulting URL is
http://www.usballoon.net/webstore/Search_Result.asp?PageNumber=1&Search=star
However, using WinHTTP and the GET command only returns the intial search page. If i immediately turn around and try to navigate to the same URL, I do get the contents of the search page, but it retrieves all 8,000 or so items. It appears as though the "star" parameter is not being passed properly.
I have written several different versions of this code using both the GET command and the POST command, all with the same results. I have tried adding
m_objWinHTTP.SetRequestHea
m_objWinHTTP.Send "PageNumber=1&Search=star"
When using POST to no avail.
there appears to be 2 things operating here:
#1: Perhaps some kind of authentication is going on, which is why you have to attempt twice to get to the page.
#2: The OnClick() event of the search form is not a URL directly, but instead calls a Javascript function which in turns sets the Action of the form to the full URL with the query parameters at the end.
Any help would be appreciated.
Tim
http://www.usballoon.net/webstore/Search_Product.asp
When you do tihs manually, searching for "star", the resulting URL is
http://www.usballoon.net/webstore/Search_Result.asp?PageNumber=1&Search=star
However, using WinHTTP and the GET command only returns the intial search page. If i immediately turn around and try to navigate to the same URL, I do get the contents of the search page, but it retrieves all 8,000 or so items. It appears as though the "star" parameter is not being passed properly.
I have written several different versions of this code using both the GET command and the POST command, all with the same results. I have tried adding
m_objWinHTTP.SetRequestHea
m_objWinHTTP.Send "PageNumber=1&Search=star"
When using POST to no avail.
there appears to be 2 things operating here:
#1: Perhaps some kind of authentication is going on, which is why you have to attempt twice to get to the page.
#2: The OnClick() event of the search form is not a URL directly, but instead calls a Javascript function which in turns sets the Action of the form to the full URL with the query parameters at the end.
Any help would be appreciated.
Tim
Function BalloonSearch_EXP()
Const BALLOON_SITE As String = "http://www.usballoon.net/webstore/Search_Result.asp?PageNumber=1&Search=STAR"
Const HTTP_MAX_TIMEOUT As Long = 10000
Dim m_objWinHTTP As WinHttp.WinHttpRequest
Dim strContent As String
Dim strHTMLBody As String
10 On Error Resume Next
20 Set m_objWinHTTP = CreateObject("WinHttp.WinHttpRequest.5.1")
30 If Err Then
40 Err.Clear
50 Set m_objWinHTTP = CreateObject("WinHttp.WinHttpRequest.5")
60 If Err Then
70 Err.Clear
80 GoTo ErrorTrap
90 End If
100 End If
110 On Error GoTo ErrorTrap
'* Convert the values
120 m_objWinHTTP.SetTimeouts 75000, HTTP_MAX_TIMEOUT * 1000, 10000, 10000
130 m_objWinHTTP.Open "GET", BALLOON_SITE, False
140 m_objWinHTTP.Send
150 m_objWinHTTP.WaitForResponse 20000
160 strContent = m_objWinHTTP.ResponseText
170 MsgBox InStr(1, strContent, "star", vbTextCompare)
180 MsgBox "Status: " & m_objWinHTTP.StatusText
190 Debug.Print strContent
'* Second time
200 m_objWinHTTP.SetTimeouts 75000, HTTP_MAX_TIMEOUT * 1000, 10000, 10000
210 m_objWinHTTP.Open "GET", BALLOON_SITE, False
220 m_objWinHTTP.Send
230 m_objWinHTTP.WaitForResponse 90000
240 strContent = m_objWinHTTP.ResponseText
250 Debug.Print strContent
260 MsgBox InStr(1, strContent, "star", vbTextCompare)
270 MsgBox "Status: " & m_objWinHTTP.StatusText
290 Exit Function
SafeExit:
300 Return
ErrorTrap:
'* Code here if desired
310 MsgBox Erl & vbCrLf & Err.Description
320 Err.Clear
End Function
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
You graded my solution with a grade B!
Please explain why did you grade with a grade B.
Please explain why did you grade with a grade B.
Oh, now I see that you stated a comment with your grading. I did not see it before.
OK, but same for you, you did not ask for explanation and I got no chance to explain the solution to you.
OK, but same for you, you did not ask for explanation and I got no chance to explain the solution to you.
Now what you can do, if you want to change the grading is to ask Community Support to reopen this question. Post a request here: http:/General/newQuestionWizard.jsp
Place also a link to this question or at least the question number in your reopen request.
Place also a link to this question or at least the question number in your reopen request.
ASKER
Zvonko:
I posted a question to the Suppport area. I have copied it below. I'd be happy to change the grade if you could possibly explain the solution a bit more. Thanks!!!
==========================
Hello.
I asked a question and it was answered: https://www.experts-exchange.com/questions/22994114/How-to-retrieve-data-from-online-form-using-winhttp-when-the-onaction-of-the-form-is-javascript.html?anchorAnswerId=20390689#a20390689.
I was thrilled to have a solution, but no explanation was given. although it was literally only one line of code, I really didn't understand it. I told the expert that I was going to give him the full 500 points regardless, but that I would appreciate further comment on how the soltuion works. Before receiving any answer I awarded the 500 points, but I only gave him a "B" because there was no explanation.
If he provides further explanation, I'd like to give him an A. However, at this point he is grumbling because I gave him a B. I think was grade was fair, especially since I gave him the full 500 points.
If he should finally provide more information, can I re-assign his grade to an A?
tim Jeffryes
I posted a question to the Suppport area. I have copied it below. I'd be happy to change the grade if you could possibly explain the solution a bit more. Thanks!!!
==========================
Hello.
I asked a question and it was answered: https://www.experts-exchange.com/questions/22994114/How-to-retrieve-data-from-online-form-using-winhttp-when-the-onaction-of-the-form-is-javascript.html?anchorAnswerId=20390689#a20390689.
I was thrilled to have a solution, but no explanation was given. although it was literally only one line of code, I really didn't understand it. I told the expert that I was going to give him the full 500 points regardless, but that I would appreciate further comment on how the soltuion works. Before receiving any answer I awarded the 500 points, but I only gave him a "B" because there was no explanation.
If he provides further explanation, I'd like to give him an A. However, at this point he is grumbling because I gave him a B. I think was grade was fair, especially since I gave him the full 500 points.
If he should finally provide more information, can I re-assign his grade to an A?
tim Jeffryes
ASKER
Zvonko:
Support has responded and they say I can change the grade. If you can provde some explanation as to the questions I asked, I will be more than happy to change the grade to an A.
Sorry for the confusion.
tim
Support has responded and they say I can change the grade. If you can provde some explanation as to the questions I asked, I will be more than happy to change the grade to an A.
Sorry for the confusion.
tim
Request posted for grade change with additional explanation: https://www.experts-exchange.com/questions/22996162/Working-Might-possibly-need-to-change-a-grade.html
Vee_Mod
Experts Exchange Moderator
Vee_Mod
Experts Exchange Moderator
OK, the explanation.
First of all, when you are a server side scripting programmer you are happy your dynamic pages are behaving at least as you expect and not hope that there is one general rule how they behave in any circumstance.
The upper usbaloon site expects a session to be created for a browser user. The session key is stored in a cookie. You can observe that even when you enter your URL in a browser address bar. You will see that even you enter the expected URL of the search result page you have to send the request for a second time. But... you cannot observe it without a tool because you cannot execute the script or submit the form without click on that "Submit Search" button.
I can do that because I have a good friend here on EE in JavaScript topic area: devic
He created a wonderful tool to look into a browser page and do scripting in the target server page. Look into his profile to get that tool: PageSpy
When you look into usbaloon page you can see that a cookie is created for the session and you can submit the form by script like this placed in PageSpy last tab labeled Run:
var theForm = document.forms[0];
theForm.action = "Search_Result.asp?PageNum
document.cookie="ASPSESSIO
theForm.submit();
The problem is that you have to use an existing session ID in your cookie. It worked in your test because you used my session ID in the time frame as long as my session with usbaloon lasted.
That was the explanation.
Now you will ask: How to get that ID for my session?
The answer: Open a new Question ;-)
Is that explanation ok for you for an A+ ;-)
First of all, when you are a server side scripting programmer you are happy your dynamic pages are behaving at least as you expect and not hope that there is one general rule how they behave in any circumstance.
The upper usbaloon site expects a session to be created for a browser user. The session key is stored in a cookie. You can observe that even when you enter your URL in a browser address bar. You will see that even you enter the expected URL of the search result page you have to send the request for a second time. But... you cannot observe it without a tool because you cannot execute the script or submit the form without click on that "Submit Search" button.
I can do that because I have a good friend here on EE in JavaScript topic area: devic
He created a wonderful tool to look into a browser page and do scripting in the target server page. Look into his profile to get that tool: PageSpy
When you look into usbaloon page you can see that a cookie is created for the session and you can submit the form by script like this placed in PageSpy last tab labeled Run:
var theForm = document.forms[0];
theForm.action = "Search_Result.asp?PageNum
document.cookie="ASPSESSIO
theForm.submit();
The problem is that you have to use an existing session ID in your cookie. It worked in your test because you used my session ID in the time frame as long as my session with usbaloon lasted.
That was the explanation.
Now you will ask: How to get that ID for my session?
The answer: Open a new Question ;-)
Is that explanation ok for you for an A+ ;-)
One more thing: the session and session ID is needed on usbaloon to maintain your ShoppingCart
ASKER
Zvonko:
Thanks very much for your hellp. I have put in a request to change your grade to an A.
Tim
Thanks very much for your hellp. I have put in a request to change your grade to an A.
Tim
ASKER
Zvonko:
Thanks very much. I have put in a requst to change your grade from a B to an A.
tim
Thanks very much. I have put in a requst to change your grade from a B to an A.
tim
ASKER
thanks for all your help.
I'd still like to know how to acquire the session ID because I'm still essentially without a soltution. However, this has enabled me to progress with my project and I thank you very much.
Basically what I'm trying to do is pull the search results from the balloon site and post them onto another site. I have no need of the shopping cart, but I would like to be able to get the session ID.
Tim
I'd still like to know how to acquire the session ID because I'm still essentially without a soltution. However, this has enabled me to progress with my project and I thank you very much.
Basically what I'm trying to do is pull the search results from the balloon site and post them onto another site. I have no need of the shopping cart, but I would like to be able to get the session ID.
Tim
Can you please open a new question for better readability and for other experts to participate.
In that new question state one single question. The question cannot be to get a working solution. Question is always a request for answer for the topic you do not understand. So state what you have understood so far and what you next question is, but please only one question.
Also keep in mind that EE will not help you to fetch other domains content. I can help you to understand technical stuff like reading and setting cookies, what you do with that knowledge is out of EE scope, you see?
In that new question state one single question. The question cannot be to get a working solution. Question is always a request for answer for the topic you do not understand. So state what you have understood so far and what you next question is, but please only one question.
Also keep in mind that EE will not help you to fetch other domains content. I can help you to understand technical stuff like reading and setting cookies, what you do with that knowledge is out of EE scope, you see?
ASKER
You truly are a genius and this works great! Thanks so much.
However, I don't understand. I guess there's an awful lot about the winHTTP object that I don't fully understand and I've been using it quite a bit these last 2 years. I'm definitely awarding you all 500 points, but can you clue me in a bit as to how the solution works? I'm somehwat familiar with what a cookie is, but have NO IDEA what "ASPSESSIONIDQATDDDTB=HOFD
More importantly, why is this line of code necessary with this website when I've never needed it before in applications using winHTTP? Is it because of the Javascript?
Sorry for so many questions. If you have an online resource to point me to, that would be sufficient - either that or just your own explanation.
Thanks again. Any further input you can add would be awesome.
Tim