Read Data Text from an external web page

Hi, I'm developing a website which transfer data to another website (website B). The website B, process that data and gives them status. Each registry has a reference.

If I want to know the status, I can go into that website (no user validation required) put the reference in a query page and submit, and in the next page, the status shows up. I need to make this automatically, I mean, from my website make a submit of that external query page and in the results page, read the values.

I suppose I can do that with XMLHTTPRequest, buy I'm not sure how to, can someone help me ?

Thanks in advance

Regards
LoThar_71Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

mk_bCommented:
You can use this to post/get quite useful.

    Dim strValues, strURL
    Dim xmlhttp

    strURL = "http://www.google.com"
    strValues = "q=test"

   set objHttp = Server.CreateObject("Msxml2.ServerXMLHTTP")
   ' set objHttp = Server.CreateObject("Msxml2.ServerXMLHTTP.4.0")
   ' set objHttp = Server.CreateObject("Microsoft.XMLHTTP")
   objHttp.open "get", strURL, false
   objHttp.setRequestHeader "Content-type", "application/x-www-form-urlencoded"
   objHttp.Send strValues


    If objHttp.Status >= 400 And objHttp.Status <= 599 Then
        Response.Write "Error Occurred : " & objHttp.Status & " - " & objHttp.statusText
    Else
        Response.Write objHttp.ResponseText
    End If

    set objHttp = nothing


../mk
0
LoThar_71Author Commented:
That's fine, but that script will only show the "website B". Taking "google.com" as website B, I will need to make a search, read the results and save 1 specific field value (let's say.. quantity of coincidences) in a variable.

I know there are websites wich reads online data from others (stock exchange, price lists, etc.) with the purpose to offer resumed information. They are doing what I need.
0
mk_bCommented:
Well if you know what you are looking for from website B you can put objHttp.ResponseText into a string and then look for what you want in the string and then do some string manipulation??
0
Cloud Class® Course: CompTIA Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

LoThar_71Author Commented:
mk you are missing something. Website B is "google.com". My data is in the search results page. Suppose that I'm looking for "imported cars", I have to submit google with that words and get, from the results page, how many coincidences found it.

About string manipulation... if it's the only way... I will do it.

0
LoThar_71Author Commented:
Let me add some data.

When I execute your code, I do not see the results page, I see the main google page.

The sentence - objHttp.Send strValues -, I suppose, is passing the search parameters, isn't it ? If that it's right, something is not working, because only shows the main google page (not even an error).


0
mk_bCommented:
For some reason google is giving me a 403 forbiddien error here are too example of subitting the for the request? you will need to look at the google.com page and see if there are any other paramiters that need to be passes to seach and maybe run a search on "scraping google". the second example is for internic and will give the domin registration details for a site this one works fine? just play around witht the google im sure you will get it working?


1.
<%

    Dim strValues, strURL
    Dim xmlhttp

    strURL = "http://www.google.com/search?"
    strValues = "hl=en&q=test&q=imported cars&btnG=Google+Search"

   set objHttp = Server.CreateObject("Msxml2.ServerXMLHTTP")
   ' set objHttp = Server.CreateObject("Msxml2.ServerXMLHTTP.4.0")
   ' set objHttp = Server.CreateObject("Microsoft.XMLHTTP")
   objHttp.open "get", strURL, false
   objHttp.setRequestHeader "Content-type", "application/x-www-form-urlencoded"
   objHttp.Send strValues


    If objHttp.Status >= 400 And objHttp.Status <= 599 Then
        Response.Write "Error Occurred : " & objHttp.Status & " - " & objHttp.statusText
    Else
        htpStr = objHttp.ResponseText
    End If

    set objHttp = nothing
%>
<% =htpStr %>


<br><br>2.
<%
Set objHttp = Server.CreateObject("MSXML2.ServerXMLHTTP")
objHttp.open "GET", "http://www-whois.internic.net/cgi/whois?whois_nic=www.internetsite.com&type=domain", False
objHttp.SetRequestHeader "Content-type", "application/x-www-form-urlencoded"
objHttp.send str

if (objHttp.status <> 200 ) then
'--HTTP error handling
elseif (objHttp.responseText = "VERIFIED") then
elseif (objHttp.responseText = "INVALID") then
else
'--error
end if

MyString = objHttp.responseText
%>
<% =MyString %>
0
LoThar_71Author Commented:
MK, none of them work. If I try with google, the best what I can get is the main google page, never searchs what I tell it. And the second, always give me a timeout.

Suppose, the target page, for example the search results of Google, works with post instead get. How could I find out the variables needed ?
0
mk_bCommented:
for me the first one work fine and the google one gives me a forbidden error?

have a look at this site(http://www.tss2000.nl/site_pages/index.php?CV=1) and see the google search on the right side. it is a custome search results from google also have a look around goole's site for how you do that? and then you might be able to use that with the script above to get someting working?

i think google might be checking where the request for a search is coming from and if its not from google it wont alow it???
0
SquareHeadCommented:
This works for me:


1st page, "search_google.html"
2nd page, "scrape_google.asp"


Search:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
      <title>Untitled</title>
</head>

<body>
<form action="scrape_google.asp" method="post" name="frm1" id="frm1">

<input type="text" name="txtSearchPhrase"><input type="submit" value="Go!">


</form>


</body>
</html>



Scrape:

<%@ Language = VBScript %>
<%

Response.Buffer = True
Dim objXMLHTTP, xml

' Create an xmlhttp object:
Set xml = Server.CreateObject("Microsoft.XMLHTTP")
' Or, for version 3.0 of XMLHTTP, use:
' Set xml = Server.CreateObject("MSXML2.ServerXMLHTTP")

' Opens the connection to the remote server.
xml.Open "GET", "http://www.google.com/search?hl=en&q=" & request.form("txtSearchPhrase"), False
   
 ' Actually Sends the request and returns the data:
xml.Send

Response.Write xml.responseText
 
  Set xml = Nothing
%>
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
LoThar_71Author Commented:
MK, the website you mention has that google search because they have an arrangement with Google. Probably they have some publicity or something with them. A couple of month ago I wanted to hire that search service (works into your page too) and if I buy a package of publicity they give me that search for free (haha, nothing is free).

Squarehead, your code works fine.

But, suppose google use the POST method instead GET, Do I have a way to find out the variables name to pass it on submit ?

0
SquareHeadCommented:
Well, the GET should do the trick, but if you need to POST, check out the in-depth info in this PAQ:

http://www.experts-exchange.com/Web/Web_Languages/XML/Q_21043023.html

0
SquareHeadCommented:
I misunderstood your last question...

If Google POSTs values, you may be able to determine the fields being POSTed by viewing the page source, but you won't see any values in there unless they are prepopulated when the page loads. Depending on what you click to submit the form, you may be able to determine values by looking at the code for the clickable item...
0
LoThar_71Author Commented:
Yes, thanks, that's all I need.

0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
ASP

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.