Submitting data to web cgi script and parsing results

My company has many products on a popular site.  Not sure I want to give the name here, but may have to eventually.  Anyway, due to technical problems from a third party they use for processing orders many of our products come down from their site because of us being out of stock, even though we aren't.  Long story short, we need to constantly search their site for our products to make sure they are up.  

I am assuming I must use their search somehow.  They use the following cgi script link from a form on their site (url changed to hide site name).

So basically from within access (preferably) or c++ I would like to search for each of our products that are on their site from our products library (in access) and see if they are up on the site.  They use a cgi script to accept data from a form.

Here is the code from their site.


<TABLE WIDTH="100&#37" BORDER="0" CELLSPACING="0" CELLPADDING="0" ID="searchBar">  
<TD WIDTH="71">
<IMG SRC="" WIDTH="71" HEIGHT="28">
<TD WIDTH="200" CLASS="searchBoxHolder">  
<INPUT NAME="page" TYPE="hidden" VALUE="search">  
<INPUT NAME="keyword" TYPE="text" ID="keyword" onClick=this.value=""; VALUE="Search Keyword or Model #" SIZE="30">

I had only found 1 question dealing with this using xmlhttp but couldn't get it to work with their cgi script because I didn't know how to pass the value.

Thanks in advance.  After I get this working, I will post another question regarding parsing the data.

Who is Participating?
The form uses a GET action, so it's going to actually be pretty easy.  I'm not sure that you can do it easily in Access, but Visual C++ makes it pretty simple.

The URL would be like:<my search term>

You can use the CHttpConnection and CHttpFile classes (in afxinet.h) to download this pretty easily.  There's the WININET/WINHTTP stuff, but the MFC classes are a little easier.  It would look something like:
    CInternetSession is("Downloader");
      CHttpConnection *http = is.GetHttpConnection("");

      CHttpFile *web = http->OpenRequest("GET", "/cgi-bin/d2.cgi?page=search&keyword=<my search term>");

      // Get the data
      CString line;
      CString data;
        data += line;

      // Cleanup
      delete web;
      delete http;
    catch(CInternetException *e)
CSBTechAuthor Commented:
I will give that a shot and get back to you.  Thanks.
CSBTechAuthor Commented:
Is this code only for visual c++?  I can't find the afxinet.h file and everything I find points to it being a visual c++ file which I don't have.  I am using dev C++.  Is this a library I can download or is this a vc++ file only?

Cloud Class® Course: Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

As far as i know, MFC is vc++. I'm not sure if dev c++ has anything for MFC. My suggestion is that you just create your own class for submitting the form data manually.

You can use the URLDownloadToFile function in urlmon as well, then read the file within your code.  You could even do this one in Access, like:

Private Declare Function URLDownloadToFile Lib "urlmon" Alias _
"URLDownloadToFileA" (ByVal pCaller As Long, ByVal szURL As String, ByVal _
szFileName As String, ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long

Private Sub Command0_Click()
  URLDownloadToFile 0, "http://myurl", "c:\local_file.html", 0, 0
End Sub
CSBTechAuthor Commented:
The access version worked great and I prefer it in access anyway.  However, I need to parse out the code that is returned (I will handle this in another question shortly if I don't figure it out).  

So my question is, can this be changed to not save the file, but rather assign it to a variable or something else so I can process the file and strip out what I need?
No.  If you want to do that, you'll have to read the data directly off of the socket.  That would be closer to the earlier example.  There's a WinHTTP library that Microsoft has with the platform SDK that you could use in Dev-CPP or Access (with a lot of declare function statements), but it will be much more of a pain.  You could also just create a socket connection to port 80 on the server and send the GET function with the HTTP headers, which would not be a big deal either.  But it's probably a lot easier just to save it and read in the file (which you could do in Access).
CSBTechAuthor Commented:

Let me explain why I am avoiding the saving thing.

I basically have to check each of our products so see if they are on the site or not.  So I have to perform the function say 200 times, each time getting the results and parsing out if it is out of stock or not from the page, then generating some type of report listing each product and the result of that products search.

So, which would you recommend, what would you suggest, and since I am unfamiliar with this, can you get me started?

I really appreciate the help.

Thanks, John
If you're doing it in Access, the URLDownloadToFile will definitely be your safer bet.  If you'll be calling it from Access, then chances are you'd have to save it to a file so that Access got the data back anyway (unless you're planning on writing an ActiveX control).  It's really up to you.  We might be able to get you started, but ultimately you'll have to finish the thing and maintain it.  If you're not even the least bit comfortable with C++, but you're comfortable with VBA, then I'd go with the URLDownloadToFile.  What do you think?
CSBTechAuthor Commented:
Well, I am familiar with C++ by not an expert.  If it is very complex, then you are correct.  As for Access, if there is no other way, then I guess I will have to do the query, save the file, open the file, parse the file, then perform the next query for each product.  Just seems like a lot of processing of opening and closing files.  I guess I will go with the Access and see how it works.  If problems arise in the processing I will revert to another option.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.