Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Howto: automated data submission to a form page

Posted on 2004-03-23
5
Medium Priority
?
303 Views
Last Modified: 2013-12-25
Is there a programming method I can use to go to a form page on the web (via a Perl script), enter some value like a zip code into a form field on that page (again via the script that is picking the inputs out of a text file, say), then when the result page is returned, retrieve it and parse it for data?  I know Perl and C, so the problem areas for me are (a) automatically submitting a piece of information to the form page (like zip code), and (b) retrieving the html of the result page so I can scrape it.  Thanks, hope that's clear.
0
Comment
Question by:mark_at_acc
  • 2
  • 2
5 Comments
 
LVL 3

Assisted Solution

by:rkosai
rkosai earned 1000 total points
ID: 10663442
You would probably want to use the Perl LWP::Simple module.

Assuming the form uses a GET request (the results page has page.cgi?formvalue=data, or something like that), you can simply retrieve the page using the LWP module.  Example code is below:

#!/usr/bin/perl
use LWP::Simple;
my $html = get("http://website/page.cgi?zipcode=12345"); #download the site

#$html now contains the HTML code for the page.  You can use a regular expression to parse it.
0
 
LVL 48

Accepted Solution

by:
Tintin earned 1000 total points
ID: 10663698
0
 

Author Comment

by:mark_at_acc
ID: 10663941
Both of you helped me, thanks a lot.  The page I care about today uses POST, so Tintin's Perl reference is actually more helpful, but rkosai's solution is clear, and can be helpful in the future.  I will be doing more of this type of web programming soon. Thanks a lot.

I am brand new to experts-exchange, and I CANNOT bookmark my questions!!  When I click "bookmark" at top left, I get nothing, with a note telling me how to click "add", etc, which I have done over and over.  I submitted the prob. to the EE bug page, but got no response yet.

Thanks again.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 10664106
As you'll soon discover the LWP bundle is great for doing all sorts of web type activities.  

Don't forget to read through the LWP tutorial

http://search.cpan.org/dist/libwww-perl/lwptut.pod

Not sure why you are having a problem with the bookmarks.  What browser are you using?
0
 

Author Comment

by:mark_at_acc
ID: 10664375
I figured it out -- appears to be user error. thanks anyhow.
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I hope you'll find this tutorial useful and interesting. So let's try to extend Tcl with a new package.  For anyone more deeply interested please check out the book "Practical Programming in Tcl and Tk". It's really one of the best written books abo‚Ķ
This article will show, step by step, how to integrate R code into a R Sweave document
The viewer will learn the basics of jQuery, including how to invoke it on a web page. Reference your jQuery libraries: (CODE) Include your new external js/jQuery file: (CODE) Write your first lines of code to setup your site for jQuery.: (CODE)
Video by: Mark
This lesson goes over how to construct ordered and unordered lists and how to create hyperlinks.
Suggested Courses

927 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question