Solved

Howto: automated data submission to a form page

Posted on 2004-03-23
5
295 Views
Last Modified: 2013-12-25
Is there a programming method I can use to go to a form page on the web (via a Perl script), enter some value like a zip code into a form field on that page (again via the script that is picking the inputs out of a text file, say), then when the result page is returned, retrieve it and parse it for data?  I know Perl and C, so the problem areas for me are (a) automatically submitting a piece of information to the form page (like zip code), and (b) retrieving the html of the result page so I can scrape it.  Thanks, hope that's clear.
0
Comment
Question by:mark_at_acc
  • 2
  • 2
5 Comments
 
LVL 3

Assisted Solution

by:rkosai
rkosai earned 250 total points
ID: 10663442
You would probably want to use the Perl LWP::Simple module.

Assuming the form uses a GET request (the results page has page.cgi?formvalue=data, or something like that), you can simply retrieve the page using the LWP module.  Example code is below:

#!/usr/bin/perl
use LWP::Simple;
my $html = get("http://website/page.cgi?zipcode=12345"); #download the site

#$html now contains the HTML code for the page.  You can use a regular expression to parse it.
0
 
LVL 48

Accepted Solution

by:
Tintin earned 250 total points
ID: 10663698
0
 

Author Comment

by:mark_at_acc
ID: 10663941
Both of you helped me, thanks a lot.  The page I care about today uses POST, so Tintin's Perl reference is actually more helpful, but rkosai's solution is clear, and can be helpful in the future.  I will be doing more of this type of web programming soon. Thanks a lot.

I am brand new to experts-exchange, and I CANNOT bookmark my questions!!  When I click "bookmark" at top left, I get nothing, with a note telling me how to click "add", etc, which I have done over and over.  I submitted the prob. to the EE bug page, but got no response yet.

Thanks again.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 10664106
As you'll soon discover the LWP bundle is great for doing all sorts of web type activities.  

Don't forget to read through the LWP tutorial

http://search.cpan.org/dist/libwww-perl/lwptut.pod

Not sure why you are having a problem with the bookmarks.  What browser are you using?
0
 

Author Comment

by:mark_at_acc
ID: 10664375
I figured it out -- appears to be user error. thanks anyhow.
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
disabled AD accounts info 3 69
.csv files 5 87
How can I modify this Excel worksheet with a script? 15 73
React or Angular? 6 52
If you get a (Blue Screen of Death), your system writes a small file called a minidump. Your first step is to make certain your computer is setup to record memory dumps. Right click My Computer, choose properties. Click on the advanced tab, an…
Making a simple AJAX shopping cart Couple years ago I made my first shopping cart, I used iframe and JavaScript, it was very good at that time, there were no sessions or AJAX, I used cookies on clients machine. Today we have more advanced techno…
In this fourth video of the Xpdf series, we discuss and demonstrate the PDFinfo utility, which retrieves the contents of a PDF's Info Dictionary, as well as some other information, including the page count. We show how to isolate the page count in a…
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now