Solved

Howto: automated data submission to a form page

Posted on 2004-03-23
5
299 Views
Last Modified: 2013-12-25
Is there a programming method I can use to go to a form page on the web (via a Perl script), enter some value like a zip code into a form field on that page (again via the script that is picking the inputs out of a text file, say), then when the result page is returned, retrieve it and parse it for data?  I know Perl and C, so the problem areas for me are (a) automatically submitting a piece of information to the form page (like zip code), and (b) retrieving the html of the result page so I can scrape it.  Thanks, hope that's clear.
0
Comment
Question by:mark_at_acc
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
5 Comments
 
LVL 3

Assisted Solution

by:rkosai
rkosai earned 250 total points
ID: 10663442
You would probably want to use the Perl LWP::Simple module.

Assuming the form uses a GET request (the results page has page.cgi?formvalue=data, or something like that), you can simply retrieve the page using the LWP module.  Example code is below:

#!/usr/bin/perl
use LWP::Simple;
my $html = get("http://website/page.cgi?zipcode=12345"); #download the site

#$html now contains the HTML code for the page.  You can use a regular expression to parse it.
0
 
LVL 48

Accepted Solution

by:
Tintin earned 250 total points
ID: 10663698
0
 

Author Comment

by:mark_at_acc
ID: 10663941
Both of you helped me, thanks a lot.  The page I care about today uses POST, so Tintin's Perl reference is actually more helpful, but rkosai's solution is clear, and can be helpful in the future.  I will be doing more of this type of web programming soon. Thanks a lot.

I am brand new to experts-exchange, and I CANNOT bookmark my questions!!  When I click "bookmark" at top left, I get nothing, with a note telling me how to click "add", etc, which I have done over and over.  I submitted the prob. to the EE bug page, but got no response yet.

Thanks again.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 10664106
As you'll soon discover the LWP bundle is great for doing all sorts of web type activities.  

Don't forget to read through the LWP tutorial

http://search.cpan.org/dist/libwww-perl/lwptut.pod

Not sure why you are having a problem with the bookmarks.  What browser are you using?
0
 

Author Comment

by:mark_at_acc
ID: 10664375
I figured it out -- appears to be user error. thanks anyhow.
0

Featured Post

Webinar: Aligning, Automating, Winning

Join Dan Russo, Senior Manager of Operations Intelligence, for an in-depth discussion on how Dealertrack, leading provider of integrated digital solutions for the automotive industry, transformed their DevOps processes to increase collaboration and move with greater velocity.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Script refinement question. 1 30
Need to combine two scripts 2 40
PS call executable Files in Order 4 31
how to customize the text in the legend in highcharts 3 31
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
The Windows functions GetTickCount and timeGetTime retrieve the number of milliseconds since the system was started. However, the value is stored in a DWORD, which means that it wraps around to zero every 49.7 days. This article shows how to solve t…
Learn the basics of lists in Python. Lists, as their name suggests, are a means for ordering and storing values. : Lists are declared using brackets; for example: t = [1, 2, 3]: Lists may contain a mix of data types; for example: t = ['string', 1, T…
Learn the basics of while and for loops in Python.  while loops are used for testing while, or until, a condition is met: The structure of a while loop is as follows:     while <condition>:         do something         repeate: The break statement m…

756 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question