Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 386
  • Last Modified:

Fetch a .jsp generated page using LWP & Perl

I would like to fetch a web page using the  LWP module, and have the results parsed and then
email users based on the contents of the page.  I have a perl scrip tthat I can modify  to do that.
My problem is,  the website I need the information from generates the page using  javascript,
 and the results appear on the browser  with the extension of .jsp (example: : landregister_new.jsp) ,
which is difficult  for me to even print the screen on, using my netscape 4.6,  let alone return something
 suitable  for parsing.

Can anyone provide  a perl script or some java code or whatever,  to convert the .jsp to html (preferably)
 or ascii, or .txt or  something that I can handle with a perl script ?
  • 2
  • 2
1 Solution
A suffix of .jsp means that Java Server Pages is being used to generate the page, but what arrives in response to the request should still be HTML, possibly with embedded Javascript. Unless some fancy games are being played, the data you want should probably be there in the HTML. Perhaps if you could provide a sample or example of what's causing you a problem?

Do you have 'lynx' available? It's a text-only browser. Can it render the data you want on these pages? If so, another approach you can take is to dump the page with lynx and use ordinary regular expressions and such to take the page apart and extract the data you're interested in.
caciquemanAuthor Commented:
I don't have lynx, and I am fetching this for use on a unix server, and then parsing
the html for certain phrases.  I think the problem is arising because the html page
that calls the .jsp page is fine, but from the .jsp page you need to make a further
selection to proceed....so basically that's why I wanted a script to be able to pass
from the .jsp page one to page two, WITH the completed choice from page one. I
know the data that I need passed, so thats not a problem.
Lynx is available for free for UNIX, so you _could_ have lynx.

Aside from that, you need to examine how the .jsp page is passing the choice made on to the next step. Is it a <form>? A list of different URL links? It should be apparent from the source of the page how it's being done -- your LWP script simply needs to simulate that step.
caciquemanAuthor Commented:
I won't go the lynx route but thank you for your information.

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 2
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now