Extracting HTML tag contents to use as a parameter

Posted on 2004-10-15
Last Modified: 2013-12-25

I'm no expert at Perl, although I can hack together basic scripts. Our Perl developer was recently released for professional negligence, so I am having to pick up the pieces of one of his projects.

What I need to be able to do is to Extract the contents of the HTML <Title> tag from the page which called the script so I can pass it as a parameterin a query string to a pricing search tool.

We currently have it calling a fixed field through:
my $q=new CGI;
my $part = $q->param('Part');

but this means altering the layout of several thousand pages to include a page specific part description. A better option is to just pass the contents of the Title tag.
This is a UNIX script.

Anyone have any ideas?


Question by:asparak
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
LVL 28

Expert Comment

ID: 12319096
Are you saying that your "database" of parts discriptions is kept within the <title> parts discriptions </title> tags and is spread across several thousand pages of static html?  Can you show us an example of the html and Perl script you're using and/or provide a link to one of the pages?

Author Comment

ID: 12319455
The Title tag looks like a standard HTML tag : <title>My Server Model here</title>

Our Current test version, only available on our internal network while I try to fix this contains the following:

<a href="/cgi-bin/pTester3.cgi?Part="My Server Model here" target="_blank">Price It <span class="rightarrow">&raquo;</span></a>

What I need to do is to change this to:
<a href="/cgi-bin/pTester3.cgi" target="_blank">Price It <span class="rightarrow">&raquo;</span></a> to remove that hard coding issue.

The cgi script then needs to parse the HTML file it was called from and extract the description from the Title tag, to complete the following piece of code within pTester3.cgi
my $q=new CGI;
#my $part = $q->param('Part');
#Need to fix the following line:
my $part=$q->(the title of the page I was called from);
print $q->redirect(-URL=>"https://$portal/webquery/Query?Query.findButton=Find&Query.prodDescOutput=$part&Query.selectedOuputColumns=partNum%2CunitPrice");

This issues a command to our secure server to display up to the minute pricing information back to an authorised user for that product group. $portal is a parameter extracted from the session cookie to point the command to the right pricing information portal.

I have anonimised things a little.
LVL 28

Expert Comment

ID: 12320522
It is not possible it do it in the manor you're wanting because the html page won't passing anything to the cgi script.  If you don't want to pass the info in the link, you'll probably need the Perl script to read and parse the tags in the calling html file, which would not be very efficient.
Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

LVL 28

Accepted Solution

FishMonger earned 125 total points
ID: 12320640
Here's a module that will help extract the info.
LVL 28

Expert Comment

ID: 12320678

Author Comment

ID: 12320864

I'll need to try and get the admins to have HeadParser installed on the server. Then try to figure out how to write the code to get HTTP_Referer and parse it.

Not sure I'm up to this, but I'll give it a go, unless you can think of a better way to approach this. All I have to go on are the few sparse notes of the developer before he was escorted from the building.

One thought I have was to embed some code in the head or something to populate the part dynamically as the Head is templated and so one change could be made to all the pages. It's just the body portion of the page we want to try to avoid having to customise.

LVL 51

Expert Comment

ID: 12331944
> The cgi script then needs to parse the HTML file it was called from and extract the description from the Title tag
i.g. impossible. Dot.
You either need active scripting on client side to do that, or you need to tell you CGI to request the page itself (which is totally unreliable).

You have following choices:
  1. leave as is, means that the generated page contains links with GET or POST request which carry "your title"
  2. build sessions server-side where you store the title and then can get it back after your CGI is called again with same session-ID

Author Comment

ID: 12336838
It's been a nasty hack ,but with the help of a friend who's far better at perl than I, we have managed to test HTML::HeadParser successfully on dev. Just need to push it out to live now.

Probably explains why it took our developer 6 months to do.

Featured Post

The Orion Papers

Are you interested in becoming an AWS Certified Solutions Architect?

Discover a new interactive way of training for the exam.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

It is a general practice to get rid of old user profiles on a computer  in a LAN environment. As I have been working with a company in a LAN environment where users move from one place to some other place at times. This will make many user profil…
This article will show, step by step, how to integrate R code into a R Sweave document
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to count occurrences of each item in an array.

688 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question