search and list

My users use a form that creates an html pages with the results of the form input. Each form submission creates the html page and assigns a data id for the filename. The html page has common field values, e.g., Name, UserID, ProjectID, and layed in typical htm page format as example shown,

RequestID: 20031022093853.html
Name: Joe Doe
UserId: jdoe1234
ProjectId: 123456
Project Description: this is my project. whatever, whatever.

Search Page for a request
Request Id [enter the id number here]  =SUBMIT=
Project Id  [enter the project Id]  =SUBMIT=
UserId [enter the userid] =SUBMIT=
List all requests =SUBMIT=

I would like to setup a search where I can enter in text fields, a RequestID, ProjectId, UserId. The RequestId will hyperlink to the location of the request page saved in html

The results will list in a table format as

RequestID             |   ProjectId   |  UserId
20031022093853   |  123456      | jdoe1234

Can someone give me an head start on how to accomplish this?


Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Hi Kwal,

Since you're not using a DB (I would recommend using an RDBMS to deal with the problem you're facing), one way to do it is to provide a simple form where user can enter those text they want to search, then use CGI module to get those texts out.  Then form a regex to search all html files under requests directory (for example see the 3rd method in my accepted answer at, then you just echo a href back to user using the request ID, which is linked with the URL for that request ID.  I think that should be good enough for a head start.

KwalAuthor Commented:
inq123, I tried the examples you provided at the link (Q20755740.html), but I'm having problems with it. The search values are dumped into a  file, each time I run the script.
That was the correct behavior for that question (the OP for that question was using that behavior).  For your question specifically, it needs to be modified of course, but the basic principle was well spelled out in that post (as I also compared the speed of the 3 versions, one from OP and two based on my suggestions).  

I was just about to modify the code a bit to suit your situation when I realized I did not really understand how exactly you want user to search?  In your OP, you said: "I would like to setup a search where I can enter in text fields, a RequestID, ProjectId, UserId. The RequestId will hyperlink to the location of the request page saved in html".  This is a bit confusing as if user enter a RequestID, then there's no need to search at all.  Why is it called search?  If user enters projectid, userid, then they might get multiple results that are listed in table as you requested.  But do your users even remember projectid and userid?

Anyway, my suggestion is that if you let user search using projectid (or userid), just use code similar to below (note that you need to add in the code for your forms, and other cgi tasks.  Code below shows how you can do the search faster than opening each file in perl and search line by line.  But if your html files contain messy format that needs careful filtering, go for the slower open each file search line by line method):

use CGI qw/:standard/;

  my @keys;
  push(@keys, "ProjectId: " . param('projectid')) if(param('projectid'));
  push(@keys, "UserId: " . param('userid')) if(param('userid'));
  my $query = join('|', @keys);

  open(IN, "egrep '$query' *.html |");
    $reqid{$1}++ if(/^(.*?):/);
    print "$1 has $query!\n";
PMI ACP® Project Management

Prepare for the PMI Agile Certified Practitioner (PMI-ACP)® exam, which formally recognizes your knowledge of agile principles and your skill with agile techniques.

KwalAuthor Commented:
I should've been a bit more forthright... sorry.

I currently use a perl script, a poorly done one but it does the basics, of taking a comma-separate csv data file saved in the format as the example in my initial post shows.  I have for days trying to modify this perl script of mine to do the same for an html file.

The csv file data is not comma-separated but separated by & e.g., name=kwal&uid=kwal123&dob=01010001

I use 3 separate perl scripts, one that creates the data file from a form submission, using the following params

my ($sec,$min,$hour,$mday,$mon,$year,$wday) = (localtime(time))[0,1,2,3,4,5,6];
my $timestamp=sprintf "%4.4d%....];
my timestamp_uniq=sprintf "%4.4d....$sec;

my formtable="data_file.$timestamp_uniq.$$.csv";

here it dumps the data in a csv file
open formtable, ">$formtable;
open formtable ">data";
print formtable "unique_id=$formtable";
print formtable "&timestamp=${timestamp}";
print formtable "&everythinggoesinhere";
close formtable;
close data;

Then I use a view and search script, which, for the view script it works like this.

my $dir="where/the/data/is/stored";
my data_file="where/the/form/table/data/files/are/stored";
my $view_data_file="/where/again/temp/dir";

Here it opens and write the formtable data file in html,

open write_view, ">$view_data_file";
print write_view <<_eot;
<P>html stuff</p>
close write_view;

it pulls in the data from the data file

open (file, $data_file);
@pairs=split/&/, $buffer);
open write_view, ">>$view_data_file";

$filelocation ="/mywebserver/formtable/data/temp/view_data_file.$formtable-$$.html";

I hope you can make sense out of my nonsense! I don't have access to my files right now to give you the full picture of what I have that works very good on raw text based files, but how can I accomplish the same for html files that are set in a web page format.

Thanks for the help.
It sounds to me that this is not a very efficient way of doing things.  First of all your flat file searching and then html searching would be redundant in functionality, and outputing flat file and html file would be too.  With the html file being searched, it's really not efficient in itself.  In effect, you're pre-processing the html data you want to return to user and store on server, instead of postponing the html code generation till only when user needs it.  My suggestion would be:

1. If you want to stay lazy, then you can get rid of those html files and only save those flat files.  Do search in flat file, and display back to user an html file with the table you want.  Only this time you don't use "" as URL, instead, use "", and this would be the one that generates html file based on the data in the flat file.

2. If you want to do it right, store all your data in a relational database.  The search would be fast, and searching each field would be easy.  Generating html would be easy too since you pretty much have the code already anyway.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
KwalAuthor Commented:
This week, I started looking at using RDB and it is definitely making this process a whole lot easier.
Thanks inq, your suggestion taken and I'm working on it.
Full points awarded.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.