search and list

Posted on 2003-10-22
Last Modified: 2010-03-05
My users use a form that creates an html pages with the results of the form input. Each form submission creates the html page and assigns a data id for the filename. The html page has common field values, e.g., Name, UserID, ProjectID, and layed in typical htm page format as example shown,

RequestID: 20031022093853.html
Name: Joe Doe
UserId: jdoe1234
ProjectId: 123456
Project Description: this is my project. whatever, whatever.

Search Page for a request
Request Id [enter the id number here]  =SUBMIT=
Project Id  [enter the project Id]  =SUBMIT=
UserId [enter the userid] =SUBMIT=
List all requests =SUBMIT=

I would like to setup a search where I can enter in text fields, a RequestID, ProjectId, UserId. The RequestId will hyperlink to the location of the request page saved in html

The results will list in a table format as

RequestID             |   ProjectId   |  UserId
20031022093853   |  123456      | jdoe1234

Can someone give me an head start on how to accomplish this?


Question by:Kwal
  • 3
  • 3

Expert Comment

ID: 9600328
Hi Kwal,

Since you're not using a DB (I would recommend using an RDBMS to deal with the problem you're facing), one way to do it is to provide a simple form where user can enter those text they want to search, then use CGI module to get those texts out.  Then form a regex to search all html files under requests directory (for example see the 3rd method in my accepted answer at, then you just echo a href back to user using the request ID, which is linked with the URL for that request ID.  I think that should be good enough for a head start.


Author Comment

ID: 9614134
inq123, I tried the examples you provided at the link (Q20755740.html), but I'm having problems with it. The search values are dumped into a  file, each time I run the script.

Expert Comment

ID: 9616481
That was the correct behavior for that question (the OP for that question was using that behavior).  For your question specifically, it needs to be modified of course, but the basic principle was well spelled out in that post (as I also compared the speed of the 3 versions, one from OP and two based on my suggestions).  

I was just about to modify the code a bit to suit your situation when I realized I did not really understand how exactly you want user to search?  In your OP, you said: "I would like to setup a search where I can enter in text fields, a RequestID, ProjectId, UserId. The RequestId will hyperlink to the location of the request page saved in html".  This is a bit confusing as if user enter a RequestID, then there's no need to search at all.  Why is it called search?  If user enters projectid, userid, then they might get multiple results that are listed in table as you requested.  But do your users even remember projectid and userid?

Anyway, my suggestion is that if you let user search using projectid (or userid), just use code similar to below (note that you need to add in the code for your forms, and other cgi tasks.  Code below shows how you can do the search faster than opening each file in perl and search line by line.  But if your html files contain messy format that needs careful filtering, go for the slower open each file search line by line method):

use CGI qw/:standard/;

  my @keys;
  push(@keys, "ProjectId: " . param('projectid')) if(param('projectid'));
  push(@keys, "UserId: " . param('userid')) if(param('userid'));
  my $query = join('|', @keys);

  open(IN, "egrep '$query' *.html |");
    $reqid{$1}++ if(/^(.*?):/);
    print "$1 has $query!\n";
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!


Author Comment

ID: 9617036
I should've been a bit more forthright... sorry.

I currently use a perl script, a poorly done one but it does the basics, of taking a comma-separate csv data file saved in the format as the example in my initial post shows.  I have for days trying to modify this perl script of mine to do the same for an html file.

The csv file data is not comma-separated but separated by & e.g., name=kwal&uid=kwal123&dob=01010001

I use 3 separate perl scripts, one that creates the data file from a form submission, using the following params

my ($sec,$min,$hour,$mday,$mon,$year,$wday) = (localtime(time))[0,1,2,3,4,5,6];
my $timestamp=sprintf "%4.4d%....];
my timestamp_uniq=sprintf "%4.4d....$sec;

my formtable="data_file.$timestamp_uniq.$$.csv";

here it dumps the data in a csv file
open formtable, ">$formtable;
open formtable ">data";
print formtable "unique_id=$formtable";
print formtable "&timestamp=${timestamp}";
print formtable "&everythinggoesinhere";
close formtable;
close data;

Then I use a view and search script, which, for the view script it works like this.

my $dir="where/the/data/is/stored";
my data_file="where/the/form/table/data/files/are/stored";
my $view_data_file="/where/again/temp/dir";

Here it opens and write the formtable data file in html,

open write_view, ">$view_data_file";
print write_view <<_eot;
<P>html stuff</p>
close write_view;

it pulls in the data from the data file

open (file, $data_file);
@pairs=split/&/, $buffer);
open write_view, ">>$view_data_file";

$filelocation ="/mywebserver/formtable/data/temp/view_data_file.$formtable-$$.html";

I hope you can make sense out of my nonsense! I don't have access to my files right now to give you the full picture of what I have that works very good on raw text based files, but how can I accomplish the same for html files that are set in a web page format.

Thanks for the help.

Accepted Solution

inq123 earned 120 total points
ID: 9618105
It sounds to me that this is not a very efficient way of doing things.  First of all your flat file searching and then html searching would be redundant in functionality, and outputing flat file and html file would be too.  With the html file being searched, it's really not efficient in itself.  In effect, you're pre-processing the html data you want to return to user and store on server, instead of postponing the html code generation till only when user needs it.  My suggestion would be:

1. If you want to stay lazy, then you can get rid of those html files and only save those flat files.  Do search in flat file, and display back to user an html file with the table you want.  Only this time you don't use "" as URL, instead, use "", and this would be the one that generates html file based on the data in the flat file.

2. If you want to do it right, store all your data in a relational database.  The search would be fast, and searching each field would be easy.  Generating html would be easy too since you pretty much have the code already anyway.

Author Comment

ID: 9652896
This week, I started looking at using RDB and it is definitely making this process a whole lot easier.
Thanks inq, your suggestion taken and I'm working on it.
Full points awarded.

Featured Post

Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Perl, group, sort, count question 6 149
Perl 101 11 85
How to prepend a string to a matched string using regex and substitution in Perl 5 89
File Find regex problem 4 68
I've just discovered very important differences between Windows an Unix formats in Perl,at least 5.xx.. MOST IMPORTANT: Use Unix file format while saving Your script. otherwise it will have ^M s or smth likely weird in the EOL, Then DO NOT use m…
On Microsoft Windows, if  when you click or type the name of a .pl file, you get an error "is not recognized as an internal or external command, operable program or batch file", then this means you do not have the .pl file extension associated with …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

740 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question