Memory or CPU cycles?

Posted on 2001-07-23
Last Modified: 2010-03-05
I am asking for an opinion here...

I am trying to decide what is the best way to output data parsed from a database into an HTML file...

Say, I have a database with a bunch of articles, which I store in an advanced structure of hashtables and stuff..and I have 2 HTML files, one with the body of the page, and one which will be iterated for every article within the body file (called article file).

Here's how it would look:
foreach (@array_containing_body_file)
  $_ =~ s/<content_tag>/&parser_subroutine(@array_containing_article_file)/eg;
  print $_;

sub parser_subroutine
  foreach (@temp_array_containing_article_file)
     print $_
In this case, everything is printed right away, however if the "content_tag" exists more then once in the body file, then the parser_subroutine will be called and executed several times...


The other option is to dump all data from parser_subroutine into a temprorary variable that is fetched BEFORE the FIRST foreach loop, thus preventing the parser_sub running more then once.

$data = &parser_subroutine(@array_containing_article_file);
foreach (@array_containing_body_file)
  $_ =~ s/<content_tag>/$data/eg;
  print $_;

sub parser_subroutine
  my $data;
  foreach (@temp_array_containing_article_file)
     $data .= $_;
  return $data;

This is an example, and the size of both the articles and HTML files can vary greatly depending on implementation...

This is more of a "comment" thing then a question, I just want to hear other people's opinions on the matter...

So feel free to bring in examples of how you do parsing and stuff :)
Question by:tolian
  • 2

Accepted Solution

bebonham earned 50 total points
ID: 6310571
after reading your question and the meta code, I could not see how the one related to the other all that well,

perhaps if you post the actual code, I would understand better.

but, I will explain, based on your question, what I think is the best way. have a body (shell or header and footer... choice of words)

and you have a template for the output of an item.

I assume by your meta code that this is a generic template which you parse with perl variables.

and you have I assume some sort of flat file database and mechinism for reading its data.

which you allude to in your code.

since everything is static EXCEPT for the data from the hash tables, then we need to focus on optimizing the output of the static data.

ultimately, your script will run best if all the html is in the code.

I assume the html code isn't huge, it shouldn't be.
if it is you are better off using no data structure to read in the html files, ie (while(<FILE>){$_=~s/regexp/;print $_;}) because that way you don't store the whole chunck of html in memory at once.

BUT, that is much slower, as we all know disk access is slower, why not load it all at the same time.


here is my return of meta code.

all your html for the top and sides of the page here


if you have stuff that goes below the content put it here


print "Content-type: text/html\n\n";
print $head;
print "<FORMATTING_TAGS>$data</TAGS>";
#in as much detail as you want here
print $foot;


I believe that is the most effective way of accomplishing it.


Author Comment

ID: 6310673
Thanks for the comment...

Your way of doing this is pretty good, and I did it that way before. The problem is I am writing an application where users will not modify the perl code, they'll have to modify the HTML templates to adapt the application to their needs.

The meta code I had in my question would produce exactly the same output, except the bottom one would need more memory while the top one might possibly have to run the same subroutine several times (and it would produce the same output each time).

I am thinking of splitting up the body file I talked about into a TOP and BOTTOM file however, that might make things easier and more efficient.

As for not using a datastructure for the file, that is actually a pretty good idea, I'll have to do some testing to see if that works better.

Ultimately I need to store the HTML code in a separate file, I am thinking though that since the users will edit the code through a special page, that I can generate a perl file where the code that the user edits is placed into $foo<<BAR;\\nCODE\\nBAR and thus I could just require the file when the code needs to be parsed with the data... Hmm

Expert Comment

ID: 6310909
yeah, if you can do anything conditionally, that will be good,

but as I said, if you are reading data in from files, the most efficient way I believe is

as a sub routine
sub parsef
my $contents;
open FILE, $file;
close FILE;
return $contents;

or even better if you can print it without having to use the /s  modifier


but if you care to give more info perhaps my advice could be more on target.

of course, we always try to make the loops as small as possible.

LVL 20

Expert Comment

ID: 9493323
Nothing has happened on this question in over 12 months.
I will leave a recommendation in the Cleanup topic area that
the answer by bebonham be accepted.

Please leave any comments here within the next seven days.


EE Cleanup Volunteer

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
work on ods spreadsheet with perl in ubuntu 4 79
compress files in RAR using perl 13 83
Using Perl DBI to query oracle 3 46
Edit Odbc.ini using perl module 2 90
I've just discovered very important differences between Windows an Unix formats in Perl,at least 5.xx.. MOST IMPORTANT: Use Unix file format while saving Your script. otherwise it will have ^M s or smth likely weird in the EOL, Then DO NOT use m…
A year or so back I was asked to have a play with MongoDB; within half an hour I had downloaded (,  installed and started the daemon, and had a console window open. After an hour or two of playing at the command …
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…

766 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question