Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win


Memory or CPU cycles?

Posted on 2001-07-23
Medium Priority
Last Modified: 2010-03-05
I am asking for an opinion here...

I am trying to decide what is the best way to output data parsed from a database into an HTML file...

Say, I have a database with a bunch of articles, which I store in an advanced structure of hashtables and stuff..and I have 2 HTML files, one with the body of the page, and one which will be iterated for every article within the body file (called article file).

Here's how it would look:
foreach (@array_containing_body_file)
  $_ =~ s/<content_tag>/&parser_subroutine(@array_containing_article_file)/eg;
  print $_;

sub parser_subroutine
  foreach (@temp_array_containing_article_file)
     print $_
In this case, everything is printed right away, however if the "content_tag" exists more then once in the body file, then the parser_subroutine will be called and executed several times...


The other option is to dump all data from parser_subroutine into a temprorary variable that is fetched BEFORE the FIRST foreach loop, thus preventing the parser_sub running more then once.

$data = &parser_subroutine(@array_containing_article_file);
foreach (@array_containing_body_file)
  $_ =~ s/<content_tag>/$data/eg;
  print $_;

sub parser_subroutine
  my $data;
  foreach (@temp_array_containing_article_file)
     $data .= $_;
  return $data;

This is an example, and the size of both the articles and HTML files can vary greatly depending on implementation...

This is more of a "comment" thing then a question, I just want to hear other people's opinions on the matter...

So feel free to bring in examples of how you do parsing and stuff :)
Question by:tolian
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2

Accepted Solution

bebonham earned 150 total points
ID: 6310571
after reading your question and the meta code, I could not see how the one related to the other all that well,

perhaps if you post the actual code, I would understand better.

but, I will explain, based on your question, what I think is the best way.

...you have a body (shell or header and footer... choice of words)

and you have a template for the output of an item.

I assume by your meta code that this is a generic template which you parse with perl variables.

and you have I assume some sort of flat file database and mechinism for reading its data.

which you allude to in your code.

since everything is static EXCEPT for the data from the hash tables, then we need to focus on optimizing the output of the static data.

ultimately, your script will run best if all the html is in the code.

I assume the html code isn't huge, it shouldn't be.
if it is you are better off using no data structure to read in the html files, ie (while(<FILE>){$_=~s/regexp/;print $_;}) because that way you don't store the whole chunck of html in memory at once.

BUT, that is much slower, as we all know disk access is slower, why not load it all at the same time.


here is my return of meta code.

all your html for the top and sides of the page here


if you have stuff that goes below the content put it here


print "Content-type: text/html\n\n";
print $head;
print "<FORMATTING_TAGS>$data</TAGS>";
#in as much detail as you want here
print $foot;


I believe that is the most effective way of accomplishing it.


Author Comment

ID: 6310673
Thanks for the comment...

Your way of doing this is pretty good, and I did it that way before. The problem is I am writing an application where users will not modify the perl code, they'll have to modify the HTML templates to adapt the application to their needs.

The meta code I had in my question would produce exactly the same output, except the bottom one would need more memory while the top one might possibly have to run the same subroutine several times (and it would produce the same output each time).

I am thinking of splitting up the body file I talked about into a TOP and BOTTOM file however, that might make things easier and more efficient.

As for not using a datastructure for the file, that is actually a pretty good idea, I'll have to do some testing to see if that works better.

Ultimately I need to store the HTML code in a separate file, I am thinking though that since the users will edit the code through a special page, that I can generate a perl file where the code that the user edits is placed into $foo<<BAR;\\nCODE\\nBAR and thus I could just require the file when the code needs to be parsed with the data... Hmm

Expert Comment

ID: 6310909
yeah, if you can do anything conditionally, that will be good,

but as I said, if you are reading data in from files, the most efficient way I believe is

as a sub routine
sub parsef
my $contents;
open FILE, $file;
close FILE;
return $contents;

or even better if you can print it without having to use the /s  modifier


but if you care to give more info perhaps my advice could be more on target.

of course, we always try to make the loops as small as possible.

LVL 20

Expert Comment

ID: 9493323
Nothing has happened on this question in over 12 months.
I will leave a recommendation in the Cleanup topic area that
the answer by bebonham be accepted.

Please leave any comments here within the next seven days.


EE Cleanup Volunteer

Featured Post

Important Lessons on Recovering from Petya

In their most recent webinar, Skyport Systems explores ways to isolate and protect critical databases to keep the core of your company safe from harm.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Many time we need to work with multiple files all together. If its windows system then we can use some GUI based editor to accomplish our task. But what if you are on putty or have only CLI(Command Line Interface) as an option to  edit your files. I…
Checking the Alert Log in AWS RDS Oracle can be a pain through their user interface.  I made a script to download the Alert Log, look for errors, and email me the trace files.  In this article I'll describe what I did and share my script.
Explain concepts important to validation of email addresses with regular expressions. Applies to most languages/tools that uses regular expressions. Consider email address RFCs: Look at HTML5 form input element (with type=email) regex pattern: T…
Six Sigma Control Plans

650 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question