Increasing CGI request time

Posted on 1997-03-05
Last Modified: 2013-12-25
I have a script which is used to email a list of members on a list. It works fine with 5/10/20 entries, but when using large numbers of members, the script can't complete the task before the browser responds with a "document contains no data" message.

The script itself is written in Perl, using, and I have already tried using the "$| = 1" statement to prevent the buffering of output, but this has no effect.

Question: How can I ensure that the script has executed fully before being terminated?
Question by:Voodoo
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions

Expert Comment

ID: 1827645
The problem should be in the code, so would you please give us a look at it - or at a significant part of it, where significant means 'able by itself to reproduce the error you are encountering'.

BTW, no buffering problem should concern you in this case.
'Documents contains no data' usually returns when your script does send a valid html header, then exits _for an error_, so not sending any content data.
You could try putting some print("step xx performed"); here and there to understand where your script exits.


Author Comment

ID: 1827646
Certainly, here is an extract of the loop which handles the emailing.


    open (DATABASE, "lists/$database");
    while (<DATABASE>) {

# Each line will have someones email followed by a pipe (|)
# followed by their name. Split each line into name and email
# using the pipe as our delimiter.

    ($recipient_email, $recipient_name) = split (/\|/, $_);

# Now use the send_mail routine in to send each
# person the email.  The routine takes 4 parameters...who
# the mail is from, who it is to, the subject and the body)

    &send_mail("$email_from","$recipient_email", "$form_data{'subject'}", "$fo

# Now print out a list of successful emails for the admin and
# close the database.

# Preempt any prints (This didn't seem to help)
      $| = 1;

      print "Mail Sent to: $recipient_email<BR>\n";

    close (DATABASE);


I stress that this routine (and script) works perfectly for smaller numbers in the list, albeit it takes a proportionate amount of time to the number of recipients in the list. So when the number exceeds, say, 150 recipients, it processes a certain number before somehow terminating with the "Document returns no data" error message. What is puzzling is that the print message doesn't print anything directly to the screen, even though the no buffer variable is set.


Expert Comment

ID: 1827647
I suppose it's a timeout problem, either in your server or in your browser.

Guide to Performance: Optimization & Monitoring

Nowadays, monitoring is a mixture of tools, systems, and codes—making it a very complex process. And with this complexity, comes variables for failure. Get DZone’s new Guide to Performance to learn how to proactively find these variables and solve them before a disruption occurs.


Expert Comment

ID: 1827648
Sounds to me like it's a timeout problem with the server.
Generally timeout for CGI scripts is somewhere between 1 - 30 minutes, but some webmasters set this even tighter.

Couple of things you might want to try:
- set the server timeout to a higher number (if possible :)
- split the process, i.e. make a form that allows you to send
mail to users "a-h", "h-o" and "o-z". Crude, but works.
- write a java applet that requests the memberlist and then
sends mail to them one by one. Probably more work than you
bargained for.

It could also be that your mailserver is just slow, and that
the actual creation of the mail falls well within the time limit.
Splitting the creation and sending of the mail might solve
your problem then. Again, slightly crude.


Expert Comment

ID: 1827649
I had a similar problem with a mail-list of my own. The program would appear to hang for some time before giving the output. In my case I found a couple of ill-formed or invalid email addresses which seemed to hang the program for some time.

If you always try your list with the first 10 or 20 entries try splitting the long list into separate sections. Test each group of 20 or so and see if any one of them gives you a problem. If so there is likely to be bad data in that section.


Expert Comment

ID: 1827650
Ok, i cannot debug Perl, neither can say why your $|=1 doesn't work.
Anyway, to make sure you are facing a timeout problem, you could consider trying to switch to server push and see if that works.

This is a sample (pseudo-code ;):

print "Content-type: multipart/x-mixed-replace;boundary=my_boundary\n\n"
open(DATABASE, "lists/$database";
while(<DATABASE>) {
  ... handle mail sending ...
  print "--my_boundary\n";
  print "Content-type: text/html\n\n";
  ... print out infos on mail sent ...
print "--my_boundary--\n";

If you'll try this, please tell us if it worked...


Expert Comment

ID: 1827651
Sorry, maybe:

while(<DATABASE>) {
  print "--my_boundary\n";
  ... handle mail sending ...

would work better.

Accepted Solution

Fordream earned 100 total points
ID: 1827652
I think the reason is not on buffering output. If so, task would be done though "Document contains no data" in browser. Maybe the reason is the codes in loop is not done. because of error on file opening, or whatever.
 Let's suppose that an error occured on opening file. then loop "while (<DATABASE>)" would not be executed. Insert some lines between "open" and "while" commands to check the reason:

 if (!<DATABASE>) {
   print "Oops! something went wrong!\n";
   # You can insert some code that prints error code to examine
   # the problem.

 If "Oops! something went wrong!" message is shown in your browser instead of "Document contains no data" Dialog box, then literally something went wrong (There was an error on file opening, or file has no data, or whatever...)
 I think you will receive the alert message instead of "Document contains no data" error after inserting the codes above. Then problem would be one of what I said. Check them!

Featured Post

Resolve Critical IT Incidents Fast

If your data, services or processes become compromised, your organization can suffer damage in just minutes and how fast you communicate during a major IT incident is everything. Learn how to immediately identify incidents & best practices to resolve them quickly and effectively.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In this tutorial I will focus on how to use WhizBase as a tool for sending ICQ messages to ICQ. Here I will use a new technology in WhizBase, published in WhizBase 5.1 version. In this tutorial I will use 3 files, pager.wbsp for the processing, e…
It is a general practice to get rid of old user profiles on a computer  in a LAN environment. As I have been working with a company in a LAN environment where users move from one place to some other place at times. This will make many user profil…
Learn the basics of while and for loops in Python.  while loops are used for testing while, or until, a condition is met: The structure of a while loop is as follows:     while <condition>:         do something         repeate: The break statement m…
In this fifth video of the Xpdf series, we discuss and demonstrate the PDFdetach utility, which is able to list and, more importantly, extract attachments that are embedded in PDF files. It does this via a command line interface, making it suitable …

736 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question