We help IT Professionals succeed at work.

We've partnered with Certified Experts, Carl Webster and Richard Faulkner, to bring you a podcast all about Citrix Workspace, moving to the cloud, and analytics & intelligence. Episode 2 coming soon!Listen Now

x

Increasing CGI request time

Voodoo
Voodoo asked
on
Medium Priority
291 Views
Last Modified: 2013-12-25
I have a script which is used to email a list of members on a list. It works fine with 5/10/20 entries, but when using large numbers of members, the script can't complete the task before the browser responds with a "document contains no data" message.

The script itself is written in Perl, using cgi-lib.pl, and I have already tried using the "$| = 1" statement to prevent the buffering of output, but this has no effect.

Question: How can I ensure that the script has executed fully before being terminated?
Comment
Watch Question

The problem should be in the code, so would you please give us a look at it - or at a significant part of it, where significant means 'able by itself to reproduce the error you are encountering'.

BTW, no buffering problem should concern you in this case.
'Documents contains no data' usually returns when your script does send a valid html header, then exits _for an error_, so not sending any content data.
You could try putting some print("step xx performed"); here and there to understand where your script exits.

Rgds

Author

Commented:
Certainly, here is an extract of the loop which handles the emailing.

-->8--

    open (DATABASE, "lists/$database");
    while (<DATABASE>) {

# Each line will have someones email followed by a pipe (|)
# followed by their name. Split each line into name and email
# using the pipe as our delimiter.

    ($recipient_email, $recipient_name) = split (/\|/, $_);

# Now use the send_mail routine in mail-lib.pl to send each
# person the email.  The routine takes 4 parameters...who
# the mail is from, who it is to, the subject and the body)

    &send_mail("$email_from","$recipient_email", "$form_data{'subject'}", "$fo
rm_data{'comments'}");

# Now print out a list of successful emails for the admin and
# close the database.

# Preempt any prints (This didn't seem to help)
      select(STDOUT);
      $| = 1;

      print "Mail Sent to: $recipient_email<BR>\n";

    }
    close (DATABASE);

-->8--

I stress that this routine (and script) works perfectly for smaller numbers in the list, albeit it takes a proportionate amount of time to the number of recipients in the list. So when the number exceeds, say, 150 recipients, it processes a certain number before somehow terminating with the "Document returns no data" error message. What is puzzling is that the print message doesn't print anything directly to the screen, even though the no buffer variable is set.

Commented:
I suppose it's a timeout problem, either in your server or in your browser.

Commented:
Sounds to me like it's a timeout problem with the server.
Generally timeout for CGI scripts is somewhere between 1 - 30 minutes, but some webmasters set this even tighter.

Couple of things you might want to try:
- set the server timeout to a higher number (if possible :)
- split the process, i.e. make a form that allows you to send
mail to users "a-h", "h-o" and "o-z". Crude, but works.
- write a java applet that requests the memberlist and then
sends mail to them one by one. Probably more work than you
bargained for.

It could also be that your mailserver is just slow, and that
the actual creation of the mail falls well within the time limit.
Splitting the creation and sending of the mail might solve
your problem then. Again, slightly crude.



icd

Commented:
I had a similar problem with a mail-list of my own. The program would appear to hang for some time before giving the output. In my case I found a couple of ill-formed or invalid email addresses which seemed to hang the program for some time.

If you always try your list with the first 10 or 20 entries try splitting the long list into separate sections. Test each group of 20 or so and see if any one of them gives you a problem. If so there is likely to be bad data in that section.

Ok, i cannot debug Perl, neither can say why your $|=1 doesn't work.
Anyway, to make sure you are facing a timeout problem, you could consider trying to switch to server push and see if that works.

This is a sample (pseudo-code ;):

print "Content-type: multipart/x-mixed-replace;boundary=my_boundary\n\n"
open(DATABASE, "lists/$database";
while(<DATABASE>) {
  ... handle mail sending ...
  print "--my_boundary\n";
  print "Content-type: text/html\n\n";
  ... print out infos on mail sent ...
}
print "--my_boundary--\n";
close(DATABASE);

If you'll try this, please tell us if it worked...

Rgds
Sorry, maybe:

while(<DATABASE>) {
  print "--my_boundary\n";
  ... handle mail sending ...

would work better.
-
Commented:
I think the reason is not on buffering output. If so, task would be done though "Document contains no data" in browser. Maybe the reason is the codes in loop is not done. because of error on file opening, or whatever.
 Let's suppose that an error occured on opening file. then loop "while (<DATABASE>)" would not be executed. Insert some lines between "open" and "while" commands to check the reason:

 if (!<DATABASE>) {
   print "Oops! something went wrong!\n";
   # You can insert some code that prints error code to examine
   # the problem.
 }

 If "Oops! something went wrong!" message is shown in your browser instead of "Document contains no data" Dialog box, then literally something went wrong (There was an error on file opening, or file has no data, or whatever...)
 I think you will receive the alert message instead of "Document contains no data" error after inserting the codes above. Then problem would be one of what I said. Check them!

Not the solution you were looking for? Getting a personalized solution is easy.

Ask the Experts
Access more of Experts Exchange with a free account
Thanks for using Experts Exchange.

Create a free account to continue.

Limited access with a free account allows you to:

  • View three pieces of content (articles, solutions, posts, and videos)
  • Ask the experts questions (counted toward content limit)
  • Customize your dashboard and profile

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.