Solved

Increasing CGI request time

Posted on 1997-03-05
8
260 Views
Last Modified: 2013-12-25
I have a script which is used to email a list of members on a list. It works fine with 5/10/20 entries, but when using large numbers of members, the script can't complete the task before the browser responds with a "document contains no data" message.

The script itself is written in Perl, using cgi-lib.pl, and I have already tried using the "$| = 1" statement to prevent the buffering of output, but this has no effect.

Question: How can I ensure that the script has executed fully before being terminated?
0
Comment
Question by:Voodoo
8 Comments
 
LVL 5

Expert Comment

by:julio011597
Comment Utility
The problem should be in the code, so would you please give us a look at it - or at a significant part of it, where significant means 'able by itself to reproduce the error you are encountering'.

BTW, no buffering problem should concern you in this case.
'Documents contains no data' usually returns when your script does send a valid html header, then exits _for an error_, so not sending any content data.
You could try putting some print("step xx performed"); here and there to understand where your script exits.

Rgds
0
 
LVL 2

Author Comment

by:Voodoo
Comment Utility
Certainly, here is an extract of the loop which handles the emailing.

-->8--

    open (DATABASE, "lists/$database");
    while (<DATABASE>) {

# Each line will have someones email followed by a pipe (|)
# followed by their name. Split each line into name and email
# using the pipe as our delimiter.

    ($recipient_email, $recipient_name) = split (/\|/, $_);

# Now use the send_mail routine in mail-lib.pl to send each
# person the email.  The routine takes 4 parameters...who
# the mail is from, who it is to, the subject and the body)

    &send_mail("$email_from","$recipient_email", "$form_data{'subject'}", "$fo
rm_data{'comments'}");

# Now print out a list of successful emails for the admin and
# close the database.

# Preempt any prints (This didn't seem to help)
      select(STDOUT);
      $| = 1;

      print "Mail Sent to: $recipient_email<BR>\n";

    }
    close (DATABASE);

-->8--

I stress that this routine (and script) works perfectly for smaller numbers in the list, albeit it takes a proportionate amount of time to the number of recipients in the list. So when the number exceeds, say, 150 recipients, it processes a certain number before somehow terminating with the "Document returns no data" error message. What is puzzling is that the print message doesn't print anything directly to the screen, even though the no buffer variable is set.

0
 
LVL 3

Expert Comment

by:pc012197
Comment Utility
I suppose it's a timeout problem, either in your server or in your browser.

0
 
LVL 4

Expert Comment

by:pluim
Comment Utility
Sounds to me like it's a timeout problem with the server.
Generally timeout for CGI scripts is somewhere between 1 - 30 minutes, but some webmasters set this even tighter.

Couple of things you might want to try:
- set the server timeout to a higher number (if possible :)
- split the process, i.e. make a form that allows you to send
mail to users "a-h", "h-o" and "o-z". Crude, but works.
- write a java applet that requests the memberlist and then
sends mail to them one by one. Probably more work than you
bargained for.

It could also be that your mailserver is just slow, and that
the actual creation of the mail falls well within the time limit.
Splitting the creation and sending of the mail might solve
your problem then. Again, slightly crude.



0
Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

 
LVL 5

Expert Comment

by:icd
Comment Utility
I had a similar problem with a mail-list of my own. The program would appear to hang for some time before giving the output. In my case I found a couple of ill-formed or invalid email addresses which seemed to hang the program for some time.

If you always try your list with the first 10 or 20 entries try splitting the long list into separate sections. Test each group of 20 or so and see if any one of them gives you a problem. If so there is likely to be bad data in that section.

0
 
LVL 5

Expert Comment

by:julio011597
Comment Utility
Ok, i cannot debug Perl, neither can say why your $|=1 doesn't work.
Anyway, to make sure you are facing a timeout problem, you could consider trying to switch to server push and see if that works.

This is a sample (pseudo-code ;):

print "Content-type: multipart/x-mixed-replace;boundary=my_boundary\n\n"
open(DATABASE, "lists/$database";
while(<DATABASE>) {
  ... handle mail sending ...
  print "--my_boundary\n";
  print "Content-type: text/html\n\n";
  ... print out infos on mail sent ...
}
print "--my_boundary--\n";
close(DATABASE);

If you'll try this, please tell us if it worked...

Rgds
0
 
LVL 5

Expert Comment

by:julio011597
Comment Utility
Sorry, maybe:

while(<DATABASE>) {
  print "--my_boundary\n";
  ... handle mail sending ...

would work better.
-
0
 
LVL 1

Accepted Solution

by:
Fordream earned 100 total points
Comment Utility
I think the reason is not on buffering output. If so, task would be done though "Document contains no data" in browser. Maybe the reason is the codes in loop is not done. because of error on file opening, or whatever.
 Let's suppose that an error occured on opening file. then loop "while (<DATABASE>)" would not be executed. Insert some lines between "open" and "while" commands to check the reason:

 if (!<DATABASE>) {
   print "Oops! something went wrong!\n";
   # You can insert some code that prints error code to examine
   # the problem.
 }

 If "Oops! something went wrong!" message is shown in your browser instead of "Document contains no data" Dialog box, then literally something went wrong (There was an error on file opening, or file has no data, or whatever...)
 I think you will receive the alert message instead of "Document contains no data" error after inserting the codes above. Then problem would be one of what I said. Check them!
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

The following is a collection of cases for strange behaviour when using advanced techniques in DOS batch files. You should have some basic experience in batch "programming", as I'm assuming some knowledge and not further explain the basics. For someā€¦
This article will show, step by step, how to integrate R code into a R Sweave document
Learn the basics of strings in Python: declaration, operations, indices, and slicing. Strings are declared with quotations; for example: s = "string": Strings are immutable.: Strings may be concatenated or multiplied using the addition and multiplicā€¦
The viewer will learn how to dynamically set the form action using jQuery.

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now