Solved

Unbuffered output to browser while CGI script is still processing

Posted on 2004-09-09
7
461 Views
Last Modified: 2013-12-25
Hey folks - first question here (it's so exciting!)

I have a CGI script (Perl) running a long process but I want the web browser to display the partial output rather than what it does now which is wait until the entire process is finished (which sometimes times out) before sending the complete output.

And just to stave off a bunch of false leads, $| = 1 is already set and it doesn't solve the problem.

Running Apache on a Windoze server.

Here's a test script. If we can get this working, we've got a solution!

#! perl.exe # I know I know. What am I doing on a Windoze server? never you mind that. :)

$| = 1; # disable STDOUT buffering

print "Content-type: text/html\n\n"; # mime type
for (my $i = 1; $i <= 20; $i++) { # loop 20 times
  print "$i <br /> \r\n"; # print a line at a time - note the newline
  my $time = time; until (time = $time+1) {} # wait 1 sec. There's probably a better way to do this but WTFC right?
}

# -------------------- END OF EXAMPLE --------------------

So currently, this thing waits 20 secs then outputs all 20 lines. I want it to display 1 line a second. Here's my thought: Perl isn't buffering it, Apache is. Please note that I do not have access to the Apache environment configuration, so any solution must not assume that I can tweak Apache itself.

Thanks! I look forward to some enlightening responses!

Tom
0
Comment
Question by:tomaugerdotcom
  • 2
  • 2
  • 2
  • +1
7 Comments
 
LVL 48

Assisted Solution

by:Tintin
Tintin earned 85 total points
ID: 12021648
There's a whole range of issues that can effect this, not least of which is the browser you are using.

I just did an experiment and found IE waited until the end of the script to display everything, whereas Firefox displayed each line.

BTW, I wrote your script as:

$|=1;
print "Content-Type: text/html\n\n";

for my $i (1..20) {
  print "$i<br>\n";
  sleep 1;
}
0
 
LVL 51

Accepted Solution

by:
ahoffmann earned 20 total points
ID: 12021706
this is a web server configuration, not one of the CGI.
The web server only prints output to the browser if the CGI returned status 0.
You need to tweak the web server to allow the CGI to write to the client directly (AFAIK this is possible with mod_perl in apache). But keep in mind that then you have to write the HTTP response header too.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 12021987
Worth having a read of the last section of http://www.hk8.org/old_web/linux/cgi/ch03_03.htm
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 14

Author Comment

by:tomaugerdotcom
ID: 12026576
Thanks folks for your responses so far. I need a little more direction on this one I think.

From the oriley book mentioned by Tintin:

"nph scripts were more common in the past, because versions of Apache prior to 1.3 buffered the output of standard CGI scripts ... However Apache 1.3 no longer buffers CGI output, so this feature of nph scripts is no longer needed with Apache"

So one would think that I wouldn't be experiencing this issue. For some reason I am. Is it a windows issue?

ahoffman mentioned mod_perl in Apache. Is this something that I can access using the Apache module from within my Perl script? Or is this something I have to do directly on the server (which I don't have administrative access on)? Some more details would be very much appreciated.

Thanks!

Tom
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 12026776
AFAIK you get something in your browser if the CGI send about 8k data before it times out.

>  Is this something that I can access using the Apache module from within my Perl script?
mod_perl loads the perl interpreter permantly into memory, so that apache does not need to start it for each (perl) CGI. AFAIK all Apache::* modules make sense with mod_perl only.
mod_perl can do much more than just imroving performance, if your scripts are programmed with mod_perl in mind (hence using Apache::*) then there is more data available, or it's easyer to get it, or ...
0
 
LVL 4

Expert Comment

by:Neil_Simpson
ID: 12031090
One thing that you could try (it depends on your server config which I know you don't have access to but it may be set up correctly) is renaming your cgi to begin with 'nph-'. This stands for non-parsed header. Basically it tells apache not to buffer the output and just to send exactly what the script outputs when it outputs it. You need to include the full HTTP header if you have a script like this.
0
 
LVL 14

Author Comment

by:tomaugerdotcom
ID: 12044205
I still haven't got this working, but I'm going to split points between Tintin and ahoffman for getting an answer in so promptly.

Apparently some of my assumptions were wrong - in fact, I have been informed that we are running Win3K server and ii6, NOT Apache, so in all likelihood the solutions posted above might indeed work on Apache, though they haven't worked in my case.

Thank you to all who took the time to respond to my query.

Tom
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

In this tutorial I will focus on how to use WhizBase as a tool for sending ICQ messages to ICQ. Here I will use a new technology in WhizBase, published in WhizBase 5.1 version. In this tutorial I will use 3 files, pager.wbsp for the processing, e…
Batch, VBS, and scripts in general are incredibly useful for repetitive tasks.  Some tasks can take a while to complete and it can be annoying to check back only to discover that your script finished 5 minutes ago.  Some scripts may complete nearly …
Learn the basics of strings in Python: declaration, operations, indices, and slicing. Strings are declared with quotations; for example: s = "string": Strings are immutable.: Strings may be concatenated or multiplied using the addition and multiplic…
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now