Solved

Unbuffered output to browser while CGI script is still processing

Posted on 2004-09-09
7
473 Views
Last Modified: 2013-12-25
Hey folks - first question here (it's so exciting!)

I have a CGI script (Perl) running a long process but I want the web browser to display the partial output rather than what it does now which is wait until the entire process is finished (which sometimes times out) before sending the complete output.

And just to stave off a bunch of false leads, $| = 1 is already set and it doesn't solve the problem.

Running Apache on a Windoze server.

Here's a test script. If we can get this working, we've got a solution!

#! perl.exe # I know I know. What am I doing on a Windoze server? never you mind that. :)

$| = 1; # disable STDOUT buffering

print "Content-type: text/html\n\n"; # mime type
for (my $i = 1; $i <= 20; $i++) { # loop 20 times
  print "$i <br /> \r\n"; # print a line at a time - note the newline
  my $time = time; until (time = $time+1) {} # wait 1 sec. There's probably a better way to do this but WTFC right?
}

# -------------------- END OF EXAMPLE --------------------

So currently, this thing waits 20 secs then outputs all 20 lines. I want it to display 1 line a second. Here's my thought: Perl isn't buffering it, Apache is. Please note that I do not have access to the Apache environment configuration, so any solution must not assume that I can tweak Apache itself.

Thanks! I look forward to some enlightening responses!

Tom
0
Comment
Question by:tomaugerdotcom
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
  • 2
  • +1
7 Comments
 
LVL 48

Assisted Solution

by:Tintin
Tintin earned 85 total points
ID: 12021648
There's a whole range of issues that can effect this, not least of which is the browser you are using.

I just did an experiment and found IE waited until the end of the script to display everything, whereas Firefox displayed each line.

BTW, I wrote your script as:

$|=1;
print "Content-Type: text/html\n\n";

for my $i (1..20) {
  print "$i<br>\n";
  sleep 1;
}
0
 
LVL 51

Accepted Solution

by:
ahoffmann earned 20 total points
ID: 12021706
this is a web server configuration, not one of the CGI.
The web server only prints output to the browser if the CGI returned status 0.
You need to tweak the web server to allow the CGI to write to the client directly (AFAIK this is possible with mod_perl in apache). But keep in mind that then you have to write the HTTP response header too.
0
 
LVL 48

Expert Comment

by:Tintin
ID: 12021987
Worth having a read of the last section of http://www.hk8.org/old_web/linux/cgi/ch03_03.htm
0
Forrester Webinar: xMatters Delivers 261% ROI

Guest speaker Dean Davison, Forrester Principal Consultant, explains how a Fortune 500 communication company using xMatters found these results: Achieved a 261% ROI, Experienced $753,280 in net present value benefits over 3 years and Reduced MTTR by 91% for tier 1 incidents.

 
LVL 14

Author Comment

by:tomaugerdotcom
ID: 12026576
Thanks folks for your responses so far. I need a little more direction on this one I think.

From the oriley book mentioned by Tintin:

"nph scripts were more common in the past, because versions of Apache prior to 1.3 buffered the output of standard CGI scripts ... However Apache 1.3 no longer buffers CGI output, so this feature of nph scripts is no longer needed with Apache"

So one would think that I wouldn't be experiencing this issue. For some reason I am. Is it a windows issue?

ahoffman mentioned mod_perl in Apache. Is this something that I can access using the Apache module from within my Perl script? Or is this something I have to do directly on the server (which I don't have administrative access on)? Some more details would be very much appreciated.

Thanks!

Tom
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 12026776
AFAIK you get something in your browser if the CGI send about 8k data before it times out.

>  Is this something that I can access using the Apache module from within my Perl script?
mod_perl loads the perl interpreter permantly into memory, so that apache does not need to start it for each (perl) CGI. AFAIK all Apache::* modules make sense with mod_perl only.
mod_perl can do much more than just imroving performance, if your scripts are programmed with mod_perl in mind (hence using Apache::*) then there is more data available, or it's easyer to get it, or ...
0
 
LVL 4

Expert Comment

by:Neil_Simpson
ID: 12031090
One thing that you could try (it depends on your server config which I know you don't have access to but it may be set up correctly) is renaming your cgi to begin with 'nph-'. This stands for non-parsed header. Basically it tells apache not to buffer the output and just to send exactly what the script outputs when it outputs it. You need to include the full HTTP header if you have a script like this.
0
 
LVL 14

Author Comment

by:tomaugerdotcom
ID: 12044205
I still haven't got this working, but I'm going to split points between Tintin and ahoffman for getting an answer in so promptly.

Apparently some of my assumptions were wrong - in fact, I have been informed that we are running Win3K server and ii6, NOT Apache, so in all likelihood the solutions posted above might indeed work on Apache, though they haven't worked in my case.

Thank you to all who took the time to respond to my query.

Tom
0

Featured Post

How our DevOps Teams Maximize Uptime

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us. Read the use case whitepaper.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

In this tutorial I will show you how to make a simple HTML bar chart with the usage of WhizBase, If you want more information about WhizBase please read my previous articles at http://www.experts-exchange.com/ARTH_5123186.html (http://www.experts-ex…
Batch, VBS, and scripts in general are incredibly useful for repetitive tasks.  Some tasks can take a while to complete and it can be annoying to check back only to discover that your script finished 5 minutes ago.  Some scripts may complete nearly …
Learn the basics of modules and packages in Python. Every Python file is a module, ending in the suffix: .py: Modules are a collection of functions and variables.: Packages are a collection of modules.: Module functions and variables are accessed us…
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question