[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 378
  • Last Modified:

Flushing echo data

I have a script that writes 100,000 values to a database in about 90 seconds.  At the top of the script I have>:

echo "<br />Adding 100,000 records to database . . .";
flush();

Open in new window


Then I go into the loop  that writes the data, and at below the loop I have

echo "<br /><br />DONE.  Id of last record inserted is $Id";
flush();

Open in new window


I expected to see the first echo when I started  the script (I start it by entering its URL in the browser address bar) and then see the second echo when the script completes. Ideally I'd also like to see some kind of progress indicator during the 90 seconds.  But instead I just see a "Connecting" note in the browser tab for the script,  with a circling electron icon at the left for 90 seconds.  At the lower left of my screen I see a note "Transferring data from [mysite]" for the 90 seconds.  When the script completes I suddenly see both the echoes above.

Does anyone know how I can see something from the script while it's running, and ideally a progress bar of some kind?

Thanks for any ideas.
Steve
0
steva
Asked:
steva
  • 6
  • 5
  • 2
1 Solution
 
Ray PaseurCommented:
I have seen this kind of thing before.  The server buffers the output until a certain string length is reached, then sends the string.  This is independent of flush() or output buffering.

My solutions to this include (1) padding the first echo with a large number of blank characters or (2) writing a bunch of blank characters followed by flush() after each addition to the data base.  You can tinker with the output size to achieve happy values.  In my particular server configuration, I found that an output string of 256 characters was enough to trigger a sending step when flush() was called.  You might want to use str_pad() to add 256 blanks to the end of the string and then see if the results are what you expect.

HTH, ~Ray
0
 
stevaAuthor Commented:
Thanks Ray, but that didn't work for me:

$str = "<br />Adding 100,000 records to database . . .";
$str = str_pad($str, 512, ".");
echo $str;
flush();

Open in new window


I see the 512 dots when it does print, so the pad is working.
0
 
Ray PaseurCommented:
Please post a link to your phpinfo() script, thanks.  Your installation may be set to buffer the output.
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
stevaAuthor Commented:
0
 
Ray PaseurCommented:
Thanks.  It looks like output buffering is turned off, but the server may still need a lot of characters before it will send the buffer.  I am assuming that you do not have anything like ob_start() in your scripts, right?

Try this link.  It seemed to work as expected.  First you see "Hello" then a 5 second pause, then "World" appears.
http://www.laprbass.com/RAY_temp_steva.php

<?php // RAY_temp_steva.php
error_reporting(E_ALL);
$var = 'Hello';
$var = str_pad($var, 1030);
echo $var;
flush();
sleep(5);
echo 'World';

Open in new window

0
 
stevaAuthor Commented:
Yours works here just as you describe. Perhaps it would help if I posted my entire script:

<?php

include ('../GLOBALS.php');
include ('../functions.php'); 

$prefix  = 696859;  // set to prefix you're given

$str = "<br />Adding 100,000 barcodes for prefix $prefix . . .";
$str = str_pad($str, 512, ".");
echo $str;
flush();


$db = db_connect();	

$i = 0;
$barcode = $prefix * 100000;

do {
	$barcode_checked = add_barcode_chksum($barcode);
	$query = "INSERT INTO barcodes ( barcode ) VALUES ( '$barcode_checked' )";
    $result = mysqli_query($db, $query) or die ("Couldn't execute query: " . mysqli_error());
	$i++;
	$barcode++;
}while ($i < 100000);


$Id = mysqli_insert_id($db);  // Get back the Id of the last bar code inserted. 
$str = "<br /><br />DONE.  Id of last bar code inserted is $Id";
$str = str_pad($str, 512, ".");

echo $str;
flush();
?>

Open in new window


The barcode_chksum( ) function is just:

function add_barcode_chksum($barcode) {
	
	$digits = str_split($barcode);
	$temp = 3 * ($digits[0] + $digits[2] + $digits[4] + $digits[6] + $digits[8] + $digits[10]);
	$temp +=  ($digits[1] + $digits[3] + $digits[5] + $digits[7] + $digits[9]);
	$temp = $temp % 10;
	if ($temp) $temp = 10-$temp;
	$barcode_chkd = ($barcode * 10) + $temp;
	return($barcode_chkd);
}

Open in new window


Thanks for looking at it.

Steve
0
 
Ray PaseurCommented:
Two suggestions: Change the str_pad() length from 512 to 1030 and inspect GLOBALS.php and functions.php to be sure that there is no output buffering started in those scripts.
0
 
stevaAuthor Commented:
Yes, 1030 lets  the first echo come out before the code drops into the loop. But padding isn't  a very satisfying solution.  Ideally I'd like to output a single "." every 1000 insertions to form a primitive progress bar.  Having to output 1030 printable characters each time makes that difficult.

When I have more time I'll have to start with something simple like your example with the 5 second delay and work toward what I have, seeing where it breaks down.

Thanks for your help.  I gave you the points.

Steve
0
 
Dave BaldwinFixer of ProblemsCommented:
In a packet driven network like the internet, you can't actually send a single character.  The smallest unit you can send is a packet with all the addressing overhead everytime.  Combine that with caching on the server and the application and the browser, it becomes next to impossible to send a 'single character'.
0
 
Ray PaseurCommented:
Agree with DaveBaldwin, and I think the demonstration shows the true case.  If I were looking at this problem, I would not spend 2 cents worrying about sending 1030 blank characters. (BTW, I picked 1030 because it is larger than 1024, no other reason).

Anyway, thanks for the points and thanks for using EE, ~Ray
0
 
stevaAuthor Commented:
Dave,

In a packet driven network like the internet, you can't actually send a single character.  The smallest unit you can send is a packet with all the addressing overhead everytime.

Yep, I know that.  But the software on either end of the network can make it look like a single character.  And there are progress bars out there that work.
S.
0
 
Dave BaldwinFixer of ProblemsCommented:
Most of the progress bars are either faked (animated GIFs) or use other methods.  https://www.google.com/search?q=PHP+progress+bar
0
 
stevaAuthor Commented:
Thanks for that.
0

Featured Post

Important Lessons on Recovering from Petya

In their most recent webinar, Skyport Systems explores ways to isolate and protect critical databases to keep the core of your company safe from harm.

  • 6
  • 5
  • 2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now