PHP libssh2 scp_recv transfers incomplete files

Is it possible you could turn on all errors/warnings/notices/etc.

Make sure there are no errors/warnings/notices in your code.

Make sure you have both a compatible library locally for your version of PHP and make sure that there are no compatibility issues with the remote version (I don't know the protocol, but sometimes, different versions of libraries don't like to communicate - can't think this is the issue, unless there is a bug).

Showing my lack of knowledge on the protocol, when you use filezilla/SFTP, is this using the EXACT SAME protocol?

If so, are you able to install Wireshark (a packet sniffer) and see if the packets are being sent/received the same way? Is there something different between the two apps?

A "have you plugged it in question" ... do you have enough disk space?

Same code on a different machine? Still fails?

Max error reporting has been enabled, and nothing is reported.  --  error_reporting(E_ALL);

I'm using the latest version of libssh2 (.18) from here.  I compiled and installed it according to the instructions:
http://sourceforge.net/projects/libssh2/

Disk space isn't a problem.

I am going to try running more tests, having it transfer from other servers, and I'll put the code on another server if necessary.  There's just no good reason I can come up with as to why this isn't working.  I hadn't thought to run a packet sniffer, but I suppose it can't hurt.

Oh, and here's the code in question, though it's pretty straightforward.

$ssh = ssh2_connect(SSHSERVER, 22);
 
if (ssh2_auth_password($ssh, $login, $password)) {
  echo "PBX Authentication Successful!<br />\n";
} else {
  die('PBX Authentication Failed...<br />\n');
}
 
$olddate = DATELASTRUN;
//set date to previous day so we are sure to get all the CSV files
$olddate = fixdate($olddate, -1);
 
$today = date("y-m-d");
$count = 0;
while ($olddate <= $today) {
	
	//new .tar.gz method to try to speed things up.
	$tarcommand = 'cd '.PBXDIR.'/'.$olddate.' && tar -zcf csv'.$olddate.'.tar.gz *';
	//echo $tarcommand.'<br />';
	$stream = ssh2_exec($ssh, $tarcommand);
	if (!ssh2_scp_recv($ssh, PBXDIR.'/'.$olddate.'/csv'.$olddate.'.tar.gz', $csvdir.'/csv'.$olddate.'.tar.gz')) {
		echo 'csv'.$olddate.'.tar.gz could not be transferred<br />';
	} else {
		echo 'csv'.$olddate.'.tar.gz transferred successfully<br />';
		$count++;
	}
	flush();
	$stream = ssh2_exec($ssh, 'rm -rf '.PBXDIR.'/'.$olddate.'/csv'.$olddate.'.tar.gz');
	
	//$command = 'tar -xvzf '.$csvdir.'/csv'.$olddate.'.tar.gz && rm -rf '.$csvdir.'/csv'.$olddate.'.tar.gz';
	$command = 'tar -xvzf '.$csvdir.'/csv'.$olddate.'.tar.gz ';
	//$command = "pwd";
	//echo $command.'<br />';
	system($command);
	$olddate = fixdate($olddate, 7);
}

Select allOpen in new window

Is it possible that issue the tar.gz command completes, but the actual process is still running on the server?

So you are grabbing the archive as it is being created.

I'm not sure about ssh workings.



Q.

Could you use your server to tell the other server to build the archive and have it send the file to your server?

Hmm. Sounds like a lot of work.


Maybe using a semaphore wrapped around the archive production and a test to see if the semaphore exists will allow you wait until the archive is complete before downloading it.

I solved the problem.

It was that the ssh2_exec() tar command was not finishing before the ssh2_scp_recv command began.  So the file would be incompletely downloaded.

The solution was to modify the tar command with the verbose flag and add the following code below the ssh2_exec() command.

stream_set_blocking($stream, true);
while($line = fgets($stream)) {
   flush();
}

Select allOpen in new window

Force accepted.
Vee_Mod
Community Support Moderator