[Webinar] Streamline your web hosting managementRegister Today

x
?
Solved

PHP libssh2 scp_recv transfers incomplete files

Posted on 2008-01-31
8
Medium Priority
?
562 Views
Last Modified: 2008-02-23
I've been using ssh2_scp_recv() to download .tar.gz files.  Each .tar.gz file ranges from 250 KB to 2 MB.  However, I only receive about 140-500 KB of those files, and all 6 I'm downloading are incomplete.

When I used FileZilla/SFTP to download the files, I was able to get the complete file with no problems.

Any ideas?

I am removing the time limit for script execution:
set_time_limit (0);
0
Comment
Question by:Joe Cool
  • 3
  • 3
7 Comments
 
LVL 40

Expert Comment

by:Richard Quadling
ID: 20810806
Is it possible you could turn on all errors/warnings/notices/etc.

Make sure there are no errors/warnings/notices in your code.

Make sure you have both a compatible library locally for your version of PHP and make sure that there are no compatibility issues with the remote version (I don't know the protocol, but sometimes, different versions of libraries don't like to communicate - can't think this is the issue, unless there is a bug).

Showing my lack of knowledge on the protocol, when you use filezilla/SFTP, is this using the EXACT SAME protocol?

If so, are you able to install Wireshark (a packet sniffer) and see if the packets are being sent/received the same way? Is there something different between the two apps?

A "have you plugged it in question" ... do you have enough disk space?

Same code on a different machine? Still fails?

0
 

Author Comment

by:Joe Cool
ID: 20824009
Max error reporting has been enabled, and nothing is reported.  --  error_reporting(E_ALL);

I'm using the latest version of libssh2 (.18) from here.  I compiled and installed it according to the instructions:
http://sourceforge.net/projects/libssh2/

Disk space isn't a problem.

I am going to try running more tests, having it transfer from other servers, and I'll put the code on another server if necessary.  There's just no good reason I can come up with as to why this isn't working.  I hadn't thought to run a packet sniffer, but I suppose it can't hurt.
0
 

Author Comment

by:Joe Cool
ID: 20824046
Oh, and here's the code in question, though it's pretty straightforward.
$ssh = ssh2_connect(SSHSERVER, 22);
 
if (ssh2_auth_password($ssh, $login, $password)) {
  echo "PBX Authentication Successful!<br />\n";
} else {
  die('PBX Authentication Failed...<br />\n');
}
 
$olddate = DATELASTRUN;
//set date to previous day so we are sure to get all the CSV files
$olddate = fixdate($olddate, -1);
 
$today = date("y-m-d");
$count = 0;
while ($olddate <= $today) {
	
	//new .tar.gz method to try to speed things up.
	$tarcommand = 'cd '.PBXDIR.'/'.$olddate.' && tar -zcf csv'.$olddate.'.tar.gz *';
	//echo $tarcommand.'<br />';
	$stream = ssh2_exec($ssh, $tarcommand);
	if (!ssh2_scp_recv($ssh, PBXDIR.'/'.$olddate.'/csv'.$olddate.'.tar.gz', $csvdir.'/csv'.$olddate.'.tar.gz')) {
		echo 'csv'.$olddate.'.tar.gz could not be transferred<br />';
	} else {
		echo 'csv'.$olddate.'.tar.gz transferred successfully<br />';
		$count++;
	}
	flush();
	$stream = ssh2_exec($ssh, 'rm -rf '.PBXDIR.'/'.$olddate.'/csv'.$olddate.'.tar.gz');
	
	//$command = 'tar -xvzf '.$csvdir.'/csv'.$olddate.'.tar.gz && rm -rf '.$csvdir.'/csv'.$olddate.'.tar.gz';
	$command = 'tar -xvzf '.$csvdir.'/csv'.$olddate.'.tar.gz ';
	//$command = "pwd";
	//echo $command.'<br />';
	system($command);
	$olddate = fixdate($olddate, 7);
}

Open in new window

0
Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

 
LVL 40

Expert Comment

by:Richard Quadling
ID: 20826666
Is it possible that issue the tar.gz command completes, but the actual process is still running on the server?

So you are grabbing the archive as it is being created.

I'm not sure about ssh workings.



Q.

Could you use your server to tell the other server to build the archive and have it send the file to your server?

Hmm. Sounds like a lot of work.


Maybe using a semaphore wrapped around the archive production and a test to see if the semaphore exists will allow you wait until the archive is complete before downloading it.

0
 

Author Comment

by:Joe Cool
ID: 20844630
I solved the problem.

It was that the ssh2_exec() tar command was not finishing before the ssh2_scp_recv command began.  So the file would be incompletely downloaded.

The solution was to modify the tar command with the verbose flag and add the following code below the ssh2_exec() command.

stream_set_blocking($stream, true);
while($line = fgets($stream)) {
   flush();
}

Open in new window

0
 
LVL 40

Accepted Solution

by:
Richard Quadling earned 2000 total points
ID: 20844979
0
 
LVL 1

Expert Comment

by:Vee_Mod
ID: 20965503
Force accepted.
Vee_Mod
Community Support Moderator
0

Featured Post

[Webinar] Improve your customer journey

A positive customer journey is important in attracting and retaining business. To improve this experience, you can use Google Maps APIs to increase checkout conversions, boost user engagement, and optimize order fulfillment. Learn how in this webinar presented by Dito.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hello to you all, I hear of many people congratulate AWS (Amazon Web Services) on how easy it is to spin up and create new EC2 (Elastic Compute Cloud) instances, but then fail and struggle to connect to them using simple tools such as SSH (Secure…
Introduction This article is intended for those who are new to PHP error handling (https://www.experts-exchange.com/articles/11769/And-by-the-way-I-am-New-to-PHP.html).  It addresses one of the most common problems that plague beginning PHP develop…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
Suggested Courses

607 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question