Basic SFTP question regarding data transfer

Posted on 2005-04-01
Medium Priority
Last Modified: 2013-12-23
I am using SFTP for transferring critical data of file size about 50 MB each  on digital UNIX 4100 Alpha Sever from one location to another. How can I check that the data is successfully transferred and not corrupted ?
Question by:mn210
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 19

Expert Comment

ID: 13681409
A fairly simple method of good confirmation is the use of one or more CRC/Hash algorithms on the data.  CRC32, MD5 and more recently SHA1 have all been well used algorithms.  The likelihood of a false positive in the scenario you give is slim to none if you can use more than one, but one is still likely to give a near-fullproof way.  If your version of Unix does not support such utilities, but does have some form of Zip-compatible utility, then you could Zip the file at either end after transfer and then check the CRC32 from the Zip header using the utility (most utils will support such).
LVL 51

Expert Comment

ID: 13681523
AFAIK the only method, beside sftp's exit status, is to use "ls file" in your sftp's batchfile.
As you still have ssh working, I'd use rsync instead of sftp. rsync also has problems (sometimes) with brocken transfers, but you simply can rerun it and it will do nothing if the remote file is ok.

Author Comment

ID: 13682756
Can you please elaborate/ walk me thro' the steps or give link info  as I am doing for the first time
Flexible connectivity for any environment

The KE6900 series can extend and deploy computers with high definition displays across multiple stations in a variety of applications that suit any environment. Expand computer use to stations across multiple rooms with dynamic access.


Accepted Solution

esanchezvela earned 200 total points
ID: 13690065

after some testing was able to crack the rsync option down, here is the log...

save@localhost:/save# /usr/local/bin/rsync -v -r  -c -e "ssh -l esv " vpd.properties remhost:/suptools/tmp/vpd.properties
building file list ... done
wrote 1029 bytes  read 36 bytes  236.67 bytes/sec
total size is 928  speedup is 0.87
save@localhost:/save# /usr/local/bin/rsync -v -r  -c -e "ssh -l esv " vpd.properties remhost:/suptools/tmp/vpd.properties
building file list ... done
wrote 61 bytes  read 20 bytes  23.14 bytes/sec
total size is 928  speedup is 11.46

just a word of caution, you have to install rsync in the remote server as well, and it looks for it in the normal system's binary paths, if you install it in a customized path, you need to specify it in the command line with the following option...

     --rsync-path=PATH       specify path to rsync on the remote machine

in the following example I am using scp but it could easily replaced by sftp without a change...

  Here I  use the sum command to compute the checksum of the file named text and send it to text.sum, as stated by Barthax there are multiple utilities in case you want to compute several signatures.
pkhost:/home/userid] sum text > text.sum

 Here I am displaying the contents of the text.sum file for exeplification use only since there is no need to know the checksum of the file in the localhost.
pkhost:/home/userid] cat text.sum
58258     2 text

  Here I transfer the file called text to the remote host called "host" and using the username "userid" and leaving the file in the user's home directory.. you could send both files text and text.sum in a single command ..... using the "text????" regular expresion.
pkhost:/home/userid] scp text userid@host:text
text                                                                   0%    0     0.0KB/s   --:-- ETA
text                                                                 100% 1956     1.9KB/s   00:00

  Now it is the turn of the checksum file, since that file is really small chances of corruption are very small.
pkhost:/home/userid] scp text userid@host:text
text.sum                                                               0%    0     0.0KB/s   --:-- ETA
text.sum                                                             100%   17     0.0KB/s   00:00

Now, in your receiving host, you should be able to execute the same (sum) command vs the "text" file and compare the output vs the original checksum file.

LVL 51

Expert Comment

ID: 13690968
> ..  or give link info ..
http://rsync.samba.org/  and see esanchezvela's comment
LVL 32

Expert Comment

ID: 13715735
Since FTP uses TCP, delivery is guarenteed, and will be exactly as it was on the transmitting host.


Featured Post

Create the perfect environment for any meeting

You might have a modern environment with all sorts of high-tech equipment, but what makes it worthwhile is how you seamlessly bring together the presentation with audio, video and lighting. The ATEN Control System provides integrated control and system automation.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

ADCs have gained traction within the last decade, largely due to increased demand for legacy load balancing appliances to handle more advanced application delivery requirements and improve application performance.
WARNING:   If you follow the instructions here, you will wipe out your VTP and VLAN configurations.  Make sure you have backed up your switch!!! I recently had some issues with a few low-end Cisco routers (RV325) and I opened a case with Cisco TA…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…
Monitoring a network: why having a policy is the best policy? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the enormous benefits of having a policy-based approach when monitoring medium and large networks. Software utilized in this v…
Suggested Courses

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question