File transfer protocols: how to verify data integrity

andy7789 used Ask the Experts™
Hi Xperts,

I am having troubles in transferring large files from windows desktop to a centos server. I've never had that problem before with small files using FTP, but large files (300MB) make a trouble. I upload a file, then download it back. When comparing hex codes, it shows me a few different bits.

I tried CuteFTP and Dreamweaver.

are there any FTP clients that verify transfer blocks? Is there a better way of achieving some better data integrity?

Please advise

Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Data integrity is an optional for FTP sever, not all of them support it. You have 2 options. First - use FTP server and client with data integrity support (for example WS_FTP). Second - transfer files in archive with CRC32 checks (.zip and .rar).
A time-proven solution would be to put larger files into a distributed .rar (not .zip) archive split up in chunks of 50-100 mb size. Use no compression, and add a .sfv checksum file (

The .sfv is for the partial rar files. Check their integrity before unpacking.


What would be a better transfer option than FTP?

I am trying SFTP via secure shell. Is it better? As I remember, it checks transfer blocks during transmission.

It is easy to check MD5, but the question remains - how to improve transfer quality to avoid re-transmissions?

The transfer quality in FTP should be as good or bad as any other transfers. It's not the transfer that I suspect here, it's the storage.

For quite some time it has been known among FTP practitioners that files originating on a Windows system, uploaded via FTP to a Linux file system and redownloaded to a Windows system again may (but need not) result in slight checksum corruption. This seems to apply to certain filetypes rather than others. Placing files inside distributed rar archives is a proof way to avoid this.

Unfortunately, I never came around to inquiring about the theoretical implications of this phenomenon. I know it sounds a bit like an urban myth. But if this happened to me, I'd try to practically counter it the way I described above.

>> What would be a better transfer option than FTP?

Have you tried Scp?

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial