FTP - ascii versus binary

IF I ftp a file without using bin (ftp from Unix to XP),  is the file then corrupted (it was an Oracle export).

If YES, is there a way to uncorrupt the file (conversion program or something).
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jeff BeckhamEngineerCommented:
If you've used ASCII to transfer a binary file, it is corrupted and is not repairable.
booksplusAuthor Commented:
Is Oracle export a binary file.
Juan OcasioApplication DeveloperCommented:
It should be a binary file.  If you still have the file you should export it as binary.  As jebeckham said, you cannot ftp a binary file using ASCII method without corrupting the file.  What happens is it tries to translate the binary data into ASCII...

Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

The main difference between transferring in Ascii and Binary mode is that in Binary mode, the file is transferred intact and unchanged, while in Ascii mode, extra characters such as line breaks and carriage-return characters are added to the destination file. (I know this is somewhat simplified, but in this case it is enough).

So, assuming your file really was binary to begin with, and you used ascii mode ftp, what you end up with is a file that is slightly bigger than the original, with all these extra characters added after every 80 bytes or so. The important thing to note is that ascii version contains everything that was in the original, plus more.

So in principle you can strip out the extra characters and get the original binary file back.

As the others have suggested, it is much simpler if you can simply re-transfer in binary mode. However, if that is not possible please post back and I'll post further tips on how to recover the file.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
r-k, please read the ftp RFC.  What you are stating is wrong.  http://www.faqs.org/rfcs/rfc959.html

There are NO extra characters added every 80 bytes or so.  When transfering in ASCII mode the sender and the receiver assumes that the file contains text only.  It will change the native record termination charaters (LF in Unix's case) to CRLF for transmission and the reciever will replace the CRLF with its native record termination character, which in this case just happens to be CRLF.

So every place there was LF in the oringally binary file, Windows will have a CRLF.  The problem is you can't just strip these out, as you have no clue if the original binary file may have just had a CRLF in it to begin with.  Which means what started out in a binary file as CRLF is now CRCRLF on the Windows side remember Unix change the LF to CRLF.

So if the original file contained LF or CRLF the new file now contains CRLF and CRCRLF.  You can't strip them out, you might be able to write a program that replaces all CRLF's with just LF and hope for the best.  However their may be other changes as technically the sender changes the character format whatever it stores its text data in nativly to NVT-ASCII and  the receiver is supposed to change the NVT-ASCII to its internal format.  There maybe other things that got changed.
"What you are stating is wrong.."

Simplified yes, but not wrong. If you read my original post more carefully, you will notice that I used phrases like "somewhat simplified.." and "..or so" and "..in principle". Your elaboration is in fact repeating what I said but pointing out the main problem, i.e. if the original binary file happened to contain CR/LF characters by chance, then recovery becomes difficult. That is why I recommended to the original poster that getting the original file sent again is the best way.

However, my statements are not just theoretical. Some time back I was called upon to write a program to fix this exact same problem (though the files were not from Oracle), which I did, and the data were restored in 100% of cases. It is true that you need some knowledge about the files, esp. you need to be able to check that the recreation was successful.

So what I am saying is that while not guaranteed, it is possible. If there are other ways to get the files restored that is certainly better.
I would say that you got lucky.  The NVT-ASCII standard uses 7-bit ASCII but transmitts the data using 8 bits.  The problem is that execpt for specially defined control charaters the msb is supposed to be set to 0 in all cases for values greater that decimal value 128, so x80 becomes x00.  Which means that if you have any data that is decimal values of 128-255 that do not represent the defined control characters, you will loose the msb and thus have garbage.  The majority of the special control characters are decimal 240-255, which means anything between 128-239 gets changed.

See http://www.scit.wlv.ac.uk/~jphb/comms/telnet.html for reference and/or search on "NVT-ASCII msb" using any search engine

If you were able to create a program to fix a file that had been transfered with ASCII and it should have been BIN and you were succesful, then I would say that the file was in a sense ASCII, that is it only contained only characters with hex vailues 0-7F.

Always use binary transfer to all non-plaintext file.

"I would say that you got lucky...."

Luck has not much to do with it. In fact almost none of the FTP implementations strip out any of the bits in ascii mode. Rather than worry about what is supposed to happen, you can easily test this by transferring a few binary files (e.g. images or movies) in ascii mode using ftp and see what happens. I just did this with three different FTP servers, and no bits were lost in ascii mode.

In any case, I am not much interested in debating this point. I will wait for the original poster to get back with more information if he has an interest in restoring his file. It would help to know something about which systems the file was transferred from/to, and using which FTP server/client?
booksplusAuthor Commented:
I performed the process over again.  Which meant that I had to fedex an external hard drive again - the file was 7 GB.  This time a ftp'd the file in binary mode.  

Thanks for all your input.  Very good discussion.  I may post a new topic to find out how to repair the file for future reference.
Thanks. Glad you were able to recover your file.
Dear R-k

I'm trying to reach you, since I have a broken binary that was transfered in ASCII mode.
Binary files transferred in ftp ascii mode experience loss of information.
Depending on what kind of file it is, recovery can be anywhere from easy to difficult to impossible.

The accepted answer is usually the best one -- re-transfer, taking care not to corrupt the file with ftp "type ascii".

Easy: Plain text files, some straightforward CR/LF munging restores what you need.
Difficult: I have successfully recovered data in corrupt zip files, but the process is not turn-key, it requires lots of computation, and it requires a heavily modified decompressor, and custom coding of a search heuristic function.  So every case is a consulting project, not a turn-key process.  But if you've lost something important and irreplaceable, maybe it's worth it.  I have a web site http://www.bukys.com/services/recovery/ that describes the services, for those rare cases where this is the right solution.
Impossible: If your can't describe something about your data that can guide the repair process (e.g. it's not very structured) then it really is infeasible to fix it.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Operating Systems

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.