How to unwrap data being blocked at 80 bytes

I am using a FTP script to receiving data from a partner and the data is being blocked with 0D0A's every 80 bytes.  Need to removed this from the data and not sure the command to use.  Thank you for the assistance
BJE
bjeAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ozoCommented:
Is that 80 bytes counting the 0D0A's, or does adding 0D0A's make it 82 bytes?
0
Duncan RoeSoftware DeveloperCommented:
0D0A is CrLf. I.e. these are fixed length lines of text in DOS format. Unless there is binary content, you should use the FTP "ascii" command before "get" of files in your FTP session - this should remove the Cr characters. You are effectively then left with punched card images - what do you want to do with them next?
0
bjeAuthor Commented:
0D0A makes it 82 bytes
0
Duncan RoeSoftware DeveloperCommented:
You have 80-column punched card images then. Can you confirm that they contain only Ascii data? (i.e. only printable characters). The command less file will pretty-much tell you that- non-ascii characters will display in reverse video. You really want to remove the Cr characters - the dos2unix utility http://sourceforge.net/projects/dos2unix/ will do that.
What else exactly do you want to do with the data?
0
simon3270Commented:
There are various ways to do this, depending onthe content of the rest of the file.  If the file only contains 0D and 0A at the ends of these 80-character lines, and not elsewhere, then to remove all 0D and 0A characters (whether or not they appear together):

    tr -d '\015\012' < ftped_file > result_file

(015 and 012 are the octal equivalents of 0x0D and 0x0A)

You might think you could use "cut" to drop the 8st and 82nd characters from all lines:
 
   cut -c1-80 < ftped_file > result-file

but this just replaces the 0D0A with 0A (it drops the 0D0A characters, then appends a 0A newline when it writes the string out).  One problem with most GNU/Linux tools is that they are line-based, so the trailing 0A is handled almost silently.

A Perl alternative which treats the file as a binary one is shown below.  This script takes two arguments - the input file (with the 0D0A) and the output file. The script reads up to 82 characters from the input file.  If the line is 82 characters, it just takes the first 80.  If the line is less than 80 characters, it takes the whole line.
my $buf;

open infile, $ARGV[0] or die "\nError opening $ARGV[0] for reading: $!\n";
binmode infile;
open outfile, ">$ARGV[1]" or die "\nError opening $ARGV[1] for writing: $!\n";
binmode outfile;

while (1) {
  read (infile, $buf, 82);
  if (length($buf) == 0) {last;}
  if (length($buf) == 82) {
    $buf = substr $buf, 0, 80;
  }
  printf outfile "%s", $buf;
};

close infile or die "Error closing $ARGV[0]: $!\n";
close outfile or die "Error closing $ARGV[1]: $!\n";

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
System Programming

From novice to tech pro — start learning today.