?
Solved

How to unwrap data being blocked at 80 bytes

Posted on 2010-08-19
5
Medium Priority
?
551 Views
Last Modified: 2013-12-26
I am using a FTP script to receiving data from a partner and the data is being blocked with 0D0A's every 80 bytes.  Need to removed this from the data and not sure the command to use.  Thank you for the assistance
BJE
0
Comment
Question by:bje
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 33482281
Is that 80 bytes counting the 0D0A's, or does adding 0D0A's make it 82 bytes?
0
 
LVL 35

Expert Comment

by:Duncan Roe
ID: 33483492
0D0A is CrLf. I.e. these are fixed length lines of text in DOS format. Unless there is binary content, you should use the FTP "ascii" command before "get" of files in your FTP session - this should remove the Cr characters. You are effectively then left with punched card images - what do you want to do with them next?
0
 

Author Comment

by:bje
ID: 33510411
0D0A makes it 82 bytes
0
 
LVL 35

Expert Comment

by:Duncan Roe
ID: 33516193
You have 80-column punched card images then. Can you confirm that they contain only Ascii data? (i.e. only printable characters). The command less file will pretty-much tell you that- non-ascii characters will display in reverse video. You really want to remove the Cr characters - the dos2unix utility http://sourceforge.net/projects/dos2unix/ will do that.
What else exactly do you want to do with the data?
0
 
LVL 19

Accepted Solution

by:
simon3270 earned 2000 total points
ID: 33540413
There are various ways to do this, depending onthe content of the rest of the file.  If the file only contains 0D and 0A at the ends of these 80-character lines, and not elsewhere, then to remove all 0D and 0A characters (whether or not they appear together):

    tr -d '\015\012' < ftped_file > result_file

(015 and 012 are the octal equivalents of 0x0D and 0x0A)

You might think you could use "cut" to drop the 8st and 82nd characters from all lines:
 
   cut -c1-80 < ftped_file > result-file

but this just replaces the 0D0A with 0A (it drops the 0D0A characters, then appends a 0A newline when it writes the string out).  One problem with most GNU/Linux tools is that they are line-based, so the trailing 0A is handled almost silently.

A Perl alternative which treats the file as a binary one is shown below.  This script takes two arguments - the input file (with the 0D0A) and the output file. The script reads up to 82 characters from the input file.  If the line is 82 characters, it just takes the first 80.  If the line is less than 80 characters, it takes the whole line.
my $buf;

open infile, $ARGV[0] or die "\nError opening $ARGV[0] for reading: $!\n";
binmode infile;
open outfile, ">$ARGV[1]" or die "\nError opening $ARGV[1] for writing: $!\n";
binmode outfile;

while (1) {
  read (infile, $buf, 82);
  if (length($buf) == 0) {last;}
  if (length($buf) == 82) {
    $buf = substr $buf, 0, 80;
  }
  printf outfile "%s", $buf;
};

close infile or die "Error closing $ARGV[0]: $!\n";
close outfile or die "Error closing $ARGV[1]: $!\n";

Open in new window

0

Featured Post

Enroll in August's Course of the Month

August's CompTIA IT Fundamentals course includes 19 hours of basic computer principle modules and prepares you for the certification exam. It's free for Premium Members, Team Accounts, and Qualified Experts!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction: Finishing the grid – keyboard support for arrow keys to manoeuvre, entering the numbers.  The PreTranslateMessage function is to be used to intercept and respond to keyboard events. Continuing from the fourth article about sudoku. …
Introduction: The undo support, implementing a stack. Continuing from the eigth article about sudoku.   We need a mechanism to keep track of the digits entered so as to implement an undo mechanism.  This should be a ‘Last In First Out’ collec…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
In this video you will find out how to export Office 365 mailboxes using the built in eDiscovery tool. Bear in mind that although this method might be useful in some cases, using PST files as Office 365 backup is troublesome in a long run (more on t…

764 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question