Solved

How to unwrap data being blocked at 80 bytes

Posted on 2010-08-19
5
536 Views
Last Modified: 2013-12-26
I am using a FTP script to receiving data from a partner and the data is being blocked with 0D0A's every 80 bytes.  Need to removed this from the data and not sure the command to use.  Thank you for the assistance
BJE
0
Comment
Question by:bje
5 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 33482281
Is that 80 bytes counting the 0D0A's, or does adding 0D0A's make it 82 bytes?
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 33483492
0D0A is CrLf. I.e. these are fixed length lines of text in DOS format. Unless there is binary content, you should use the FTP "ascii" command before "get" of files in your FTP session - this should remove the Cr characters. You are effectively then left with punched card images - what do you want to do with them next?
0
 

Author Comment

by:bje
ID: 33510411
0D0A makes it 82 bytes
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 33516193
You have 80-column punched card images then. Can you confirm that they contain only Ascii data? (i.e. only printable characters). The command less file will pretty-much tell you that- non-ascii characters will display in reverse video. You really want to remove the Cr characters - the dos2unix utility http://sourceforge.net/projects/dos2unix/ will do that.
What else exactly do you want to do with the data?
0
 
LVL 19

Accepted Solution

by:
simon3270 earned 500 total points
ID: 33540413
There are various ways to do this, depending onthe content of the rest of the file.  If the file only contains 0D and 0A at the ends of these 80-character lines, and not elsewhere, then to remove all 0D and 0A characters (whether or not they appear together):

    tr -d '\015\012' < ftped_file > result_file

(015 and 012 are the octal equivalents of 0x0D and 0x0A)

You might think you could use "cut" to drop the 8st and 82nd characters from all lines:
 
   cut -c1-80 < ftped_file > result-file

but this just replaces the 0D0A with 0A (it drops the 0D0A characters, then appends a 0A newline when it writes the string out).  One problem with most GNU/Linux tools is that they are line-based, so the trailing 0A is handled almost silently.

A Perl alternative which treats the file as a binary one is shown below.  This script takes two arguments - the input file (with the 0D0A) and the output file. The script reads up to 82 characters from the input file.  If the line is 82 characters, it just takes the first 80.  If the line is less than 80 characters, it takes the whole line.
my $buf;

open infile, $ARGV[0] or die "\nError opening $ARGV[0] for reading: $!\n";
binmode infile;
open outfile, ">$ARGV[1]" or die "\nError opening $ARGV[1] for writing: $!\n";
binmode outfile;

while (1) {
  read (infile, $buf, 82);
  if (length($buf) == 0) {last;}
  if (length($buf) == 82) {
    $buf = substr $buf, 0, 80;
  }
  printf outfile "%s", $buf;
};

close infile or die "Error closing $ARGV[0]: $!\n";
close outfile or die "Error closing $ARGV[1]: $!\n";

Open in new window

0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
repeateFront java challenge 31 94
difference between String.subString() and String.subSequence() 6 237
matchUp  challenge 6 69
Use of condition with 'serial' in ansible 2 71
In this article, I'll describe -- and show pictures of -- some of the significant additions that have been made available to programmers in the MFC Feature Pack for Visual C++ 2008.  These same feature are in the MFC libraries that come with Visual …
Introduction: Hints for the grid button.  Nested classes, templated collections.  Squash that darned bug! Continuing from the sixth article about sudoku.   Open the project in visual studio. First we will finish with the SUD_SETVALUE messa…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
A short tutorial showing how to set up an email signature in Outlook on the Web (previously known as OWA). For free email signatures designs, visit https://www.mail-signatures.com/articles/signature-templates/?sts=6651 If you want to manage em…

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question