Solved

How to unwrap data being blocked at 80 bytes

Posted on 2010-08-19
5
535 Views
Last Modified: 2013-12-26
I am using a FTP script to receiving data from a partner and the data is being blocked with 0D0A's every 80 bytes.  Need to removed this from the data and not sure the command to use.  Thank you for the assistance
BJE
0
Comment
Question by:bje
5 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 33482281
Is that 80 bytes counting the 0D0A's, or does adding 0D0A's make it 82 bytes?
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 33483492
0D0A is CrLf. I.e. these are fixed length lines of text in DOS format. Unless there is binary content, you should use the FTP "ascii" command before "get" of files in your FTP session - this should remove the Cr characters. You are effectively then left with punched card images - what do you want to do with them next?
0
 

Author Comment

by:bje
ID: 33510411
0D0A makes it 82 bytes
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 33516193
You have 80-column punched card images then. Can you confirm that they contain only Ascii data? (i.e. only printable characters). The command less file will pretty-much tell you that- non-ascii characters will display in reverse video. You really want to remove the Cr characters - the dos2unix utility http://sourceforge.net/projects/dos2unix/ will do that.
What else exactly do you want to do with the data?
0
 
LVL 19

Accepted Solution

by:
simon3270 earned 500 total points
ID: 33540413
There are various ways to do this, depending onthe content of the rest of the file.  If the file only contains 0D and 0A at the ends of these 80-character lines, and not elsewhere, then to remove all 0D and 0A characters (whether or not they appear together):

    tr -d '\015\012' < ftped_file > result_file

(015 and 012 are the octal equivalents of 0x0D and 0x0A)

You might think you could use "cut" to drop the 8st and 82nd characters from all lines:
 
   cut -c1-80 < ftped_file > result-file

but this just replaces the 0D0A with 0A (it drops the 0D0A characters, then appends a 0A newline when it writes the string out).  One problem with most GNU/Linux tools is that they are line-based, so the trailing 0A is handled almost silently.

A Perl alternative which treats the file as a binary one is shown below.  This script takes two arguments - the input file (with the 0D0A) and the output file. The script reads up to 82 characters from the input file.  If the line is 82 characters, it just takes the first 80.  If the line is less than 80 characters, it takes the whole line.
my $buf;

open infile, $ARGV[0] or die "\nError opening $ARGV[0] for reading: $!\n";
binmode infile;
open outfile, ">$ARGV[1]" or die "\nError opening $ARGV[1] for writing: $!\n";
binmode outfile;

while (1) {
  read (infile, $buf, 82);
  if (length($buf) == 0) {last;}
  if (length($buf) == 82) {
    $buf = substr $buf, 0, 80;
  }
  printf outfile "%s", $buf;
};

close infile or die "Error closing $ARGV[0]: $!\n";
close outfile or die "Error closing $ARGV[1]: $!\n";

Open in new window

0

Featured Post

Announcing the Most Valuable Experts of 2016

MVEs are more concerned with the satisfaction of those they help than with the considerable points they can earn. They are the types of people you feel privileged to call colleagues. Join us in honoring this amazing group of Experts.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
CRON statement to run every 3 minutes except on Monday morning 12am to 2am 7 94
dog bark java program 15 94
no14 challenge 14 66
EvenOdd challenge 10 105
This is to be the first in a series of articles demonstrating the development of a complete windows based application using the MFC classes.  I’ll try to keep each article focused on one (or a couple) of the tasks that one may meet.   Introductio…
If you use Adobe Reader X it is possible you can't open OLE PDF documents in the standard. The reason is the 'save box mode' in adobe reader X. Many people think the protected Mode of adobe reader x is only to stop the write access. But this fe…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
Established in 1997, Technology Architects has become one of the most reputable technology solutions companies in the country. TA have been providing businesses with cost effective state-of-the-art solutions and unparalleled service that is designed…

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now