Solved

How to unwrap data being blocked at 80 bytes

Posted on 2010-08-19
5
530 Views
Last Modified: 2013-12-26
I am using a FTP script to receiving data from a partner and the data is being blocked with 0D0A's every 80 bytes.  Need to removed this from the data and not sure the command to use.  Thank you for the assistance
BJE
0
Comment
Question by:bje
5 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 33482281
Is that 80 bytes counting the 0D0A's, or does adding 0D0A's make it 82 bytes?
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 33483492
0D0A is CrLf. I.e. these are fixed length lines of text in DOS format. Unless there is binary content, you should use the FTP "ascii" command before "get" of files in your FTP session - this should remove the Cr characters. You are effectively then left with punched card images - what do you want to do with them next?
0
 

Author Comment

by:bje
ID: 33510411
0D0A makes it 82 bytes
0
 
LVL 34

Expert Comment

by:Duncan Roe
ID: 33516193
You have 80-column punched card images then. Can you confirm that they contain only Ascii data? (i.e. only printable characters). The command less file will pretty-much tell you that- non-ascii characters will display in reverse video. You really want to remove the Cr characters - the dos2unix utility http://sourceforge.net/projects/dos2unix/ will do that.
What else exactly do you want to do with the data?
0
 
LVL 19

Accepted Solution

by:
simon3270 earned 500 total points
ID: 33540413
There are various ways to do this, depending onthe content of the rest of the file.  If the file only contains 0D and 0A at the ends of these 80-character lines, and not elsewhere, then to remove all 0D and 0A characters (whether or not they appear together):

    tr -d '\015\012' < ftped_file > result_file

(015 and 012 are the octal equivalents of 0x0D and 0x0A)

You might think you could use "cut" to drop the 8st and 82nd characters from all lines:
 
   cut -c1-80 < ftped_file > result-file

but this just replaces the 0D0A with 0A (it drops the 0D0A characters, then appends a 0A newline when it writes the string out).  One problem with most GNU/Linux tools is that they are line-based, so the trailing 0A is handled almost silently.

A Perl alternative which treats the file as a binary one is shown below.  This script takes two arguments - the input file (with the 0D0A) and the output file. The script reads up to 82 characters from the input file.  If the line is 82 characters, it just takes the first 80.  If the line is less than 80 characters, it takes the whole line.
my $buf;

open infile, $ARGV[0] or die "\nError opening $ARGV[0] for reading: $!\n";
binmode infile;
open outfile, ">$ARGV[1]" or die "\nError opening $ARGV[1] for writing: $!\n";
binmode outfile;

while (1) {
  read (infile, $buf, 82);
  if (length($buf) == 0) {last;}
  if (length($buf) == 82) {
    $buf = substr $buf, 0, 80;
  }
  printf outfile "%s", $buf;
};

close infile or die "Error closing $ARGV[0]: $!\n";
close outfile or die "Error closing $ARGV[1]: $!\n";

Open in new window

0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Introduction: Displaying information on the statusbar.   Continuing from the third article about sudoku.   Open the project in visual studio. Status bar – let’s display the timestamp there.  We need to get the timestamp from the document s…
Introduction: Dialogs (2) modeless dialog and a worker thread.  Handling data shared between threads.  Recursive functions. Continuing from the tenth article about sudoku.   Last article we worked with a modal dialog to help maintain informat…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
Access reports are powerful and flexible. Learn how to create a query and then a grouped report using the wizard. Modify the report design after the wizard is done to make it look better. There will be another video to explain how to put the final p…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now