[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now


comparing two pipe delimited files

Posted on 2012-09-18
Medium Priority
Last Modified: 2012-10-04

  I was looking for a quick but reliable way to compare two flat files A and B. Both are pipe delimited. They have the same columns and hence same data format and ordering.
I am looking to see if file A differs from file B in any columns 1-199. Both have a total of 200 columns and the last column values are known to be different.

Is there a way to do this in unix?
Question by:LuckyLucks
  • 2
LVL 27

Accepted Solution

wilcoxon earned 2000 total points
ID: 38410287
This should do it...
cut -d \| -f 1-199 < file1 > /tmp/cut1
cut -d \| -f 1-199 < file2 > /tmp/cut2
diff -ub /tmp/cut1 /tmp/cut2

Open in new window

Alternately, it would be simple to write up a perl script (or likely awk) to do this.

Author Comment

ID: 38410585
>head -10 results.txt
< |||||||||||||||||||||||||||||||||||
< |-12.055556|0.090909|-0.818182|||||228.000000|0.000000|15.000000|1205.000000||0.655589|-2.217391|-1.565217|-6.141079||||||||||||||||||-51.000000|
> |||||||||||||||||||||||||||||||||||1
> 778|-12.055556|0.090909|-0.818182||||1|228.000000|0.000000|15.000000|1205.000000||0.655589|-2.217391|-1.565217|-6.141079||||||||||||||||||-51.000000|2
< |||||||||||||||||||||||||||||||||||
< |||||||||||||||||||||||||||||||||||

WOuld you have an idea how to read this difference. I compared 98-133.
LVL 27

Expert Comment

ID: 38410900
The top section (above ---) is from file 1 (indicated by the < at the start of the line) and the bottom section is from file 2 (indicated by the >).

You could try "diff -cb" instead for a different output format you might like better.

If you want to know specifically which fields differ, you'll have to use perl, awk, or some other programming language (sort of in the case of awk).

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Many time we need to work with multiple files all together. If its windows system then we can use some GUI based editor to accomplish our task. But what if you are on putty or have only CLI(Command Line Interface) as an option to  edit your files. I…
Why Shell Scripting? Shell scripting is a powerful method of accessing UNIX systems and it is very flexible. Shell scripts are required when we want to execute a sequence of commands in Unix flavored operating systems. “Shell” is the command line i…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Six Sigma Control Plans
Suggested Courses

873 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question