Solved

Identifying difference between two text files (linux)

Posted on 2009-05-20
10
911 Views
Last Modified: 2013-12-16
Hi,

I have two text files, A and B. What I'm after is a nice little command line utility that can output:

- What is in B that is not in A.
- What is in A that is not in B.
- What is common to both.

... as three separate simple text file reports. I'm not after a standard 'diff' output which has the "<" or the ">" type comments, just nice simple clear lists.

Anyone help?

Thanks,
B
0
Comment
Question by:bcops
  • 5
  • 3
  • 2
10 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 24429507
if the files are sorted, you can use
comm
0
 

Author Comment

by:bcops
ID: 24429634
Thanks, but I think it needs to be more sophisticated than that.

The data is collections of URL's. So, the files are of the format:

ColumnA: source URL
ColumnB: destination URL
Comma separated

So sorting doesn't necessarily help.
Thanks though.


0
 
LVL 84

Expert Comment

by:ozo
ID: 24429680
so what would in B that is not in A. and  in A that is not in B and  common to both mean?

would sorting help after replacing commas with newlines?
0
 
LVL 48

Expert Comment

by:Tintin
ID: 24429700
Just to clarify, files A and B have entries like the following?

http://example.com/page1.html,http://example.com/page2.html

0
 

Author Comment

by:bcops
ID: 24429706

1) ozo:
so what would in B that is not in A. and  in A that is not in B and  common to both mean?
>> It would mean which lines, which pairings of source/destn url were in B and not A, etc

would sorting help after replacing commas with newlines?
>> Not really. It's the identification of different pairings I'm after.

2) Tintin: yup - your example is correct.

Thanks,
B

0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 84

Expert Comment

by:ozo
ID: 24429765
I'm still not clear on why you say that a comm of sorted files doesn't help.
Could you give examples of file A, file B, and the three separate simple text file reports that you would want to produce from them?
0
 
LVL 48

Accepted Solution

by:
Tintin earned 350 total points
ID: 24429852
comm -32 a b >in-a-not-in-b
comm -31 a b '>in-b-not-in-a
comm -21 a b >common-to-bot
0
 
LVL 84

Expert Comment

by:ozo
ID: 24429895
That was my first suggestion, but then it was said that it needs to be more sophisticated, and that comm would not help even if the files were sorted.
0
 

Author Comment

by:bcops
ID: 24430196
hi ozo and tintin,

Tintin's latest suggestion does broadly seem to work, one or two items that slip through - so thanks.
Can't find any documentation to explain what -32, -31, and -21 do. What do they do?

ozo - sorry if this is what you meant. You weren't explicit enough for me though ....

B
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 150 total points
ID: 24430239
I assumed you had access to
man comm

NAME
     comm -- select or reject lines common to two files

SYNOPSIS
     comm [-123] file1 file2

DESCRIPTION
     The comm utility reads file1 and file2, which should be sorted lexically,
     and produces three text columns as output: lines only in file1; lines
     only in file2; and lines in both files.

     The filename ``-'' means the standard input.

     The following options are available:

     -1      Suppress printing of column 1.

     -2      Suppress printing of column 2.

     -3      Suppress printing of column 3.

But you were not explicit about why that wouldn't work or what you would need that was more sophisticated.t, or the one or two items that slip through
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Setting up Secure Ubuntu server on VMware 1.      Insert the Ubuntu Server distribution CD or attach the ISO of the CD which is in the “Datastore”. Note that it is important to install the x64 edition on servers, not the X86 editions. 2.      Power on th…
I. Introduction There's an interesting discussion going on now in an Experts Exchange Group — Attachments with no extension (http://www.experts-exchange.com/discussions/210281/Attachments-with-no-extension.html). This reminded me of questions tha…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now