Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

Unix diff command to find the minus of two text files

Posted on 2014-03-17
5
Medium Priority
?
1,414 Views
Last Modified: 2014-03-21
I want to print the content of file 1a that are not present in the file 1b. Both files contain one line of similar pattered text.
Example:
$ cat 1a
1234
3456
4567
$ cat 1b
1234
5566
9999
3456
$ grep -vf 1a 1b
5566
9999


$ grep -vf 1b 1a shows those present in 1a but not in 1b. And it works perfectly, But when the files are big (1000K+ records each), the above diff command hangs. Could you suggest workaround or alternate solution that might work? Thanks you.
0
Comment
Question by:toooki
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 2000 total points
ID: 39935760
You could use "comm". The drawback here is that both files must be sorted.

sort 1b > 1b.sort

sort 1a | comm -2 -3 - 1b.sort

will show the records present in file 1a but not in 1b(.sort)

I don't think that your grep command actually "hangs". Probably it just takes quite a long time to complete.
0
 
LVL 84

Expert Comment

by:ozo
ID: 39935771
Your example seems to show a grep command, not a diff command.
If we can use other commands, this should work:
 perl -lne '@ARGV?$s{$_}++:$s{$_}||print' 1a 1b
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39935789
If you want to use "diff" then this one:

diff 1b 1a | grep "^>"

will show the lines of 1a which are not in 1b, preceeded by ">".

This one

diff 1a 1b |grep "^<"

will do the same, but the lines in question will be preceeded by "<".

This will remove the prefix:

diff 1a 1b |awk '/^</ {print $2}'
0
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 39938069
Off topic comment deleted.

Gerwin Jansen
EE Topic Advisor
0
 

Author Comment

by:toooki
ID: 39946613
Many thanks to all.
sort 1b > 1b.sort
sort 1a | comm -2 -3 - 1b.sort

The above worked for me!
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

WARNING:   If you follow the instructions here, you will wipe out your VTP and VLAN configurations.  Make sure you have backed up your switch!!! I recently had some issues with a few low-end Cisco routers (RV325) and I opened a case with Cisco TA…
This article is in regards to the Cisco QSFP-4SFP10G-CU1M cables, which are designed to uplink/downlink 40GB ports to 10GB SFP ports. I recently experienced this and found very little configuration documentation on how these are supposed to be confi…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…
Monitoring a network: why having a policy is the best policy? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the enormous benefits of having a policy-based approach when monitoring medium and large networks. Software utilized in this v…
Suggested Courses

609 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question