Solved

narrowing down a file

Posted on 2011-03-07
6
263 Views
Last Modified: 2012-05-11
I have two files.

File1.txt is tab-separated and contains several fields.
File2.txt contains just one field.

I want to get a subset of file 1, such that field 3 of it matches exactly(1) one of the lines in File2.txt.  

The match has to be complete, not partial (so if one says foxnews.com/blah.html and the other says foxnews.com or vice versa - that would not be a match).  Otherwise, I could have just done grep –F –f File2.txt File1.txt.

How would I do that in a shell script?
0
Comment
Question by:aturetsky
  • 3
  • 2
6 Comments
 
LVL 16

Accepted Solution

by:
sjklein42 earned 300 total points
ID: 35064864
Should work.  Save as "joinIt.pl".

Without any test data, hard to test.  If it doesn't work, please post some test data.

# usage:   perl joinIt.pl infile.txt keyfile.txt

$infile = shift(@ARGV);
$keyfile = shift(@ARGV);

if ( ! open(INFILE, "<$infile") ) { die "*** can't open $infile: $!\n"; }
if ( ! open(KEYFILE, "<$keyfile") ) { die "*** can't open $keyfile: $!\n"; }

while ( <KEYFILE> )
{
	s/[\r\n]//g;
	$key{$_} = 1;
}

while ( <INFILE> )
{
	s/[\r\n]//g;
	@x = split(/\t/);
	if ( $key{$x[2]} ) { print "$_\n"; }
}

Open in new window

0
 
LVL 8

Expert Comment

by:point_pleasant
ID: 35069520
here is a shellscript that should work too, the delimiter in the cut commaned is a tab


for i in `cat file1 | cut -f3 -d'       '`
do
        grep -x $i file2
done
0
 
LVL 1

Author Comment

by:aturetsky
ID: 35072123
thanks, sjklein42 - it worked!

can I ask you - if I wanted to do the exact opposite and get only what's not in the keyfile - what would that look like?
0
Secure Your Active Directory - April 20, 2017

Active Directory plays a critical role in your company’s IT infrastructure and keeping it secure in today’s hacker-infested world is a must.
Microsoft published 300+ pages of guidance, but who has the time, money, and resources to implement? Register now to find an easier way.

 
LVL 1

Author Comment

by:aturetsky
ID: 35072688
actually, I think this might do the job (for that reverse task):


# usage:   perl joinIt.pl infile.txt keyfile.txt

$infile = shift(@ARGV);
$keyfile = shift(@ARGV);

if ( ! open(INFILE, "<$infile") ) { die "*** can't open $infile: $!\n"; }
if ( ! open(KEYFILE, "<$keyfile") ) { die "*** can't open $keyfile: $!\n"; }

while ( <KEYFILE> )
{
       s/[\r\n]//g;
       $key{$_} = 1;
}

while ( <INFILE> )
{
       s/[\r\n]//g;
       @x = split(/\t/);
       if ( not exists($key{$x[2]}) ) { print "$_\n"; }
}
0
 
LVL 8

Expert Comment

by:point_pleasant
ID: 35073184
the shell script to do the reverse would be as follows.  the echo statement is there to seperate each column 3 element from file1

for i in `cat file1 | cut -f3 -d'       '`
do
        echo ================== $i from file1 ======================
        grep -x -v $i file2
done
0
 
LVL 8

Assisted Solution

by:point_pleasant
point_pleasant earned 200 total points
ID: 35073548
sorry didn'd realize you wanted the whole line from file1.  Here is a shell script to do it.  if you want the reverse just add the -v option to grep


while read i
do
        col3=`echo $i | awk '{ print $3 }'`
        found=`grep -x -v $col3 file2`
        if [ "$found" != "" ]
        then
                echo $i
        fi
done < file1
0

Featured Post

On Demand Webinar - Networking for the Cloud Era

This webinar discusses:
-Common barriers companies experience when moving to the cloud
-How SD-WAN changes the way we look at networks
-Best practices customers should employ moving forward with cloud migration
-What happens behind the scenes of SteelConnect’s one-click button

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Utilizing an array to gracefully append to a list of EmailAddresses
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…

740 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question