Solved

narrowing down a file

Posted on 2011-03-07
6
260 Views
Last Modified: 2012-05-11
I have two files.

File1.txt is tab-separated and contains several fields.
File2.txt contains just one field.

I want to get a subset of file 1, such that field 3 of it matches exactly(1) one of the lines in File2.txt.  

The match has to be complete, not partial (so if one says foxnews.com/blah.html and the other says foxnews.com or vice versa - that would not be a match).  Otherwise, I could have just done grep –F –f File2.txt File1.txt.

How would I do that in a shell script?
0
Comment
Question by:aturetsky
  • 3
  • 2
6 Comments
 
LVL 16

Accepted Solution

by:
sjklein42 earned 300 total points
ID: 35064864
Should work.  Save as "joinIt.pl".

Without any test data, hard to test.  If it doesn't work, please post some test data.

# usage:   perl joinIt.pl infile.txt keyfile.txt

$infile = shift(@ARGV);
$keyfile = shift(@ARGV);

if ( ! open(INFILE, "<$infile") ) { die "*** can't open $infile: $!\n"; }
if ( ! open(KEYFILE, "<$keyfile") ) { die "*** can't open $keyfile: $!\n"; }

while ( <KEYFILE> )
{
	s/[\r\n]//g;
	$key{$_} = 1;
}

while ( <INFILE> )
{
	s/[\r\n]//g;
	@x = split(/\t/);
	if ( $key{$x[2]} ) { print "$_\n"; }
}

Open in new window

0
 
LVL 8

Expert Comment

by:point_pleasant
ID: 35069520
here is a shellscript that should work too, the delimiter in the cut commaned is a tab


for i in `cat file1 | cut -f3 -d'       '`
do
        grep -x $i file2
done
0
 
LVL 1

Author Comment

by:aturetsky
ID: 35072123
thanks, sjklein42 - it worked!

can I ask you - if I wanted to do the exact opposite and get only what's not in the keyfile - what would that look like?
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 1

Author Comment

by:aturetsky
ID: 35072688
actually, I think this might do the job (for that reverse task):


# usage:   perl joinIt.pl infile.txt keyfile.txt

$infile = shift(@ARGV);
$keyfile = shift(@ARGV);

if ( ! open(INFILE, "<$infile") ) { die "*** can't open $infile: $!\n"; }
if ( ! open(KEYFILE, "<$keyfile") ) { die "*** can't open $keyfile: $!\n"; }

while ( <KEYFILE> )
{
       s/[\r\n]//g;
       $key{$_} = 1;
}

while ( <INFILE> )
{
       s/[\r\n]//g;
       @x = split(/\t/);
       if ( not exists($key{$x[2]}) ) { print "$_\n"; }
}
0
 
LVL 8

Expert Comment

by:point_pleasant
ID: 35073184
the shell script to do the reverse would be as follows.  the echo statement is there to seperate each column 3 element from file1

for i in `cat file1 | cut -f3 -d'       '`
do
        echo ================== $i from file1 ======================
        grep -x -v $i file2
done
0
 
LVL 8

Assisted Solution

by:point_pleasant
point_pleasant earned 200 total points
ID: 35073548
sorry didn'd realize you wanted the whole line from file1.  Here is a shell script to do it.  if you want the reverse just add the -v option to grep


while read i
do
        col3=`echo $i | awk '{ print $3 }'`
        found=`grep -x -v $col3 file2`
        if [ "$found" != "" ]
        then
                echo $i
        fi
done < file1
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

Suggested Solutions

Background Still having to process all these year-end "csv" files received from all these sources (including Government entities), sometimes we have the need to examine the contents due to data error, etc... As a "Unix" shop, our only readily …
Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now