Solved

Searching for strings in one file from another file

Posted on 2013-01-08
16
299 Views
Last Modified: 2013-01-09
I have a file A  with four Id's . I need to check if these Id's  exist in file B and send a simple email if a particular id does not exist in B
For example file A has
Id's
1001
1002
1003
1004

If Id 1003 does not exist in file B a email should be sent out that 1003 does not exist

the lenght of Id is fixed i.e 4, and Id in file B is the first column  and file B could be having 1000's of rows
Attached are sample files
ids.txt
trailer.txt
0
Comment
Question by:uco
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 6
  • 4
16 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 38756014
while read ID; do
 if ! grep -q ^$ID fileB; then
   echo $ID not found in fileB
 fi ; done < fileA | mail -s "Missing ID(s)" recipient@domain.tld
0
 

Author Comment

by:uco
ID: 38756130
we are giving fileA  as input at the end ?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 38756150
Yep, right after "done", as "input" for the search patterns.

The file to search in is the one just after "...grep -q ^$ID"
0
Free learning courses: Active Directory Deep Dive

Get a firm grasp on your IT environment when you learn Active Directory best practices with Veeam! Watch all, or choose any amount, of this three-part webinar series to improve your skills. From the basics to virtualization and backup, we got you covered.

 
LVL 84

Accepted Solution

by:
ozo earned 500 total points
ID: 38756223
awk '!/ID/{sub(/\r/,"");print}' ids.txt | sort > ids.sort
awk '{print $1}' trailer.txt | sort > trailer.sort
comm -23 ids.sort trailer.sort
0
 

Author Comment

by:uco
ID: 38756524
while read ID; do
 if ! grep -q ^$ID fileB; then
   echo $ID not found in fileB
 fi ; done < fileA | mail -s "Missing ID(s)" recipient@domain.tld

It is only showing 1001 is missing in fileB, eventhough 1003 is also missing.
Also it has to look only the first column in fileB and column lenght is 4.
email has to be sent only once as summary of id's missing

thanks
0
 
LVL 84

Expert Comment

by:ozo
ID: 38756553
I see a 1003 in the last line of trailer.txt
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 38756563
1) 1003 is not missing, at least not in the sample you posted. Look at the last line.

2) It does look only at the first column, see the caret  ("^$ID"). Length 4 is given by the length of the pattern in fileA, which you said is 4.

3) Email is sent only once. See the pipe to mail at the very end (after ... done).

Why do people post flack before testing?
0
 

Author Comment

by:uco
ID: 38756656
I did test it , Please use this file , for this one it is showing 1001 is missing , 1004 is also missing
Thanks for the  help
trailer.txt
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 38756676
This is what my script gives with your new file:

1001 not found in trailer.txt
1004 not found in trailer.txt

Where's the problem?
0
 

Author Comment

by:uco
ID: 38756779
this is what I am doing
I am appending mail content  to empty file

while read ID; do
 if ! grep -q ^$ID trailer.txt; then
   echo  "  $ID not found in trailer        :    " >> test_file.txt  
 fi ; done < ids.txt
 mailx -s "ISSUE: " -c $Mail    < test_file.txt

Anything wrong here?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 38756816
Nothing wrong, but why this intermediate file?

My original script accomplishes the same result without creating such a file inbetween.

Your mailx command lacks the recipient's address. "-c" specifies the recipient(s) of a copy.
0
 

Author Comment

by:uco
ID: 38756917
Ok I removed the extra file , I am running exactly the same script as you gave  but still I am getting the

ID not found in trailer :
1001 not found in trailer :

2nd
I executed with trailer file completely deleted, but then it is only showing until 1003
ID not found ID :
1001 not found ID :
1002 not found ID :
1003 not found ID :

I think it is not going to the end of the fileA (ID file)
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 38756992
1)  Does this line "ID not found in trailer :" bother you?
That's because you have the header line "ID" in ids.txt but not in the trailer file. We could easily filter it out if needed:

while read ID; do
 if ! grep -q ^$ID trailer.txt; then
    [[ "$ID" != "ID" ]] && echo $ID not found in trailer.txt
 fi ; done < ids.txt | mail -s "Missing ID(s)" recipient@domain.tld

2) I think I can't understand your second issue.

With an empty "trailer.txt" file I get this (old version):

ID not found in trailer.txt
1001 not found in trailer.txt
1002 not found in trailer.txt
1003 not found in trailer.txt
1004 not found in trailer.txt

and with trailer.txt completely deleted (= removed) I additionally get error messages from "grep", which go to the terminal but not into the mail:

grep: trailer.txt: No such file or directory
ID not found in trailer.txt
grep: trailer.txt: No such file or directory
1001 not found in trailer.txt
grep: trailer.txt: No such file or directory
1002 not found in trailer.txt
grep: trailer.txt: No such file or directory
1003 not found in trailer.txt
grep: trailer.txt: No such file or directory
1004 not found in trailer.txt

I admit that the second output isn't really "nice" but it isn't wrong either, and we can easily suppress those error messages, if desired.

In any case ids.txt is read up to the end. Are you sure that you're using the version of ids.txt which you posted in your question?
0
 
LVL 84

Expert Comment

by:ozo
ID: 38757215
Which shell are you using, and which version?
Some versions of some shells have had issues with redirection into a do loop
0
 

Author Comment

by:uco
ID: 38758626
I am using bash shell

I have no issue with "ID not found in trailer :"

I am using the same version of id file that I sent . But number of my id's could increase or decrease

This one gives correct results , but I need to print  them like
"ID 1001 is missing"  


awk '!/ID/{sub(/\r/,"");print}' ids.txt | sort > ids.sort
awk '{print $1}' trailer.txt | sort > trailer.sort
comm -23 ids.sort trailer.sort
ids.txt
trailer.txt
0
 
LVL 84

Expert Comment

by:ozo
ID: 38759847
comm -23 ids.sort trailer.sort | awk '{print "ID "$1" is missing" }'
0

Featured Post

Simplifying Server Workload Migrations

This use case outlines the migration challenges that organizations face and how the Acronis AnyData Engine supports physical-to-physical (P2P), physical-to-virtual (P2V), virtual to physical (V2P), and cross-virtual (V2V) migration scenarios to address these challenges.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I. Introduction There's an interesting discussion going on now in an Experts Exchange Group — Attachments with no extension (http://www.experts-exchange.com/discussions/210281/Attachments-with-no-extension.html). This reminded me of questions tha…
Google Drive is extremely cheap offsite storage, and it's even possible to get extra storage for free for two years.  You can use the free account 15GB, and if you have an Android device..when you install Google Drive for the first time it will give…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question