Solved

How To grep for a list of options in Solaris

Posted on 2007-11-28
9
325 Views
Last Modified: 2013-12-27
Hi,
I am trying to figure out how to grep through log files, but rather than grepping for one item, I want to search for a number of items. I have tried a number of different approaches, but none have worked.

What I have is a list of domain names in a text file with one name per line. I then want to search through a number of log files looking for any log entry that matches any of the domains in the list. I also have a similar list, though it contains IP addresses instead of names that I also need to search for.

Does anyone know of a good way to search for a number of items in one command, or do I have to grep for each one - which would be extremely time consuming?

I would greatly appreciate any help on this.

Thanks,
Jeff
0
Comment
Question by:jpetter
  • 5
  • 4
9 Comments
 
LVL 40

Accepted Solution

by:
omarfarid earned 500 total points
ID: 20366742
Hi,

grep can take the patterns / strings from a file. So, you may put your search strings in a file and then

grep -f stringsfile file(s)

stringsfile is the file that has the strings or patterns

file(s) file(s) that contain the logs

you may also use the -F option with the command

e.g.

grep -F -f stringsfile logfile

http://unixhelp.ed.ac.uk/CGI/man-cgi?grep
0
 

Author Comment

by:jpetter
ID: 20366841
Hi,
Thanks for responding so quickly. That is very similar to what I have been trying - almost identical. Here is one of my attempts:
bash-2.05$ gzcat logfilename.log.gz | /usr/xpg4/bin/grep -F -f test.txt, but rather than returning only those entries that match the file entries, it returns the entire file.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20366878
OK,

can you provide sample of those files (you do not have to give real info, and can replace by dummy values)?

I will try to see if I can do something. It could be the (.) in the IPs or domains
0
 

Author Comment

by:jpetter
ID: 20367229
Thanks. I have changed the IP's and names, but this should give you an idea.

These seven lines you'll need to save to a file and then gzip: (they look like alot more than seven lines with all the wraps)
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_NC_MISS GET http://catalog.video.syndication.msn.com/videoservice/videoByTag.ashx?tag=ENAPus_ENAPus&ns=MSNVideo_Top_Cat&mk=en-ap&sd=-1&sf=ActiveStartDate&ps=5&rand=2080 HTTP/1.0 200 11797 http://img.video.ap.org/p/s/p2/lg_hz_search.swf Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.0.3705;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
192.176.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://www.priceline.com/zp/utils/transport.js HTTP/1.0 200 71520 http://www.priceline.com/hotels/Lang/en-us/region_star_price.asp?session_key=5C0011AC5D0011AC20071127182935ba1e40674203&plf=pcln Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - Travel
10.1.1.20 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://samantha48616e61.com/images/article_tn.jpg HTTP/1.0 304 289 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
10.2.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_MISS GET http://samantha48616e61.com/images/article2_tn.jpg HTTP/1.0 304 286 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://stb.msn.com/i/48/86F1396496DFE1BAD68AB5F28409.gif HTTP/1.0 200 782 http://www.msnbc.msn.com/id/3032113/?ta=y Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
10.2.2.2 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://money.msn.com/MSNQuoteData.xml HTTP/1.0 200 414 - Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322;%20%20MSN%209.0;MSN%209.1;%20MSNbVZ02;%20MSNmen-us;%20MSNcOTH) - Search_Engines/Portals
192.176.5.5 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://image.weather.com/web/multimedia/images/miscellaneous/ec206_open.jpg HTTP/1.0 200 1826 http://www.weather.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - News/Media

Next, I created a file called tmp.txt that looked like this:
192.176.1.10
192.176.1.15

Here is the command I ran:
bash-2.05$ gzcat tmplogfile.txt.gz | /usr/xpg4/bin/grep -F -f tmp.txt

and rather than returning the three lines that match, it returns all seven.

If you could spot something, I would greatly appreciate it.

Thanks,
Jeff
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 40

Expert Comment

by:omarfarid
ID: 20367409
Hi,

Which solaris version do you have?

I tried it on my system and it works perfect !

I have solaris 8 with latest patches.

the /usr/bin/grep does not take the -f or -F , so I used egrep and /usr/xpg4/bin/grep and both gave correct results (3 lines with correct ips).

So check your system or try the egrep
0
 

Author Comment

by:jpetter
ID: 20367594
We are running Solaris SunOS 5.9 with all the patches.

It doesn't work on this system, as the man pages would suggest. Also, in egrep, I didn't see a switch similar to the -F that would treat the file entries as strings rather than expressions.

Thanks for trying. I'll have to see if there is another way to do it as no matter how I try it, I return the whole file.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20367621
Hi,

The -F option is not needed (I did not use it). Try without it.

What is your shell? I am using ksh.

0
 

Author Comment

by:jpetter
ID: 20368048
I was using the bash shell, but tried ksh and had the same results.

Thanks,
Jeff
0
 

Author Comment

by:jpetter
ID: 20373443
OK, I found out what the problem was. The "tmp.txt" file that I was using had a blank line at the end. As soon as I removed that line, it worked as expected.

Thanks for your help,
Jeff
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
pauing printer deamon in AIX 10 57
check unix curl command return value 7 84
Linux: disable vim auto-comment 7 98
Writing a UNIX script menu allowing for multiple options. 5 33
I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Every server (virtual or physical) needs a console: and the console can be provided through hardware directly connected, software for remote connections, local connections, through a KVM, etc. This document explains the different types of consol…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

895 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now