asked on

How To grep for a list of options in Solaris

Hi,
I am trying to figure out how to grep through log files, but rather than grepping for one item, I want to search for a number of items. I have tried a number of different approaches, but none have worked.

What I have is a list of domain names in a text file with one name per line. I then want to search through a number of log files looking for any log entry that matches any of the domains in the list. I also have a similar list, though it contains IP addresses instead of names that I also need to search for.

Does anyone know of a good way to search for a number of items in one command, or do I have to grep for each one - which would be extremely time consuming?

I would greatly appreciate any help on this.

Thanks,
Jeff

ASKER CERTIFIED SOLUTION

omarfarid

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

jpetter

ASKER

Hi,
Thanks for responding so quickly. That is very similar to what I have been trying - almost identical. Here is one of my attempts:
bash-2.05$ gzcat logfilename.log.gz | /usr/xpg4/bin/grep -F -f test.txt, but rather than returning only those entries that match the file entries, it returns the entire file.

Thanks,
Jeff

omarfarid

OK,

can you provide sample of those files (you do not have to give real info, and can replace by dummy values)?

I will try to see if I can do something. It could be the (.) in the IPs or domains

jpetter

ASKER

Thanks. I have changed the IP's and names, but this should give you an idea.

These seven lines you'll need to save to a file and then gzip: (they look like alot more than seven lines with all the wraps)
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_NC_MISS GET http://catalog.video.syndication.msn.com/videoservice/videoByTag.ashx?tag=ENAPus_ENAPus&ns=MSNVideo_Top_Cat&mk=en-ap&sd=-1&sf=ActiveStartDate&ps=5&rand=2080 HTTP/1.0 200 11797 http://img.video.ap.org/p/s/p2/lg_hz_search.swf Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.0.3705;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
192.176.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://www.priceline.com/zp/utils/transport.js HTTP/1.0 200 71520 http://www.priceline.com/hotels/Lang/en-us/region_star_price.asp?session_key=5C0011AC5D0011AC20071127182935ba1e40674203&plf=pcln Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - Travel
10.1.1.20 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://samantha48616e61.com/images/article_tn.jpg HTTP/1.0 304 289 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
10.2.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_MISS GET http://samantha48616e61.com/images/article2_tn.jpg HTTP/1.0 304 286 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://stb.msn.com/i/48/86F1396496DFE1BAD68AB5F28409.gif HTTP/1.0 200 782 http://www.msnbc.msn.com/id/3032113/?ta=y Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
10.2.2.2 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://money.msn.com/MSNQuoteData.xml HTTP/1.0 200 414 - Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322;%20%20MSN%209.0;MSN%209.1;%20MSNbVZ02;%20MSNmen-us;%20MSNcOTH) - Search_Engines/Portals
192.176.5.5 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://image.weather.com/web/multimedia/images/miscellaneous/ec206_open.jpg HTTP/1.0 200 1826 http://www.weather.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - News/Media

Next, I created a file called tmp.txt that looked like this:
192.176.1.10
192.176.1.15

Here is the command I ran:
bash-2.05$ gzcat tmplogfile.txt.gz | /usr/xpg4/bin/grep -F -f tmp.txt

and rather than returning the three lines that match, it returns all seven.

If you could spot something, I would greatly appreciate it.

Thanks,
Jeff

omarfarid

Hi,

Which solaris version do you have?

I tried it on my system and it works perfect !

I have solaris 8 with latest patches.

the /usr/bin/grep does not take the -f or -F , so I used egrep and /usr/xpg4/bin/grep and both gave correct results (3 lines with correct ips).

So check your system or try the egrep

jpetter

ASKER

We are running Solaris SunOS 5.9 with all the patches.

It doesn't work on this system, as the man pages would suggest. Also, in egrep, I didn't see a switch similar to the -F that would treat the file entries as strings rather than expressions.

Thanks for trying. I'll have to see if there is another way to do it as no matter how I try it, I return the whole file.

Thanks,
Jeff

omarfarid

Hi,

The -F option is not needed (I did not use it). Try without it.

What is your shell? I am using ksh.

jpetter

ASKER

I was using the bash shell, but tried ksh and had the same results.

Thanks,
Jeff

jpetter

ASKER

OK, I found out what the problem was. The "tmp.txt" file that I was using had a blank line at the end. As soon as I removed that line, it worked as expected.

Thanks for your help,
Jeff