Solved

How To grep for a list of options in Solaris

Posted on 2007-11-28
9
331 Views
Last Modified: 2013-12-27
Hi,
I am trying to figure out how to grep through log files, but rather than grepping for one item, I want to search for a number of items. I have tried a number of different approaches, but none have worked.

What I have is a list of domain names in a text file with one name per line. I then want to search through a number of log files looking for any log entry that matches any of the domains in the list. I also have a similar list, though it contains IP addresses instead of names that I also need to search for.

Does anyone know of a good way to search for a number of items in one command, or do I have to grep for each one - which would be extremely time consuming?

I would greatly appreciate any help on this.

Thanks,
Jeff
0
Comment
Question by:jpetter
  • 5
  • 4
9 Comments
 
LVL 40

Accepted Solution

by:
omarfarid earned 500 total points
ID: 20366742
Hi,

grep can take the patterns / strings from a file. So, you may put your search strings in a file and then

grep -f stringsfile file(s)

stringsfile is the file that has the strings or patterns

file(s) file(s) that contain the logs

you may also use the -F option with the command

e.g.

grep -F -f stringsfile logfile

http://unixhelp.ed.ac.uk/CGI/man-cgi?grep
0
 

Author Comment

by:jpetter
ID: 20366841
Hi,
Thanks for responding so quickly. That is very similar to what I have been trying - almost identical. Here is one of my attempts:
bash-2.05$ gzcat logfilename.log.gz | /usr/xpg4/bin/grep -F -f test.txt, but rather than returning only those entries that match the file entries, it returns the entire file.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20366878
OK,

can you provide sample of those files (you do not have to give real info, and can replace by dummy values)?

I will try to see if I can do something. It could be the (.) in the IPs or domains
0
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 

Author Comment

by:jpetter
ID: 20367229
Thanks. I have changed the IP's and names, but this should give you an idea.

These seven lines you'll need to save to a file and then gzip: (they look like alot more than seven lines with all the wraps)
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_NC_MISS GET http://catalog.video.syndication.msn.com/videoservice/videoByTag.ashx?tag=ENAPus_ENAPus&ns=MSNVideo_Top_Cat&mk=en-ap&sd=-1&sf=ActiveStartDate&ps=5&rand=2080 HTTP/1.0 200 11797 http://img.video.ap.org/p/s/p2/lg_hz_search.swf Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.0.3705;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
192.176.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://www.priceline.com/zp/utils/transport.js HTTP/1.0 200 71520 http://www.priceline.com/hotels/Lang/en-us/region_star_price.asp?session_key=5C0011AC5D0011AC20071127182935ba1e40674203&plf=pcln Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - Travel
10.1.1.20 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://samantha48616e61.com/images/article_tn.jpg HTTP/1.0 304 289 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
10.2.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_MISS GET http://samantha48616e61.com/images/article2_tn.jpg HTTP/1.0 304 286 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://stb.msn.com/i/48/86F1396496DFE1BAD68AB5F28409.gif HTTP/1.0 200 782 http://www.msnbc.msn.com/id/3032113/?ta=y Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
10.2.2.2 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://money.msn.com/MSNQuoteData.xml HTTP/1.0 200 414 - Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322;%20%20MSN%209.0;MSN%209.1;%20MSNbVZ02;%20MSNmen-us;%20MSNcOTH) - Search_Engines/Portals
192.176.5.5 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://image.weather.com/web/multimedia/images/miscellaneous/ec206_open.jpg HTTP/1.0 200 1826 http://www.weather.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - News/Media

Next, I created a file called tmp.txt that looked like this:
192.176.1.10
192.176.1.15

Here is the command I ran:
bash-2.05$ gzcat tmplogfile.txt.gz | /usr/xpg4/bin/grep -F -f tmp.txt

and rather than returning the three lines that match, it returns all seven.

If you could spot something, I would greatly appreciate it.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20367409
Hi,

Which solaris version do you have?

I tried it on my system and it works perfect !

I have solaris 8 with latest patches.

the /usr/bin/grep does not take the -f or -F , so I used egrep and /usr/xpg4/bin/grep and both gave correct results (3 lines with correct ips).

So check your system or try the egrep
0
 

Author Comment

by:jpetter
ID: 20367594
We are running Solaris SunOS 5.9 with all the patches.

It doesn't work on this system, as the man pages would suggest. Also, in egrep, I didn't see a switch similar to the -F that would treat the file entries as strings rather than expressions.

Thanks for trying. I'll have to see if there is another way to do it as no matter how I try it, I return the whole file.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20367621
Hi,

The -F option is not needed (I did not use it). Try without it.

What is your shell? I am using ksh.

0
 

Author Comment

by:jpetter
ID: 20368048
I was using the bash shell, but tried ksh and had the same results.

Thanks,
Jeff
0
 

Author Comment

by:jpetter
ID: 20373443
OK, I found out what the problem was. The "tmp.txt" file that I was using had a blank line at the end. As soon as I removed that line, it worked as expected.

Thanks for your help,
Jeff
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hello fellow BSD lovers, I've created a patch process for patching openjdk6 for BSD (FreeBSD specifically), although I tried to keep all BSD versions in mind when creating my patch. Welcome to OpenJDK6 on BSD First let me start with a little …
FreeBSD on EC2 FreeBSD (https://www.freebsd.org) is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

790 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question