?
Solved

How To grep for a list of options in Solaris

Posted on 2007-11-28
9
Medium Priority
?
370 Views
Last Modified: 2013-12-27
Hi,
I am trying to figure out how to grep through log files, but rather than grepping for one item, I want to search for a number of items. I have tried a number of different approaches, but none have worked.

What I have is a list of domain names in a text file with one name per line. I then want to search through a number of log files looking for any log entry that matches any of the domains in the list. I also have a similar list, though it contains IP addresses instead of names that I also need to search for.

Does anyone know of a good way to search for a number of items in one command, or do I have to grep for each one - which would be extremely time consuming?

I would greatly appreciate any help on this.

Thanks,
Jeff
0
Comment
Question by:jpetter
  • 5
  • 4
9 Comments
 
LVL 40

Accepted Solution

by:
omarfarid earned 2000 total points
ID: 20366742
Hi,

grep can take the patterns / strings from a file. So, you may put your search strings in a file and then

grep -f stringsfile file(s)

stringsfile is the file that has the strings or patterns

file(s) file(s) that contain the logs

you may also use the -F option with the command

e.g.

grep -F -f stringsfile logfile

http://unixhelp.ed.ac.uk/CGI/man-cgi?grep
0
 

Author Comment

by:jpetter
ID: 20366841
Hi,
Thanks for responding so quickly. That is very similar to what I have been trying - almost identical. Here is one of my attempts:
bash-2.05$ gzcat logfilename.log.gz | /usr/xpg4/bin/grep -F -f test.txt, but rather than returning only those entries that match the file entries, it returns the entire file.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20366878
OK,

can you provide sample of those files (you do not have to give real info, and can replace by dummy values)?

I will try to see if I can do something. It could be the (.) in the IPs or domains
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 

Author Comment

by:jpetter
ID: 20367229
Thanks. I have changed the IP's and names, but this should give you an idea.

These seven lines you'll need to save to a file and then gzip: (they look like alot more than seven lines with all the wraps)
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_NC_MISS GET http://catalog.video.syndication.msn.com/videoservice/videoByTag.ashx?tag=ENAPus_ENAPus&ns=MSNVideo_Top_Cat&mk=en-ap&sd=-1&sf=ActiveStartDate&ps=5&rand=2080 HTTP/1.0 200 11797 http://img.video.ap.org/p/s/p2/lg_hz_search.swf Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.0.3705;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
192.176.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://www.priceline.com/zp/utils/transport.js HTTP/1.0 200 71520 http://www.priceline.com/hotels/Lang/en-us/region_star_price.asp?session_key=5C0011AC5D0011AC20071127182935ba1e40674203&plf=pcln Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - Travel
10.1.1.20 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://samantha48616e61.com/images/article_tn.jpg HTTP/1.0 304 289 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
10.2.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_MISS GET http://samantha48616e61.com/images/article2_tn.jpg HTTP/1.0 304 286 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://stb.msn.com/i/48/86F1396496DFE1BAD68AB5F28409.gif HTTP/1.0 200 782 http://www.msnbc.msn.com/id/3032113/?ta=y Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
10.2.2.2 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://money.msn.com/MSNQuoteData.xml HTTP/1.0 200 414 - Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322;%20%20MSN%209.0;MSN%209.1;%20MSNbVZ02;%20MSNmen-us;%20MSNcOTH) - Search_Engines/Portals
192.176.5.5 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://image.weather.com/web/multimedia/images/miscellaneous/ec206_open.jpg HTTP/1.0 200 1826 http://www.weather.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - News/Media

Next, I created a file called tmp.txt that looked like this:
192.176.1.10
192.176.1.15

Here is the command I ran:
bash-2.05$ gzcat tmplogfile.txt.gz | /usr/xpg4/bin/grep -F -f tmp.txt

and rather than returning the three lines that match, it returns all seven.

If you could spot something, I would greatly appreciate it.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20367409
Hi,

Which solaris version do you have?

I tried it on my system and it works perfect !

I have solaris 8 with latest patches.

the /usr/bin/grep does not take the -f or -F , so I used egrep and /usr/xpg4/bin/grep and both gave correct results (3 lines with correct ips).

So check your system or try the egrep
0
 

Author Comment

by:jpetter
ID: 20367594
We are running Solaris SunOS 5.9 with all the patches.

It doesn't work on this system, as the man pages would suggest. Also, in egrep, I didn't see a switch similar to the -F that would treat the file entries as strings rather than expressions.

Thanks for trying. I'll have to see if there is another way to do it as no matter how I try it, I return the whole file.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20367621
Hi,

The -F option is not needed (I did not use it). Try without it.

What is your shell? I am using ksh.

0
 

Author Comment

by:jpetter
ID: 20368048
I was using the bash shell, but tried ksh and had the same results.

Thanks,
Jeff
0
 

Author Comment

by:jpetter
ID: 20373443
OK, I found out what the problem was. The "tmp.txt" file that I was using had a blank line at the end. As soon as I removed that line, it worked as expected.

Thanks for your help,
Jeff
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

My previous tech tip, Installing the Solaris OS From the Flash Archive On a Tape (http://www.experts-exchange.com/articles/OS/Unix/Solaris/Installing-the-Solaris-OS-From-the-Flash-Archive-on-a-Tape.html), discussed installing the Solaris Operating S…
Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.
Suggested Courses
Course of the Month14 days, 21 hours left to enroll

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question