Solved

How To grep for a list of options in Solaris

Posted on 2007-11-28
9
324 Views
Last Modified: 2013-12-27
Hi,
I am trying to figure out how to grep through log files, but rather than grepping for one item, I want to search for a number of items. I have tried a number of different approaches, but none have worked.

What I have is a list of domain names in a text file with one name per line. I then want to search through a number of log files looking for any log entry that matches any of the domains in the list. I also have a similar list, though it contains IP addresses instead of names that I also need to search for.

Does anyone know of a good way to search for a number of items in one command, or do I have to grep for each one - which would be extremely time consuming?

I would greatly appreciate any help on this.

Thanks,
Jeff
0
Comment
Question by:jpetter
  • 5
  • 4
9 Comments
 
LVL 40

Accepted Solution

by:
omarfarid earned 500 total points
ID: 20366742
Hi,

grep can take the patterns / strings from a file. So, you may put your search strings in a file and then

grep -f stringsfile file(s)

stringsfile is the file that has the strings or patterns

file(s) file(s) that contain the logs

you may also use the -F option with the command

e.g.

grep -F -f stringsfile logfile

http://unixhelp.ed.ac.uk/CGI/man-cgi?grep
0
 

Author Comment

by:jpetter
ID: 20366841
Hi,
Thanks for responding so quickly. That is very similar to what I have been trying - almost identical. Here is one of my attempts:
bash-2.05$ gzcat logfilename.log.gz | /usr/xpg4/bin/grep -F -f test.txt, but rather than returning only those entries that match the file entries, it returns the entire file.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20366878
OK,

can you provide sample of those files (you do not have to give real info, and can replace by dummy values)?

I will try to see if I can do something. It could be the (.) in the IPs or domains
0
 

Author Comment

by:jpetter
ID: 20367229
Thanks. I have changed the IP's and names, but this should give you an idea.

These seven lines you'll need to save to a file and then gzip: (they look like alot more than seven lines with all the wraps)
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_NC_MISS GET http://catalog.video.syndication.msn.com/videoservice/videoByTag.ashx?tag=ENAPus_ENAPus&ns=MSNVideo_Top_Cat&mk=en-ap&sd=-1&sf=ActiveStartDate&ps=5&rand=2080 HTTP/1.0 200 11797 http://img.video.ap.org/p/s/p2/lg_hz_search.swf Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.0.3705;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
192.176.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://www.priceline.com/zp/utils/transport.js HTTP/1.0 200 71520 http://www.priceline.com/hotels/Lang/en-us/region_star_price.asp?session_key=5C0011AC5D0011AC20071127182935ba1e40674203&plf=pcln Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - Travel
10.1.1.20 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://samantha48616e61.com/images/article_tn.jpg HTTP/1.0 304 289 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
10.2.1.15 - - [27/Nov/2007:13:29:39 -0500] TCP_MISS GET http://samantha48616e61.com/images/article2_tn.jpg HTTP/1.0 304 286 http://samantha48616e61.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Newsgroups/Forums
192.176.1.10 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://stb.msn.com/i/48/86F1396496DFE1BAD68AB5F28409.gif HTTP/1.0 200 782 http://www.msnbc.msn.com/id/3032113/?ta=y Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.0;%20.NET%20CLR%201.1.4322) - Search_Engines/Portals
10.2.2.2 - - [27/Nov/2007:13:29:39 -0500] TCP_CLIENT_REFRESH GET http://money.msn.com/MSNQuoteData.xml HTTP/1.0 200 414 - Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322;%20%20MSN%209.0;MSN%209.1;%20MSNbVZ02;%20MSNmen-us;%20MSNcOTH) - Search_Engines/Portals
192.176.5.5 - - [27/Nov/2007:13:29:39 -0500] TCP_HIT GET http://image.weather.com/web/multimedia/images/miscellaneous/ec206_open.jpg HTTP/1.0 200 1826 http://www.weather.com/ Mozilla/4.0%20(compatible;%20MSIE%206.0;%20Windows%20NT%205.1;%20SV1;%20.NET%20CLR%201.1.4322) - News/Media

Next, I created a file called tmp.txt that looked like this:
192.176.1.10
192.176.1.15

Here is the command I ran:
bash-2.05$ gzcat tmplogfile.txt.gz | /usr/xpg4/bin/grep -F -f tmp.txt

and rather than returning the three lines that match, it returns all seven.

If you could spot something, I would greatly appreciate it.

Thanks,
Jeff
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 40

Expert Comment

by:omarfarid
ID: 20367409
Hi,

Which solaris version do you have?

I tried it on my system and it works perfect !

I have solaris 8 with latest patches.

the /usr/bin/grep does not take the -f or -F , so I used egrep and /usr/xpg4/bin/grep and both gave correct results (3 lines with correct ips).

So check your system or try the egrep
0
 

Author Comment

by:jpetter
ID: 20367594
We are running Solaris SunOS 5.9 with all the patches.

It doesn't work on this system, as the man pages would suggest. Also, in egrep, I didn't see a switch similar to the -F that would treat the file entries as strings rather than expressions.

Thanks for trying. I'll have to see if there is another way to do it as no matter how I try it, I return the whole file.

Thanks,
Jeff
0
 
LVL 40

Expert Comment

by:omarfarid
ID: 20367621
Hi,

The -F option is not needed (I did not use it). Try without it.

What is your shell? I am using ksh.

0
 

Author Comment

by:jpetter
ID: 20368048
I was using the bash shell, but tried ksh and had the same results.

Thanks,
Jeff
0
 

Author Comment

by:jpetter
ID: 20373443
OK, I found out what the problem was. The "tmp.txt" file that I was using had a blank line at the end. As soon as I removed that line, it worked as expected.

Thanks for your help,
Jeff
0

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
forget admin password 8 23
Convert OverPunch 25 57
NTP Server in VMware 5 115
Write an app 10 61
Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
Introduction Regular patching is part of a system administrator's tasks. However, many patches require that the system be in single-user mode before they can be installed. A cluster patch in particular can take quite a while to apply if the machine…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now