Solved

Need Multiple Patterns In Grep

Posted on 2014-11-11
9
298 Views
Last Modified: 2014-11-11
I have the need to grep a log file on a continuous basis.  I have it broken down to 3 steps I can get two of them to work but I am unclear how to add the 3rd grep.

This works but it doesn't cull out the .jpg and such from the list.
egrep -w "Nov" /var/log/access.log | grep -P 'HTTP/1.1\" 404' /var/log/access.log


This is what I have tried that doesn't work.

egrep -w "Nov" /var/log/access.log | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
 | grep -P 'HTTP/1.1\" 404' /data/log/access_thp.log-20141109

Obviously, I want a way to add in the following portion to a grep that works.
grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"

Thanks,
0
Comment
Question by:sharingsunshine
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40435460
The part that doesn't work is containing a second filename: /data/log/access_thp.log-20141109

Do you want to grep in that file as well?
0
 
LVL 84

Expert Comment

by:ozo
ID: 40435466
Can you give an example of a line that was returned by your grep attempts that you did not want, or a line that was not returned that you did want?
0
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40435469
You'd normally pipe multiple grep's after each other like this:

egrep -w "Nov" /var/log/access.log | grep "something" | grep "something again" | grep ...
0
Manage your data center from practically anywhere

The KN8164V features HD resolution of 1920 x 1200, FIPS 140-2 with level 1 security standards and virtual media transmissions at twice the speed. Built for reliability, the KN series provides local console and remote over IP access, ensuring 24/7 availability to all servers.

 

Author Comment

by:sharingsunshine
ID: 40435803
When I pipe them together it stops the 404 grep.  Consequently, I get all the file types not just the 404's.  Here is what I tried combining them as you suggested:

egrep -w "Nov" /data/log/access_thp.log-20141109 | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
 | grep -P 'HTTP/1.1\" 404

That file name was a misprint on my part it should be /var/log/access.log  instead.

here is an example what i want to remove from
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281
0
 
LVL 84

Expert Comment

by:ozo
ID: 40435846
If you want to remove
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281
then shouldn't it be
grep -v 'HTTP/1.1\" 404'
0
 

Author Comment

by:sharingsunshine
ID: 40435931
sorry I didn't reference back to the original question.  I want the 404's I don't want the specific file types.  You asked for an example of what I didn't want.  Which is this line because it is referencing the .jpg file type.
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281

Even though this is a 404 I don't want any of these file types
grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
0
 
LVL 62

Expert Comment

by:gheist
ID: 40436026
grep -e cat -e dog -e -i november -e -v 2014
0
 
LVL 38

Accepted Solution

by:
Gerwin Jansen, EE MVE earned 500 total points
ID: 40436048
How about this:
grep "Nov" /var/log/access.log | grep -P 'HTTP/1.1\" 404' | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"

Open in new window


(in text: You want all 404's from November but not the favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg lines)
0
 

Author Closing Comment

by:sharingsunshine
ID: 40436232
Great job, thanks for the help and the line of code.
0

Featured Post

[Webinar] How Hackers Steal Your Credentials

Do You Know How Hackers Steal Your Credentials? Join us and Skyport Systems to learn how hackers steal your credentials and why Active Directory must be secure to stop them. Thursday, July 13, 2017 10:00 A.M. PDT

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Linux users are sometimes dumbfounded by the severe lack of documentation on a topic. Sometimes, the documentation is copious, but other times, you end up with some obscure "it varies depending on your distribution" over and over when searching for …
Google Drive is extremely cheap offsite storage, and it's even possible to get extra storage for free for two years.  You can use the free account 15GB, and if you have an Android device..when you install Google Drive for the first time it will give…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses

631 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question