Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Need Multiple Patterns In Grep

Posted on 2014-11-11
9
Medium Priority
?
310 Views
Last Modified: 2014-11-11
I have the need to grep a log file on a continuous basis.  I have it broken down to 3 steps I can get two of them to work but I am unclear how to add the 3rd grep.

This works but it doesn't cull out the .jpg and such from the list.
egrep -w "Nov" /var/log/access.log | grep -P 'HTTP/1.1\" 404' /var/log/access.log


This is what I have tried that doesn't work.

egrep -w "Nov" /var/log/access.log | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
 | grep -P 'HTTP/1.1\" 404' /data/log/access_thp.log-20141109

Obviously, I want a way to add in the following portion to a grep that works.
grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"

Thanks,
0
Comment
Question by:sharingsunshine
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40435460
The part that doesn't work is containing a second filename: /data/log/access_thp.log-20141109

Do you want to grep in that file as well?
0
 
LVL 84

Expert Comment

by:ozo
ID: 40435466
Can you give an example of a line that was returned by your grep attempts that you did not want, or a line that was not returned that you did want?
0
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40435469
You'd normally pipe multiple grep's after each other like this:

egrep -w "Nov" /var/log/access.log | grep "something" | grep "something again" | grep ...
0
Not sure which OpenStack Certification to get?

So you’ve realized you might want to get certified in OpenStack, but you’re not sure what the benefits might be or even which one you should take. You know there are several certification courses you can choose from, but how do you know which one is right for you?

 

Author Comment

by:sharingsunshine
ID: 40435803
When I pipe them together it stops the 404 grep.  Consequently, I get all the file types not just the 404's.  Here is what I tried combining them as you suggested:

egrep -w "Nov" /data/log/access_thp.log-20141109 | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
 | grep -P 'HTTP/1.1\" 404

That file name was a misprint on my part it should be /var/log/access.log  instead.

here is an example what i want to remove from
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281
0
 
LVL 84

Expert Comment

by:ozo
ID: 40435846
If you want to remove
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281
then shouldn't it be
grep -v 'HTTP/1.1\" 404'
0
 

Author Comment

by:sharingsunshine
ID: 40435931
sorry I didn't reference back to the original question.  I want the 404's I don't want the specific file types.  You asked for an example of what I didn't want.  Which is this line because it is referencing the .jpg file type.
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281

Even though this is a 404 I don't want any of these file types
grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
0
 
LVL 62

Expert Comment

by:gheist
ID: 40436026
grep -e cat -e dog -e -i november -e -v 2014
0
 
LVL 38

Accepted Solution

by:
Gerwin Jansen, EE MVE earned 2000 total points
ID: 40436048
How about this:
grep "Nov" /var/log/access.log | grep -P 'HTTP/1.1\" 404' | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"

Open in new window


(in text: You want all 404's from November but not the favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg lines)
0
 

Author Closing Comment

by:sharingsunshine
ID: 40436232
Great job, thanks for the help and the line of code.
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Ready to improve network connectivity? Watch this webinar to learn how SD-WANs and a one-click instant connect tool can boost provisions, deployment, and management of your cloud connection.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Linux users are sometimes dumbfounded by the severe lack of documentation on a topic. Sometimes, the documentation is copious, but other times, you end up with some obscure "it varies depending on your distribution" over and over when searching for …
It is possible to boost certain documents at query time in Solr. Query time boosting can be a powerful resource for finding the most relevant and "best" content. Of course the more information you index, the more fields you will be able to use for y…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
Suggested Courses

670 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question