Solved

Need Multiple Patterns In Grep

Posted on 2014-11-11
9
284 Views
Last Modified: 2014-11-11
I have the need to grep a log file on a continuous basis.  I have it broken down to 3 steps I can get two of them to work but I am unclear how to add the 3rd grep.

This works but it doesn't cull out the .jpg and such from the list.
egrep -w "Nov" /var/log/access.log | grep -P 'HTTP/1.1\" 404' /var/log/access.log


This is what I have tried that doesn't work.

egrep -w "Nov" /var/log/access.log | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
 | grep -P 'HTTP/1.1\" 404' /data/log/access_thp.log-20141109

Obviously, I want a way to add in the following portion to a grep that works.
grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"

Thanks,
0
Comment
Question by:sharingsunshine
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40435460
The part that doesn't work is containing a second filename: /data/log/access_thp.log-20141109

Do you want to grep in that file as well?
0
 
LVL 84

Expert Comment

by:ozo
ID: 40435466
Can you give an example of a line that was returned by your grep attempts that you did not want, or a line that was not returned that you did want?
0
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40435469
You'd normally pipe multiple grep's after each other like this:

egrep -w "Nov" /var/log/access.log | grep "something" | grep "something again" | grep ...
0
Manage your data center from practically anywhere

The KN8164V features HD resolution of 1920 x 1200, FIPS 140-2 with level 1 security standards and virtual media transmissions at twice the speed. Built for reliability, the KN series provides local console and remote over IP access, ensuring 24/7 availability to all servers.

 

Author Comment

by:sharingsunshine
ID: 40435803
When I pipe them together it stops the 404 grep.  Consequently, I get all the file types not just the 404's.  Here is what I tried combining them as you suggested:

egrep -w "Nov" /data/log/access_thp.log-20141109 | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
 | grep -P 'HTTP/1.1\" 404

That file name was a misprint on my part it should be /var/log/access.log  instead.

here is an example what i want to remove from
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281
0
 
LVL 84

Expert Comment

by:ozo
ID: 40435846
If you want to remove
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281
then shouldn't it be
grep -v 'HTTP/1.1\" 404'
0
 

Author Comment

by:sharingsunshine
ID: 40435931
sorry I didn't reference back to the original question.  I want the 404's I don't want the specific file types.  You asked for an example of what I didn't want.  Which is this line because it is referencing the .jpg file type.
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281

Even though this is a 404 I don't want any of these file types
grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
0
 
LVL 62

Expert Comment

by:gheist
ID: 40436026
grep -e cat -e dog -e -i november -e -v 2014
0
 
LVL 38

Accepted Solution

by:
Gerwin Jansen, EE MVE earned 500 total points
ID: 40436048
How about this:
grep "Nov" /var/log/access.log | grep -P 'HTTP/1.1\" 404' | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"

Open in new window


(in text: You want all 404's from November but not the favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg lines)
0
 

Author Closing Comment

by:sharingsunshine
ID: 40436232
Great job, thanks for the help and the line of code.
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Fine Tune your automatic Updates for Ubuntu / Debian
Introduction This article is intended for those who are new to PHP error handling (https://www.experts-exchange.com/articles/11769/And-by-the-way-I-am-New-to-PHP.html).  It addresses one of the most common problems that plague beginning PHP develop…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question