Solved

Need Multiple Patterns In Grep

Posted on 2014-11-11
9
285 Views
Last Modified: 2014-11-11
I have the need to grep a log file on a continuous basis.  I have it broken down to 3 steps I can get two of them to work but I am unclear how to add the 3rd grep.

This works but it doesn't cull out the .jpg and such from the list.
egrep -w "Nov" /var/log/access.log | grep -P 'HTTP/1.1\" 404' /var/log/access.log


This is what I have tried that doesn't work.

egrep -w "Nov" /var/log/access.log | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
 | grep -P 'HTTP/1.1\" 404' /data/log/access_thp.log-20141109

Obviously, I want a way to add in the following portion to a grep that works.
grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"

Thanks,
0
Comment
Question by:sharingsunshine
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40435460
The part that doesn't work is containing a second filename: /data/log/access_thp.log-20141109

Do you want to grep in that file as well?
0
 
LVL 84

Expert Comment

by:ozo
ID: 40435466
Can you give an example of a line that was returned by your grep attempts that you did not want, or a line that was not returned that you did want?
0
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 40435469
You'd normally pipe multiple grep's after each other like this:

egrep -w "Nov" /var/log/access.log | grep "something" | grep "something again" | grep ...
0
Create the perfect environment for any meeting

You might have a modern environment with all sorts of high-tech equipment, but what makes it worthwhile is how you seamlessly bring together the presentation with audio, video and lighting. The ATEN Control System provides integrated control and system automation.

 

Author Comment

by:sharingsunshine
ID: 40435803
When I pipe them together it stops the 404 grep.  Consequently, I get all the file types not just the 404's.  Here is what I tried combining them as you suggested:

egrep -w "Nov" /data/log/access_thp.log-20141109 | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
 | grep -P 'HTTP/1.1\" 404

That file name was a misprint on my part it should be /var/log/access.log  instead.

here is an example what i want to remove from
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281
0
 
LVL 84

Expert Comment

by:ozo
ID: 40435846
If you want to remove
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281
then shouldn't it be
grep -v 'HTTP/1.1\" 404'
0
 

Author Comment

by:sharingsunshine
ID: 40435931
sorry I didn't reference back to the original question.  I want the 404's I don't want the specific file types.  You asked for an example of what I didn't want.  Which is this line because it is referencing the .jpg file type.
1.1.1.1 - - [11/Nov/2014:15:19:37 +0000] "GET /392-large_default/Black_Walnut_100_Capsules_p_658.jpg HTTP/1.1" 404 281

Even though this is a 404 I don't want any of these file types
grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"
0
 
LVL 62

Expert Comment

by:gheist
ID: 40436026
grep -e cat -e dog -e -i november -e -v 2014
0
 
LVL 38

Accepted Solution

by:
Gerwin Jansen, EE MVE earned 500 total points
ID: 40436048
How about this:
grep "Nov" /var/log/access.log | grep -P 'HTTP/1.1\" 404' | grep  -v -E "favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg"

Open in new window


(in text: You want all 404's from November but not the favicon.ico|robots.txt|mailalerts.js|btn_bg.gif|cgi-bin|.jpg lines)
0
 

Author Closing Comment

by:sharingsunshine
ID: 40436232
Great job, thanks for the help and the line of code.
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Samba 4, Users Permission, 5 62
plsql job on oracle 18 72
SMTP log file for IMSVA 5 26
Enable SPF on IMSVA 6 8
Hi, in this article I'm going to teach you how to run your own site, and how to let people in (without IP). I'll talk about and explain each step... :) By the way, everything in this Tutorial is completely free and legal. This article is for …
The purpose of this article is to demonstrate how we can use conditional statements using Python.
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

679 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question