Solved

find, grep, awk or perl ... to find the next value to a string

Posted on 2015-01-28
6
97 Views
Last Modified: 2016-04-08
Linux/Unix Gurus,

Need some help, I need to extract this value (HOST=11.22.33.44) from 75909 files, and then have to sort them uniquely too, please assist, how I can take out this information from all these log files, basically, just need to extract the host ip addresses of all the machines mentioned in these log files, and then obviously have to sort them uniquely too.

Thanks in advance.
0
Comment
Question by:mkhandba
  • 3
  • 2
6 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 300 total points
ID: 40575712
This is for one single file "inputfile":

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sort -u

"inputfile" can be wildcarded, e. g. "*.log" (without quotation marks).

For all files in a directory "./mydir":

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sort -u

The above leaves the part "HOST=" in place. To strip it off use

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sed  's/HOST=//' | sort -u

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sed  's/HOST=//' | sort -u
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 200 total points
ID: 40575999
perl -lne 'END{print for sort keys %h}/HOST=(\S+)/&&++$h{$1}' 75909 files
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 300 total points
ID: 40576096
Note: My solutions take into account that the IP address might be followed by something other than a space or a newline, e. g. a comma, a semicolon, a colon, a pipe etc., and that there might be more than one occurrence of "HOST=..." per line.
0
Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

 

Author Comment

by:mkhandba
ID: 40582466
thanks ... I need to try them all and will update you, which one works for me, thanks again for your assistance.
0
 

Author Comment

by:mkhandba
ID: 40719296
sorry need some more time to test/check/verify ... thanks for the patience
0
 

Author Comment

by:mkhandba
ID: 40916784
still couldn't ... sorry for the delay, but please bear with me ... a little more :(
0

Featured Post

What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Using libpcap/Jpcap to capture and send packets on Solaris version (10/11) Library used: 1.      Libpcap (http://www.tcpdump.org) Version 1.2 2.      Jpcap(http://netresearch.ics.uci.edu/kfujii/Jpcap/doc/index.html) Version 0.6 Prerequisite: 1.      GCC …
Introduction We as admins face situation where we need to redirect websites to another. This may be required as a part of an upgrade keeping the old URL but website should be served from new URL. This document would brief you on different ways ca…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

806 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question