Solved

find, grep, awk or perl ... to find the next value to a string

Posted on 2015-01-28
6
101 Views
Last Modified: 2016-04-08
Linux/Unix Gurus,

Need some help, I need to extract this value (HOST=11.22.33.44) from 75909 files, and then have to sort them uniquely too, please assist, how I can take out this information from all these log files, basically, just need to extract the host ip addresses of all the machines mentioned in these log files, and then obviously have to sort them uniquely too.

Thanks in advance.
0
Comment
Question by:mkhandba
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 300 total points
ID: 40575712
This is for one single file "inputfile":

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sort -u

"inputfile" can be wildcarded, e. g. "*.log" (without quotation marks).

For all files in a directory "./mydir":

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sort -u

The above leaves the part "HOST=" in place. To strip it off use

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sed  's/HOST=//' | sort -u

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sed  's/HOST=//' | sort -u
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 200 total points
ID: 40575999
perl -lne 'END{print for sort keys %h}/HOST=(\S+)/&&++$h{$1}' 75909 files
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 300 total points
ID: 40576096
Note: My solutions take into account that the IP address might be followed by something other than a space or a newline, e. g. a comma, a semicolon, a colon, a pipe etc., and that there might be more than one occurrence of "HOST=..." per line.
0
Back Up Your Microsoft Windows Server®

Back up all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

 

Author Comment

by:mkhandba
ID: 40582466
thanks ... I need to try them all and will update you, which one works for me, thanks again for your assistance.
0
 

Author Comment

by:mkhandba
ID: 40719296
sorry need some more time to test/check/verify ... thanks for the patience
0
 

Author Comment

by:mkhandba
ID: 40916784
still couldn't ... sorry for the delay, but please bear with me ... a little more :(
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Linux users are sometimes dumbfounded by the severe lack of documentation on a topic. Sometimes, the documentation is copious, but other times, you end up with some obscure "it varies depending on your distribution" over and over when searching for …
In the first part of this tutorial we will cover the prerequisites for installing SQL Server vNext on Linux.
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Suggested Courses

734 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question