Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

find, grep, awk or perl ... to find the next value to a string

Posted on 2015-01-28
6
Medium Priority
?
108 Views
Last Modified: 2016-04-08
Linux/Unix Gurus,

Need some help, I need to extract this value (HOST=11.22.33.44) from 75909 files, and then have to sort them uniquely too, please assist, how I can take out this information from all these log files, basically, just need to extract the host ip addresses of all the machines mentioned in these log files, and then obviously have to sort them uniquely too.

Thanks in advance.
0
Comment
Question by:Mushfique Khan
  • 3
  • 2
6 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 1200 total points
ID: 40575712
This is for one single file "inputfile":

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sort -u

"inputfile" can be wildcarded, e. g. "*.log" (without quotation marks).

For all files in a directory "./mydir":

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sort -u

The above leaves the part "HOST=" in place. To strip it off use

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sed  's/HOST=//' | sort -u

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sed  's/HOST=//' | sort -u
0
 
LVL 85

Assisted Solution

by:ozo
ozo earned 800 total points
ID: 40575999
perl -lne 'END{print for sort keys %h}/HOST=(\S+)/&&++$h{$1}' 75909 files
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 1200 total points
ID: 40576096
Note: My solutions take into account that the IP address might be followed by something other than a space or a newline, e. g. a comma, a semicolon, a colon, a pipe etc., and that there might be more than one occurrence of "HOST=..." per line.
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 

Author Comment

by:Mushfique Khan
ID: 40582466
thanks ... I need to try them all and will update you, which one works for me, thanks again for your assistance.
0
 

Author Comment

by:Mushfique Khan
ID: 40719296
sorry need some more time to test/check/verify ... thanks for the patience
0
 

Author Comment

by:Mushfique Khan
ID: 40916784
still couldn't ... sorry for the delay, but please bear with me ... a little more :(
0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Little introduction about CP: CP is a command on linux that use to copy files and folder from one location to another location. Example usage of CP as follow: cp /myfoder /pathto/destination/folder/ cp abc.tar.gz /pathto/destination/folder/ab…
FreeBSD on EC2 FreeBSD (https://www.freebsd.org) is a robust Unix-like operating system that has been around for many years. FreeBSD is available on Amazon EC2 through Amazon Machine Images (AMIs) provided by FreeBSD developer and security office…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Suggested Courses
Course of the Month10 days, 18 hours left to enroll

885 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question