Solved

find, grep, awk or perl ... to find the next value to a string

Posted on 2015-01-28
6
91 Views
Last Modified: 2016-04-08
Linux/Unix Gurus,

Need some help, I need to extract this value (HOST=11.22.33.44) from 75909 files, and then have to sort them uniquely too, please assist, how I can take out this information from all these log files, basically, just need to extract the host ip addresses of all the machines mentioned in these log files, and then obviously have to sort them uniquely too.

Thanks in advance.
0
Comment
Question by:mkhandba
  • 3
  • 2
6 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 300 total points
Comment Utility
This is for one single file "inputfile":

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sort -u

"inputfile" can be wildcarded, e. g. "*.log" (without quotation marks).

For all files in a directory "./mydir":

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sort -u

The above leaves the part "HOST=" in place. To strip it off use

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sed  's/HOST=//' | sort -u

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sed  's/HOST=//' | sort -u
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 200 total points
Comment Utility
perl -lne 'END{print for sort keys %h}/HOST=(\S+)/&&++$h{$1}' 75909 files
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 300 total points
Comment Utility
Note: My solutions take into account that the IP address might be followed by something other than a space or a newline, e. g. a comma, a semicolon, a colon, a pipe etc., and that there might be more than one occurrence of "HOST=..." per line.
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 

Author Comment

by:mkhandba
Comment Utility
thanks ... I need to try them all and will update you, which one works for me, thanks again for your assistance.
0
 

Author Comment

by:mkhandba
Comment Utility
sorry need some more time to test/check/verify ... thanks for the patience
0
 

Author Comment

by:mkhandba
Comment Utility
still couldn't ... sorry for the delay, but please bear with me ... a little more :(
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

If you have a server on collocation with the super-fast CPU, that doesn't mean that you get it running at full power. Here is a preamble. When doing inventory of Linux servers, that I'm administering, I've found that some of them are running on l…
Little introduction about CP: CP is a command on linux that use to copy files and folder from one location to another location. Example usage of CP as follow: cp /myfoder /pathto/destination/folder/ cp abc.tar.gz /pathto/destination/folder/ab…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now