Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

find, grep, awk or perl ... to find the next value to a string

Posted on 2015-01-28
6
Medium Priority
?
107 Views
Last Modified: 2016-04-08
Linux/Unix Gurus,

Need some help, I need to extract this value (HOST=11.22.33.44) from 75909 files, and then have to sort them uniquely too, please assist, how I can take out this information from all these log files, basically, just need to extract the host ip addresses of all the machines mentioned in these log files, and then obviously have to sort them uniquely too.

Thanks in advance.
0
Comment
Question by:Mushfique Khan
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 2
6 Comments
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 1200 total points
ID: 40575712
This is for one single file "inputfile":

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sort -u

"inputfile" can be wildcarded, e. g. "*.log" (without quotation marks).

For all files in a directory "./mydir":

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sort -u

The above leaves the part "HOST=" in place. To strip it off use

grep -o -E -h 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' inputfile | sed  's/HOST=//' | sort -u

grep -o -E -h -R 'HOST=[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' ./mydir | sed  's/HOST=//' | sort -u
0
 
LVL 84

Assisted Solution

by:ozo
ozo earned 800 total points
ID: 40575999
perl -lne 'END{print for sort keys %h}/HOST=(\S+)/&&++$h{$1}' 75909 files
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 1200 total points
ID: 40576096
Note: My solutions take into account that the IP address might be followed by something other than a space or a newline, e. g. a comma, a semicolon, a colon, a pipe etc., and that there might be more than one occurrence of "HOST=..." per line.
0
The Orion Papers

Are you interested in becoming an AWS Certified Solutions Architect?

Discover a new interactive way of training for the exam.

 

Author Comment

by:Mushfique Khan
ID: 40582466
thanks ... I need to try them all and will update you, which one works for me, thanks again for your assistance.
0
 

Author Comment

by:Mushfique Khan
ID: 40719296
sorry need some more time to test/check/verify ... thanks for the patience
0
 

Author Comment

by:Mushfique Khan
ID: 40916784
still couldn't ... sorry for the delay, but please bear with me ... a little more :(
0

Featured Post

Moving data to the cloud? Find out if you’re ready

Before moving to the cloud, it is important to carefully define your db needs, plan for the migration & understand prod. environment. This wp explains how to define what you need from a cloud provider, plan for the migration & what putting a cloud solution into practice entails.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded…
Fine Tune your automatic Updates for Ubuntu / Debian
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Suggested Courses

670 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question