Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

extract ip addresses from any file.

Posted on 2008-10-23
8
2,606 Views
Last Modified: 2012-05-05
I need to extract ip addresses from an unformatted text file.  The addresses do not have brackets or any other delimiters around them.  I need to output this to another file.
0
Comment
Question by:jeffsmall
8 Comments
 
LVL 1

Expert Comment

by:LosBear
ID: 22787424
is the file tab delimited at least? can you provide a sample of the file you want to parse?

0
 
LVL 1

Expert Comment

by:WANM
ID: 22787698
assuming they at least have a space at each side of the address, and one address somewhere on each line:

cat file | sed -r 's/(.*?) ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) (.*)/\2/'

If you could provide and example of the file it would make this easier....
0
 

Author Comment

by:jeffsmall
ID: 22788244
Sorry, I want this to work on just about any text file and there may or may not be spaces around the address.  The addresses might be interspersed and there could be more than one on a line.

Here is some sample text

; generated by /sbin/dhclient-script
nameserver 192.168.1.147
nameserver 192.168.1.1
ip address=192.168.1.148
0
How our DevOps Teams Maximize Uptime

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us. Read the use case whitepaper.

 

Author Comment

by:jeffsmall
ID: 22788450
search aus.us.siteprotect.com
nameserver 216.139.253.2
nameserver 216.139.253.3

the regex provided above matches the first line also.  I just need to pull the ip addresses and output them to a temp file.

>cat /etc/resolv.conf | sed -r 's/(.*?) ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) (.*)/\2/'
search aus.us.siteprotect.com
nameserver 216.139.253.2
nameserver 216.139.253.3
0
 
LVL 5

Accepted Solution

by:
zmo earned 125 total points
ID: 22788998
well to remove those lines you can do :

% cat /etc/resolv.conf | grep -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'
nameserver 216.139.253.2
nameserver 216.139.253.3

but if you join both you'll still get :
% cat /etc/resolv.conf | grep -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' | sed -r 's/(.*?) ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) (.*)/\2/'
nameserver 216.139.253.2
nameserver 216.139.253.3

and that's because of the needed ' ' between the regexp and the rest of the line.

A fix would be :
% cat /etc/resolv.conf | sed -r 's/^.* ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*$/\1/' | grep -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'
216.139.253.2
216.139.253.3

but you just can't match several IPs on the same line, because of how sed works internally (that's fun I said the same thing this morning on another topic about regexps). Sed works with finite stack automatas, which means it can't produce n times the same pattern in a s///. You have to do it programatically.

A solution would be to get all the first IPs of every lines containing IPs. Then use sed to remove all those IPs and pipe it to sed again so it gets all the first IPs of every lines, and so on until there are no more IPs.
0
 
LVL 5

Expert Comment

by:zmo
ID: 22789028
% cat /etc/resolv.conf | sed -r 's/^[^0-9]*([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*$/\1/' | grep -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'
216.139.253.2
216.139.253.3

is what you want (ie no need for leading white space)
0
 

Author Closing Comment

by:jeffsmall
ID: 31509269
Thank you!  That does what I need very nicely. I appreciate your help and the knowledge I have gained from your work.
0
 
LVL 5

Expert Comment

by:zmo
ID: 22789190
to be more picky, and correct all previous mistakes, you could use :

% cat /etc/resolv.conf | sed -r 's/^.*?[^0-9](25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)[^0-9].*$/\1.\2.\3.\4/' | grep -E "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)"
216.139.253.2
216.139.253.3

the differences are :
1/ '' ^.*? '' : matches any character any time at the beginning of the line
2/ '' [^0-9] '' : match any character that is not a number one time
3/ '' (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?) '' : matches an IP address that is valid (ie 265.24.256.3 is not valid)
4/ '' [^0-9] '' same as 2
5/ '' .*?$ '' : same as 1/ for the end

\1.\2.\3.\4 reconstructs the address from the parts in parenthesis in pattern 3/
0

Featured Post

Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Example unit tests with AngularJS 3 47
Need some help with powershell script 5 55
change the windows script file to BAT 10 49
Awk : two files math 8 20
Recently I have been answering a lot of questions like this in IT forums that I frequent. The question posed is usually something along the lines of "We have software X installed and need to uninstall it for reason Y" or some other variant of the sa…
In this tutorial I will show you how to make a simple HTML bar chart with the usage of WhizBase, If you want more information about WhizBase please read my previous articles at http://www.experts-exchange.com/ARTH_5123186.html (http://www.experts-ex…
Learn the basics of while and for loops in Python.  while loops are used for testing while, or until, a condition is met: The structure of a while loop is as follows:     while <condition>:         do something         repeate: The break statement m…
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question