Python: Parsing TXT for IP Address

Hello All,

I am using the below code to download a text file to a directory specified, then open the text file and search for all ip addresses contained in that text file. Below you will find my current code and current output. The issue is that I am not certain how to accomplish my final three and remaining tasks with this code and need some guidance. I have made it this far on my own and I do not want anyone to do it for me just point me in the right direction.

Questions:

1. How can I modify the code below in a way to find all addresses ending in .255 and remove them? This would leave only addresses with .0 remaining. I suppose I could modify the regex pattern to find all ip addresses with .0 at the end. Not sure of the regex pattern to accomplish that.
2. I would like to append a /24 to each of the ip addresses below.
3. Formatting; Each IP address should have its own row such as:
    1.1.1.0/24
    2.2.2.0/24
    3.3.3.0/24

''' MY CURRENT CODE '''

from urllib.request import urlretrieve
import re

files_www = 'http://feeds.dshield.org/block.txt'
files_dlloc = 'c:\dshield.txt'
urlretrieve(files_www, files_dlloc)

f = open(files_dlloc, 'r')
raw_text = str(f.readlines())
f.close()
ip_address = r'(?:[\d]{1,3})\.(?:[\d]{1,3})\.(?:[\d]{1,3})\.(?:[\d]{1,3})'
foundip = re.findall( ip_address, raw_text )

print (foundip)

Open in new window


''' MY CURRENT OUTPUT '''

C:\>dshield.py
['184.105.139.0', '184.105.139.255', '61.221.83.0', '61.221.83.255', '212.83.148
.0', '212.83.148.255', '123.8.126.0', '123.8.126.255', '122.225.109.0', '122.225
.109.255', '218.77.79.0', '218.77.79.255', '74.82.47.0', '74.82.47.255', '162.21
2.181.0', '162.212.181.255', '176.31.101.0', '176.31.101.255', '192.3.205.0', '1
92.3.205.255', '60.173.11.0', '60.173.11.255', '104.194.25.0', '104.194.25.255',
 '222.43.119.0', '222.43.119.255', '115.28.239.0', '115.28.239.255', '197.243.16
.0', '197.243.16.255', '190.143.107.0', '190.143.107.255', '184.105.247.0', '184
.105.247.255', '93.178.3.0', '93.178.3.255', '61.174.51.0', '61.174.51.255', '71
.6.216.0', '71.6.216.255']

Open in new window

BERITMAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Terry WoodsIT GuruCommented:
Not sure about the code, but I should be able to help with the regular expression. The following expressions will match IP addresses ending with 255 and 0 respectively:
r'(?:[\d]{1,3})\.(?:[\d]{1,3})\.(?:[\d]{1,3})\.255'

Open in new window

r'(?:[\d]{1,3})\.(?:[\d]{1,3})\.(?:[\d]{1,3})\.0'

Open in new window

0
BERITMAuthor Commented:
I updated the regex:

ip_address = r'(?:[\d]{1,3})\.(?:[\d]{1,3})\.(?:[\d]{1,3})\.0'

Open in new window


Here is my new output:

C:\>dshield.py
['184.105.139.0', '122.225.97.0', '222.186.34.0', '74.82.47.0', '176.31.101.0',
'162.212.181.0', '218.77.79.0', '184.105.247.0', '173.201.39.0', '71.6.216.0', '
192.3.205.0', '176.31.251.0', '5.196.77.0', '92.222.181.0', '93.174.93.0', '124.
232.142.0', '62.210.90.0', '85.25.242.0', '89.248.162.0', '204.42.253.0']

Open in new window


When I change my print to:

print (foundip) + "/24"

Open in new window


I get the following error:

C:\>dshield.py
['184.105.139.0', '122.225.97.0', '222.186.34.0', '74.82.47.0', '176.31.101.0',
'162.212.181.0', '218.77.79.0', '184.105.247.0', '173.201.39.0', '71.6.216.0', '
192.3.205.0', '176.31.251.0', '5.196.77.0', '92.222.181.0', '93.174.93.0', '124.
232.142.0', '62.210.90.0', '85.25.242.0', '89.248.162.0', '204.42.253.0']
Traceback (most recent call last):
  File "C:\dshield.py", line 19, in <module>
    print (foundip) + "/24"
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

Open in new window

0
Terry WoodsIT GuruCommented:
To add the /24 on the end, perhaps something like this?
ip_address = r'(?:[\d]{1,3})\.(?:[\d]{1,3})\.(?:[\d]{1,3})\.0'
foundip = re.findall( ip_address, raw_text )
for ip in foundip:
  print ip + "/24"

Open in new window

0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Terry WoodsIT GuruCommented:
Not sure if that's the correct way to concat strings in python, sorry. If my latest suggestion doesn't work, then perhaps try a comma instead of a plus:
  print ip, "/24"

Edit: looks like + is the correct one to use, so ignore this...
0
BERITMAuthor Commented:
Okay, So my problem is this. The output is a single list of ip addresses: so if ip append the string "/24" to the end it looks like below. I think the next phase is to learn how to break each of these ip addresses out of the list. Once that is done then I can print any string on the end or in from of them i want. Any idea how to break each of these addresses out of the list?

['117.27.158.0', '62.210.83.0', '195.154.252.0', '61.174.51.0', '218.77.79.0',
162.212.181.0', '176.31.101.0', '192.126.120.0', '219.129.237.0', '60.173.10.0
 '222.186.51.0', '104.194.15.0', '171.111.153.0', '1.182.119.0', '197.243.16.0
 '80.82.64.0', '78.93.247.0', '222.186.15.0', '115.238.247.0', '61.160.247.0']/24
0
BERITMAuthor Commented:
Okay, so I just changed my print line to this:

print (foundip[0])
print (foundip[1])
print (foundip[2])

and my output is

C:\>C:\dshield.py
117.27.158.0
62.210.83.0
195.154.252.0

I know for sure now that the list is good to go so I need to learn to print the items in an arrary or something.
0
BERITMAuthor Commented:
Got IT! I needed to iterate the [list]

Here is my final code for this matter

''' -- Imports -- '''
import urllib.request
import re

''' --- URL To Read -- '''
dshield_url = "http://feeds.dshield.org/block.txt"

''' -- Open URL, Read IT, String IT, Find IP, & Sort -- '''
dshield_raw_text = urllib.request.urlopen(dshield_url)
dshield_string = str(dshield_raw_text.read())
ipv4_pattern = r'\d{1,3}\.\d{1,3}\.\d{1,3}\.0'
iplist = re.findall( ipv4_pattern, dshield_string )
iplist.sort()

''' -- Create Final RSC, Populate Dangerous Class C -- '''
dshield_rsc = "C:\DShield.rsc"
dshield_clean_text = open(dshield_rsc, "w")
dshield_clean_text.write("/ip firewall address-list\n")
for ip in iplist:
    dshield_clean_text.write("add list=DShield address=" + (ip) + "/24 comment=DShield\n")
dshield_clean_text.close()

Open in new window


Here is the output in an RSC file:

/ip firewall address-list
add list=DShield address=111.73.46.0/24 comment=DShield
add list=DShield address=114.43.16.0/24 comment=DShield
add list=DShield address=117.21.173.0/24 comment=DShield
add list=DShield address=119.5.155.0/24 comment=DShield
add list=DShield address=122.141.234.0/24 comment=DShield
add list=DShield address=122.225.109.0/24 comment=DShield
add list=DShield address=122.225.97.0/24 comment=DShield
add list=DShield address=158.255.1.0/24 comment=DShield
add list=DShield address=162.212.181.0/24 comment=DShield
add list=DShield address=176.31.101.0/24 comment=DShield
add list=DShield address=194.73.113.0/24 comment=DShield
add list=DShield address=210.14.152.0/24 comment=DShield
add list=DShield address=218.77.79.0/24 comment=DShield
add list=DShield address=222.186.21.0/24 comment=DShield
add list=DShield address=222.186.56.0/24 comment=DShield
add list=DShield address=222.187.32.0/24 comment=DShield
add list=DShield address=60.173.11.0/24 comment=DShield
add list=DShield address=61.174.51.0/24 comment=DShield
add list=DShield address=74.118.193.0/24 comment=DShield
add list=DShield address=93.174.93.0/24 comment=DShield

Open in new window


Thank You,
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Terry WoodsIT GuruCommented:
Sorry I didn't get back to this earlier; glad you got it solved! Could you please accept an answer to the question?

You can accept your own solution as the answer, but you can assign some points to my comments if they were helpful...
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Python

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.