Manipulating keys in a python dictionary
I have built a dict of timestamps and icmp delays from an icmp log file (see below) in order to evaluate the average icmp response time. My code is attached - there must be a better way to do this but I haven't figured it out.
As you can see, the script accounts for reported timestamps but not for down-time, when there is no connection at all. In order to factor in down-time, I would have to manually add up the hours reported by the time_delta function, assign each hour a time-out value (for example 2000 ms) and recalculate.
To avoid manual calculations, I would like to insert a timestamp into the time_warp dict for each hour where none is reported, assign that timestamp a standard time-out value of 2000 ms, and stop the iteration when a reported hourly timestamp is found. How to do this?
As you can see, the script accounts for reported timestamps but not for down-time, when there is no connection at all. In order to factor in down-time, I would have to manually add up the hours reported by the time_delta function, assign each hour a time-out value (for example 2000 ms) and recalculate.
To avoid manual calculations, I would like to insert a timestamp into the time_warp dict for each hour where none is reported, assign that timestamp a standard time-out value of 2000 ms, and stop the iteration when a reported hourly timestamp is found. How to do this?
from datetime import datetime, date, time, timedelta
import re
import os, sys
day = timedelta(minutes=1440)
hour = timedelta(seconds=3600)
hour_plus = timedelta(seconds=3601)
InLog = open('ping.log', 'r')
time_warp = {}
intervals = []
odds = []
evens = []
def log_dict():
for line in InLog:
line = line.strip()
if len(line) == 0:
continue
elif 'None' in line:
continue
elif 'Timestamp' in line:
time_value = line[11:]
date_object = datetime.strptime(time_value, '%a %b %d %H:%M:%S %Y')
delay_value = ' '
intervals.append(date_object)
else:
my_values = line.split(' ')
time_value = str(my_values[0])
delay_str = str(my_values[6])
delay_val = str(delay_str[0:-2])
date_object = datetime.strptime(time_value, '%Y-%m-%d-%H:%M:%S')
if re.search("[a-zA-Z_']", delay_val):
delay_value = 2000
else:
delay_value = round(float(delay_val), 2)
time_warp[date_object] = {}
time_warp[date_object] = delay_value
# print keys and values:
for key in sorted(time_warp.keys()):
delay_value = time_warp[key]
if delay_value == ' ':
print key
elif delay_value > 200:
print key, 'delayed:', delay_value
else:
print key, delay_value
# Calculate average delay:
count = 0
total = 0
for i in time_warp.values():
if i != ' ':
count = count + 1
total += i
average = float(total/count) # 158.28 ms
print 'The average icmp delay is', round(average, 2), 'ms'
InLog.close()
return time_warp
def divide_list(intervals):
for i, timestamp in enumerate(intervals):
if i % 2 == 0:
evens.append(timestamp)
time_even = timestamp
else:
odds.append(timestamp)
time_odd = timestamp
return odds, evens
def time_delta(odds, evens):
down = []
for i, time_y in enumerate(odds):
for j, time_x in enumerate(evens):
if i == j:
delta = time_y - time_x
if delta > hour_plus:
down.append('Downtime starts here:')
fx = time_x.strftime('%a %b %d %H:%M:%S %Y')
down.append(fx)
down.append('Downtime ends here:')
fy = time_y.strftime('%a %b %d %H:%M:%S %Y')
down.append(fy)
down.append('And the delta is:')
down.append(delta)
down.append('='*25)
print ('+'*35)
for list_item in down:
print list_item
log_dict()
divide_list(intervals)
time_delta(odds, evens)#Log file (over 13,000 lines)
None
Timestamp: Tue Dec 14 00:00:01 2010
2010-12-14-00:00:02 ping yahoo.com... icmp reply in 137.48ms
2010-12-14-00:00:03 ping yahoo.com... icmp reply in 84.98ms
2010-12-14-00:00:04 ping yahoo.com... icmp reply in 82.96ms
2010-12-14-00:00:05 ping yahoo.com... icmp reply in 142.28ms
None
Timestamp: Tue Dec 14 01:00:01 2010
2010-12-14-01:00:02 ping yahoo.com... icmp reply in 181.07ms
2010-12-14-01:00:03 ping yahoo.com... icmp reply in 116.49ms
2010-12-14-01:00:04 ping yahoo.com... icmp reply in 85.55ms
2010-12-14-01:00:05 ping yahoo.com... icmp reply in 82.51ms
None
Timestamp: Tue Dec 14 02:00:01 2010
2010-12-14-02:00:02 ping yahoo.com... icmp reply in 95.00ms
2010-12-14-02:00:03 ping yahoo.com... icmp reply in 83.00ms
2010-12-14-02:00:04 ping yahoo.com... icmp reply in 142.07ms
2010-12-14-02:00:05 ping yahoo.com... icmp reply in 146.46msASKER
"None" represents the output of a function that is part of a ping script I found, attached. I don't understand enough about socket connections to know whether I should change the script or my system configuration to get rid of the "None" output. I welcome your feedback of course, but I'd probably need to send you the entire script, and the answer might be complicated, not sure.
ICMP time-outs are represented but down times are not (the ping script depends on the host making a socket connection; if the host is down, that won't be attempted). This brings up a good point, since it's important to isolate down-time issues. But I would prefer to deal with that in the dictionary itself, by assigning a different value to the key, than to simply have missing time increments as I do now.
Yes. The ping script that writes to the input file is running on my laptop which is almost always on; for it to tell a better story, it should run from my server. But I'm not comfortable with the script yet (that could take a while, lol).
ICMP time-outs are represented but down times are not (the ping script depends on the host making a socket connection; if the host is down, that won't be attempted). This brings up a good point, since it's important to isolate down-time issues. But I would prefer to deal with that in the dictionary itself, by assigning a different value to the key, than to simply have missing time increments as I do now.
Yes. The ping script that writes to the input file is running on my laptop which is almost always on; for it to tell a better story, it should run from my server. But I'm not comfortable with the script yet (that could take a while, lol).
def do_one(dest_addr, timeout):
"""
Returns either the delay (in seconds) or none on timeout.
"""
icmp = socket.getprotobyname("icmp")
try:
my_socket = socket.socket(socket.AF_INET, socket.SOCK_RAW, icmp)
except socket.error, (errno, msg):
if errno == 1:
# Operation not permitted
msg = msg + (
" - Note that ICMP messages can only be sent from processes"
" running as root."
)
raise socket.error(msg)
raise # raise the original error
my_ID = os.getpid() & 0xFFFF
send_one_ping(my_socket, dest_addr, my_ID)
delay = receive_one_ping(my_socket, my_ID, timeout)
my_socket.close()
return delay
Have you thought about maybe using the 1st timestamp in the input file as a reference point, and computing the subsequent timestamps from this staring one? Therefore, all future/subsequent timestamps would be be a timedelta from this reference one.
What version of Python are you using?
If it is 2.7 or beyond, you might even consider an ordered-dictionary...
What version of Python are you using?
If it is 2.7 or beyond, you might even consider an ordered-dictionary...
ASKER
Yes I have, but I'm stuck on evaluating keys as I iterate through a loop. The only way I could find to do that was to split the keys (on the hourly timestamps, which are the ones I use to evaluate downtime) into two lists and subtract one set of keys from the other to find the time delta.
I have found many examples of loop iteration, including setting up a class which uses a next() function to get the value of the next key, but all I can manage to do is print the keys and values.
key.next() won't work because datetime objects have no attribute 'next'
I could assign a 'next' attribute to the keys to evaluate how much later one key's datetime value occurs from the previous one, but I don't know how to do it.
I have found many examples of loop iteration, including setting up a class which uses a next() function to get the value of the next key, but all I can manage to do is print the keys and values.
key.next() won't work because datetime objects have no attribute 'next'
I could assign a 'next' attribute to the keys to evaluate how much later one key's datetime value occurs from the previous one, but I don't know how to do it.
ASKER
I'm using python 2.5 - I could use a higher version since this is on my home computer but I hesitate to do that since I have 2.5 at work, I'm a beginner and more likely to get mixed up.
Yeah, I understand. Challenges abound.
That's why I brought up the possibility of using keys that are, in fact, timedelta from the 1st time seen in the log.
> ... including setting up a class which uses a next() function ...
Yeah, these are called generators. These tend to be useful when you have a large number of entries that you can't, or just don't want to, load into some data structure in memory. Consider the range() and xrange() functions. Calling range() will result in a list of value over the specified range. However, you wouldn't want to do this with too large a range (e.g., millions of values). For this, the xrange() function (which, in fact is a generator), may be more appropriate.
> key.next() won't work because datetime objects have no attribute 'next'
If you don't want to (or can't) use an ordered dictionary. You might consider maintaining an ordered list of values that are the dictionary keys. This would allow you to keep entries straight simply by adding a list data structure.
Does this make sense?
That's why I brought up the possibility of using keys that are, in fact, timedelta from the 1st time seen in the log.
> ... including setting up a class which uses a next() function ...
Yeah, these are called generators. These tend to be useful when you have a large number of entries that you can't, or just don't want to, load into some data structure in memory. Consider the range() and xrange() functions. Calling range() will result in a list of value over the specified range. However, you wouldn't want to do this with too large a range (e.g., millions of values). For this, the xrange() function (which, in fact is a generator), may be more appropriate.
> key.next() won't work because datetime objects have no attribute 'next'
If you don't want to (or can't) use an ordered dictionary. You might consider maintaining an ordered list of values that are the dictionary keys. This would allow you to keep entries straight simply by adding a list data structure.
Does this make sense?
Actually, the .next() is more general. It does not belong only to generators. It should be the supported interface of any iterable object. Try the example:
The iterkeys should work since Python 2.2.
However, you may assume something that is not there. Look at the lines 38 and 39 in the original code. The line 39 overrides the effect of the line 38 (the line 38 makes no sense).
D:\>python
Python 2.7.1 (r271:86832, Nov 27 2010, 17:19:03) [MSC v.1500 64 bit (AMD64)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> d = {3: 'ccc', 2: 'bbb', 1: 'aaa'}
>>> d
{1: 'aaa', 2: 'bbb', 3: 'ccc'}
>>> it = d.iterkeys()
>>> it
<dictionary-keyiterator object at 0x0000000002169638>
>>> it.next()
1
>>> it.next()
2
>>> it.next()
3
>>> it.next()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>>The iterkeys should work since Python 2.2.
However, you may assume something that is not there. Look at the lines 38 and 39 in the original code. The line 39 overrides the effect of the line 38 (the line 38 makes no sense).
> It does not belong only to generators.
True.
> Look at the lines 38 and 39 in the original code.
ok
> The line 39 overrides the effect of the line 38 (the line 38 makes no sense).
Yes, line 39 immediately replaces the value assigned in line 38, therefore line 38 serves no purpose.
Isn't it your code though?
> ... the script accounts for reported timestamps but not for down-time, when there is no connection at all.
You could add entries in your dictionary when no connection exists, and have the associated value be None.
True.
> Look at the lines 38 and 39 in the original code.
ok
> The line 39 overrides the effect of the line 38 (the line 38 makes no sense).
Yes, line 39 immediately replaces the value assigned in line 38, therefore line 38 serves no purpose.
Isn't it your code though?
> ... the script accounts for reported timestamps but not for down-time, when there is no connection at all.
You could add entries in your dictionary when no connection exists, and have the associated value be None.
HonorGod >Isn't it your code though?
No, it's by Sara ;)
Sara, could you attach a bit longer fragment of the log file with all the solved cases? Can you describe in words again your intention? In my opinion, you started "somehow" and you try to build on that code. However, the existing code may distract you from better solution.
No, it's by Sara ;)
Sara, could you attach a bit longer fragment of the log file with all the solved cases? Can you describe in words again your intention? In my opinion, you started "somehow" and you try to build on that code. However, the existing code may distract you from better solution.
ASKER
Thank you both for the feedback!! Let me try something and get back to you.
ASKER
Ok I made some changes to make the output more useful but I'm getting different results (no missing time intervals) so it's a bit confusing. I attach my updated code below and a longer version of the log file. I get over 13,000 entries in the time_warp dict and 344 entries where the icmp response time exceeds 200 ms.
None
Timestamp: Fri Apr 1 18:00:01 2011
2011-04-01-18:00:02 ping yahoo.com... icmp reply in 116.71ms
2011-04-01-18:00:03 ping yahoo.com... icmp reply in 86.62ms
2011-04-01-18:00:04 ping yahoo.com... icmp reply in 138.66ms
2011-04-01-18:00:05 ping yahoo.com... icmp reply in 141.66ms
None
Timestamp: Fri Apr 1 19:00:01 2011
2011-04-01-19:00:02 ping yahoo.com... icmp reply in 145.33ms
2011-04-01-19:00:03 ping yahoo.com... icmp reply in 85.63ms
2011-04-01-19:00:04 ping yahoo.com... icmp reply in 86.01ms
2011-04-01-19:00:05 ping yahoo.com... icmp reply in 146.25ms
None
Timestamp: Fri Apr 1 20:00:01 2011
2011-04-01-20:00:02 ping yahoo.com... icmp reply in 170.94ms
2011-04-01-20:00:03 ping yahoo.com... icmp reply in 121.02ms
2011-04-01-20:00:04 ping yahoo.com... icmp reply in 113.12ms
2011-04-01-20:00:05 ping yahoo.com... icmp reply in 89.75ms
None
Timestamp: Fri Apr 1 21:00:01 2011
2011-04-01-21:00:02 ping yahoo.com... icmp reply in 144.55ms
2011-04-01-21:00:03 ping yahoo.com... icmp reply in 140.68ms
2011-04-01-21:00:04 ping yahoo.com... icmp reply in 119.19ms
2011-04-01-21:00:05 ping yahoo.com... icmp reply in 88.59ms
None
Timestamp: Fri Apr 1 22:00:01 2011
2011-04-01-22:00:02 ping yahoo.com... icmp reply in 95.36ms
2011-04-01-22:00:03 ping yahoo.com... icmp reply in 139.14ms
2011-04-01-22:00:04 ping yahoo.com... icmp reply in 138.48ms
2011-04-01-22:00:05 ping yahoo.com... icmp reply in 119.92ms
None
Timestamp: Fri Apr 1 23:00:01 2011
2011-04-01-23:00:02 ping yahoo.com... icmp reply in 101.23ms
2011-04-01-23:00:03 ping yahoo.com... icmp reply in 143.91ms
2011-04-01-23:00:04 ping yahoo.com... icmp reply in 137.48ms
2011-04-01-23:00:05 ping yahoo.com... icmp reply in 122.28ms
None
Timestamp: Sat Apr 2 00:00:01 2011
2011-04-02-00:00:02 ping yahoo.com... icmp reply in 122.86ms
2011-04-02-00:00:03 ping yahoo.com... icmp reply in 87.15ms
2011-04-02-00:00:04 ping yahoo.com... icmp reply in 143.38ms
2011-04-02-00:00:05 ping yahoo.com... icmp reply in 144.23ms
None
Timestamp: Sat Apr 2 01:00:01 2011
2011-04-02-01:00:02 ping yahoo.com... icmp reply in 147.01ms
2011-04-02-01:00:03 ping yahoo.com... icmp reply in 87.53ms
2011-04-02-01:00:04 ping yahoo.com... icmp reply in 86.24ms
2011-04-02-01:00:05 ping yahoo.com... icmp reply in 146.87ms
None
Timestamp: Sat Apr 2 02:00:01 2011
2011-04-02-02:00:02 ping yahoo.com... failed. (timeout within 2sec.)
2011-04-02-02:00:03 ping yahoo.com... icmp reply in 153.01ms
2011-04-02-02:00:04 ping yahoo.com... icmp reply in 86.78ms
2011-04-02-02:00:05 ping yahoo.com... icmp reply in 85.99ms
None
Timestamp: Sat Apr 2 03:00:01 2011
2011-04-02-03:00:02 ping yahoo.com... icmp reply in 166.88ms
2011-04-02-03:00:03 ping yahoo.com... icmp reply in 138.36ms
2011-04-02-03:00:04 ping yahoo.com... icmp reply in 118.62ms
2011-04-02-03:00:05 ping yahoo.com... icmp reply in 86.86ms
None
Timestamp: Sat Apr 2 04:00:01 2011
2011-04-02-04:00:02 ping yahoo.com... icmp reply in 126.70ms
2011-04-02-04:00:03 ping yahoo.com... icmp reply in 144.53ms
2011-04-02-04:00:04 ping yahoo.com... icmp reply in 143.99ms
2011-04-02-04:00:05 ping yahoo.com... icmp reply in 116.89ms
None
Timestamp: Sat Apr 2 05:00:01 2011
2011-04-02-05:00:02 ping yahoo.com... icmp reply in 119.11ms
2011-04-02-05:00:03 ping yahoo.com... icmp reply in 88.44ms
2011-04-02-05:00:04 ping yahoo.com... icmp reply in 86.18ms
2011-04-02-05:00:05 ping yahoo.com... icmp reply in 145.99ms
None
Timestamp: Sat Apr 2 06:00:01 2011
2011-04-02-06:00:02 ping yahoo.com... icmp reply in 161.82ms
2011-04-02-06:00:03 ping yahoo.com... icmp reply in 119.04ms
2011-04-02-06:00:04 ping yahoo.com... icmp reply in 93.36ms
2011-04-02-06:00:05 ping yahoo.com... icmp reply in 85.55ms
None
Timestamp: Sat Apr 2 07:00:01 2011
2011-04-02-07:00:02 ping yahoo.com... icmp reply in 167.47ms
2011-04-02-07:00:03 ping yahoo.com... icmp reply in 141.47ms
2011-04-02-07:00:04 ping yahoo.com... icmp reply in 115.80ms
2011-04-02-07:00:05 ping yahoo.com... icmp reply in 86.00ms
None
Timestamp: Sat Apr 2 08:00:01 2011
2011-04-02-08:00:02 ping yahoo.com... icmp reply in 109.56ms
2011-04-02-08:00:03 ping yahoo.com... icmp reply in 144.48ms
2011-04-02-08:00:04 ping yahoo.com... icmp reply in 140.31ms
2011-04-02-08:00:05 ping yahoo.com... icmp reply in 116.83ms
None
Timestamp: Sat Apr 2 09:00:01 2011
2011-04-02-09:00:02 ping yahoo.com... icmp reply in 113.82ms
2011-04-02-09:00:03 ping yahoo.com... icmp reply in 85.12ms
2011-04-02-09:00:04 ping yahoo.com... icmp reply in 141.70ms
2011-04-02-09:00:05 ping yahoo.com... icmp reply in 142.25ms
None
Timestamp: Sat Apr 2 10:00:01 2011
2011-04-02-10:00:02 ping yahoo.com... icmp reply in 134.08ms
2011-04-02-10:00:03 ping yahoo.com... icmp reply in 89.87ms
2011-04-02-10:00:04 ping yahoo.com... icmp reply in 85.26ms
2011-04-02-10:00:05 ping yahoo.com... icmp reply in 138.19ms
None
Timestamp: Sat Apr 2 11:00:01 2011
2011-04-02-11:00:02 ping yahoo.com... icmp reply in 92.17ms
2011-04-02-11:00:03 ping yahoo.com... icmp reply in 143.20ms
2011-04-02-11:00:04 ping yahoo.com... icmp reply in 142.92ms
2011-04-02-11:00:05 ping yahoo.com... icmp reply in 118.15ms
None
Timestamp: Sat Apr 2 12:00:01 2011
2011-04-02-12:00:02 ping yahoo.com... icmp reply in 117.61ms
2011-04-02-12:00:03 ping yahoo.com... icmp reply in 85.39ms
2011-04-02-12:00:04 ping yahoo.com... icmp reply in 138.18ms
2011-04-02-12:00:05 ping yahoo.com... icmp reply in 143.40ms
None
Timestamp: Sat Apr 2 13:00:01 2011
2011-04-02-13:00:02 ping yahoo.com... icmp reply in 146.89ms
2011-04-02-13:00:03 ping yahoo.com... icmp reply in 96.13ms
2011-04-02-13:00:04 ping yahoo.com... icmp reply in 139.28ms
2011-04-02-13:00:05 ping yahoo.com... icmp reply in 144.69ms
None
Timestamp: Sat Apr 2 14:00:01 2011
2011-04-02-14:00:02 ping yahoo.com... icmp reply in 121.86ms
2011-04-02-14:00:03 ping yahoo.com... icmp reply in 88.96ms
2011-04-02-14:00:04 ping yahoo.com... icmp reply in 85.81ms
2011-04-02-14:00:05 ping yahoo.com... icmp reply in 140.84ms
None
Timestamp: Sat Apr 2 15:00:01 2011
2011-04-02-15:00:02 ping yahoo.com... icmp reply in 136.24ms
2011-04-02-15:00:03 ping yahoo.com... icmp reply in 117.00ms
2011-04-02-15:00:04 ping yahoo.com... icmp reply in 86.22ms
2011-04-02-15:00:05 ping yahoo.com... icmp reply in 85.59ms
None
Timestamp: Sat Apr 2 16:00:01 2011
2011-04-02-16:00:02 ping yahoo.com... icmp reply in 145.21ms
2011-04-02-16:00:03 ping yahoo.com... icmp reply in 136.97ms
2011-04-02-16:00:04 ping yahoo.com... icmp reply in 118.16ms
2011-04-02-16:00:05 ping yahoo.com... icmp reply in 87.71ms
None
Timestamp: Sat Apr 2 17:00:01 2011
2011-04-02-17:00:02 ping yahoo.com... icmp reply in 89.49ms
2011-04-02-17:00:03 ping yahoo.com... icmp reply in 145.37ms
2011-04-02-17:00:04 ping yahoo.com... icmp reply in 138.57ms
2011-04-02-17:00:05 ping yahoo.com... icmp reply in 118.83ms
None
Timestamp: Sat Apr 2 18:00:01 2011
2011-04-02-18:00:02 ping yahoo.com... icmp reply in 89.37ms
2011-04-02-18:00:03 ping yahoo.com... icmp reply in 85.66ms
2011-04-02-18:00:04 ping yahoo.com... icmp reply in 145.44ms
2011-04-02-18:00:05 ping yahoo.com... icmp reply in 138.55ms
None
Timestamp: Sat Apr 2 19:00:01 2011
2011-04-02-19:00:02 ping yahoo.com... icmp reply in 145.32ms
2011-04-02-19:00:03 ping yahoo.com... icmp reply in 117.85ms
2011-04-02-19:00:04 ping yahoo.com... icmp reply in 86.21ms
2011-04-02-19:00:05 ping yahoo.com... icmp reply in 89.30ms
None
Timestamp: Sat Apr 2 20:00:01 2011
2011-04-02-20:00:02 ping yahoo.com... icmp reply in 144.11ms
2011-04-02-20:00:03 ping yahoo.com... icmp reply in 145.03ms
2011-04-02-20:00:04 ping yahoo.com... icmp reply in 118.16ms
2011-04-02-20:00:05 ping yahoo.com... icmp reply in 97.10ms
None
Timestamp: Sat Apr 2 21:00:01 2011
2011-04-02-21:00:02 ping yahoo.com... icmp reply in 127.76ms
2011-04-02-21:00:03 ping yahoo.com... icmp reply in 145.23ms
2011-04-02-21:00:04 ping yahoo.com... icmp reply in 139.74ms
2011-04-02-21:00:05 ping yahoo.com... icmp reply in 118.86ms
None
Timestamp: Sat Apr 2 22:00:01 2011
2011-04-02-22:00:02 ping yahoo.com... icmp reply in 93.33ms
2011-04-02-22:00:03 ping yahoo.com... icmp reply in 94.66ms
2011-04-02-22:00:04 ping yahoo.com... icmp reply in 150.82ms
2011-04-02-22:00:05 ping yahoo.com... icmp reply in 140.38ms
None
Timestamp: Sat Apr 2 23:00:01 2011
2011-04-02-23:00:02 ping yahoo.com... icmp reply in 151.80ms
2011-04-02-23:00:03 ping yahoo.com... icmp reply in 90.86ms
2011-04-02-23:00:04 ping yahoo.com... icmp reply in 95.15ms
2011-04-02-23:00:05 ping yahoo.com... icmp reply in 146.72ms
None
Timestamp: Sun Apr 3 00:00:01 2011
2011-04-03-00:00:02 ping yahoo.com... icmp reply in 173.94ms
2011-04-03-00:00:03 ping yahoo.com... icmp reply in 118.67ms
2011-04-03-00:00:04 ping yahoo.com... icmp reply in 86.70ms
2011-04-03-00:00:05 ping yahoo.com... icmp reply in 87.36ms
None
Timestamp: Sun Apr 3 01:00:01 2011
2011-04-03-01:00:02 ping yahoo.com... icmp reply in 118.35ms
2011-04-03-01:00:03 ping yahoo.com... icmp reply in 86.57ms
2011-04-03-01:00:04 ping yahoo.com... icmp reply in 91.01ms
2011-04-03-01:00:05 ping yahoo.com... icmp reply in 141.02ms
None
Timestamp: Sun Apr 3 02:00:01 2011
2011-04-03-02:00:02 ping yahoo.com... icmp reply in 170.85ms
2011-04-03-02:00:03 ping yahoo.com... icmp reply in 116.75ms
2011-04-03-02:00:04 ping yahoo.com... icmp reply in 86.23ms
2011-04-03-02:00:05 ping yahoo.com... icmp reply in 95.19ms
None
Timestamp: Sun Apr 3 03:00:01 2011
2011-04-03-03:00:02 ping yahoo.com... icmp reply in 172.70ms
2011-04-03-03:00:03 ping yahoo.com... icmp reply in 143.92ms
2011-04-03-03:00:04 ping yahoo.com... icmp reply in 116.90ms
2011-04-03-03:00:05 ping yahoo.com... icmp reply in 86.25ms
None
Timestamp: Sun Apr 3 04:00:01 2011
2011-04-03-04:00:02 ping yahoo.com... icmp reply in 153.48ms
2011-04-03-04:00:03 ping yahoo.com... icmp reply in 140.42ms
2011-04-03-04:00:04 ping yahoo.com... icmp reply in 139.98ms
2011-04-03-04:00:05 ping yahoo.com... icmp reply in 117.43ms
None
Timestamp: Sun Apr 3 05:00:01 2011
2011-04-03-05:00:02 ping yahoo.com... icmp reply in 119.92ms
2011-04-03-05:00:03 ping yahoo.com... icmp reply in 85.92ms
2011-04-03-05:00:04 ping yahoo.com... icmp reply in 138.63ms
2011-04-03-05:00:05 ping yahoo.com... icmp reply in 138.44ms
None
Timestamp: Sun Apr 3 06:00:01 2011
2011-04-03-06:00:02 ping yahoo.com... icmp reply in 154.25ms
2011-04-03-06:00:03 ping yahoo.com... icmp reply in 140.43ms
2011-04-03-06:00:04 ping yahoo.com... icmp reply in 85.78ms
2011-04-03-06:00:05 ping yahoo.com... failed. (timeout within 2sec.)
None
Timestamp: Sun Apr 3 07:00:01 2011
2011-04-03-07:00:02 ping yahoo.com... icmp reply in 142.52ms
2011-04-03-07:00:03 ping yahoo.com... icmp reply in 116.73ms
2011-04-03-07:00:04 ping yahoo.com... icmp reply in 88.66ms
2011-04-03-07:00:05 ping yahoo.com... icmp reply in 85.78ms
None
Timestamp: Sun Apr 3 08:00:01 2011
2011-04-03-08:00:02 ping yahoo.com... icmp reply in 163.20ms
2011-04-03-08:00:03 ping yahoo.com... icmp reply in 137.86ms
2011-04-03-08:00:04 ping yahoo.com... icmp reply in 118.17ms
2011-04-03-08:00:05 ping yahoo.com... failed. (timeout within 2sec.)
None
Timestamp: Sun Apr 3 09:00:01 2011
2011-04-03-09:00:02 ping yahoo.com... icmp reply in 89.70ms
2011-04-03-09:00:03 ping yahoo.com... icmp reply in 143.56ms
2011-04-03-09:00:04 ping yahoo.com... icmp reply in 135.73ms
2011-04-03-09:00:05 ping yahoo.com... icmp reply in 117.98ms
None
Timestamp: Sun Apr 3 10:00:01 2011
2011-04-03-10:00:02 ping yahoo.com... icmp reply in 128.65ms
2011-04-03-10:00:03 ping yahoo.com... icmp reply in 85.15ms
2011-04-03-10:00:04 ping yahoo.com... icmp reply in 137.49ms
2011-04-03-10:00:05 ping yahoo.com... icmp reply in 141.63ms
None
Timestamp: Sun Apr 3 11:00:01 2011
2011-04-03-11:00:02 ping yahoo.com... icmp reply in 131.34ms
2011-04-03-11:00:03 ping yahoo.com... icmp reply in 86.05ms
2011-04-03-11:00:04 ping yahoo.com... icmp reply in 85.57ms
2011-04-03-11:00:05 ping yahoo.com... icmp reply in 136.89ms
None
Timestamp: Sun Apr 3 12:00:01 2011
2011-04-03-12:00:02 ping yahoo.com... icmp reply in 167.87ms
2011-04-03-12:00:03 ping yahoo.com... icmp reply in 124.57ms
2011-04-03-12:00:04 ping yahoo.com... icmp reply in 147.07ms
2011-04-03-12:00:05 ping yahoo.com... icmp reply in 86.23ms
None
Timestamp: Sun Apr 3 13:00:01 2011
2011-04-03-13:00:02 ping yahoo.com... icmp reply in 89.91ms
2011-04-03-13:00:03 ping yahoo.com... icmp reply in 139.59ms
2011-04-03-13:00:04 ping yahoo.com... icmp reply in 141.73ms
2011-04-03-13:00:05 ping yahoo.com... icmp reply in 118.15ms
None
Timestamp: Sun Apr 3 14:00:01 2011
2011-04-03-14:00:02 ping yahoo.com... icmp reply in 90.38ms
2011-04-03-14:00:03 ping yahoo.com... icmp reply in 87.45ms
2011-04-03-14:00:04 ping yahoo.com... icmp reply in 145.27ms
2011-04-03-14:00:05 ping yahoo.com... icmp reply in 143.37ms
None
Timestamp: Sun Apr 3 15:00:01 2011
2011-04-03-15:00:02 ping yahoo.com... icmp reply in 137.64ms
2011-04-03-15:00:03 ping yahoo.com... icmp reply in 88.30ms
2011-04-03-15:00:04 ping yahoo.com... icmp reply in 85.49ms
2011-04-03-15:00:05 ping yahoo.com... icmp reply in 146.52ms
None
Timestamp: Sun Apr 3 16:00:01 2011
2011-04-03-16:00:02 ping yahoo.com... icmp reply in 144.31ms
2011-04-03-16:00:03 ping yahoo.com... icmp reply in 118.08ms
2011-04-03-16:00:04 ping yahoo.com... icmp reply in 93.82ms
2011-04-03-16:00:05 ping yahoo.com... icmp reply in 92.58ms
None
Timestamp: Sun Apr 3 17:00:01 2011
2011-04-03-17:00:02 ping yahoo.com... icmp reply in 150.59ms
2011-04-03-17:00:03 ping yahoo.com... icmp reply in 136.55ms
2011-04-03-17:00:04 ping yahoo.com... icmp reply in 120.39ms
2011-04-03-17:00:05 ping yahoo.com... icmp reply in 87.88ms
None
Timestamp: Sun Apr 3 18:00:01 2011
2011-04-03-18:00:02 ping yahoo.com... icmp reply in 89.27ms
2011-04-03-18:00:03 ping yahoo.com... icmp reply in 141.44ms
2011-04-03-18:00:04 ping yahoo.com... icmp reply in 144.60ms
2011-04-03-18:00:05 ping yahoo.com... icmp reply in 123.16ms
None
Timestamp: Sun Apr 3 19:00:01 2011
2011-04-03-19:00:02 ping yahoo.com... icmp reply in 89.81ms
2011-04-03-19:00:03 ping yahoo.com... icmp reply in 86.78ms
2011-04-03-19:00:04 ping yahoo.com... icmp reply in 139.76ms
2011-04-03-19:00:05 ping yahoo.com... icmp reply in 137.90ms
None
Timestamp: Sun Apr 3 20:00:01 2011
2011-04-03-20:00:02 ping yahoo.com... icmp reply in 124.43ms
2011-04-03-20:00:03 ping yahoo.com... icmp reply in 90.87ms
2011-04-03-20:00:04 ping yahoo.com... icmp reply in 97.20ms
2011-04-03-20:00:05 ping yahoo.com... icmp reply in 146.93ms
None
Timestamp: Sun Apr 3 21:00:01 2011
2011-04-03-21:00:02 ping yahoo.com... icmp reply in 1069.80ms
2011-04-03-21:00:03 ping yahoo.com... icmp reply in 1503.66ms
2011-04-03-21:00:04 ping yahoo.com... icmp reply in 1593.94ms
2011-04-03-21:00:05 ping yahoo.com... icmp reply in 1805.51ms
None
Timestamp: Sun Apr 3 22:00:02 2011
2011-04-03-22:00:03 ping yahoo.com... icmp reply in 164.05ms
2011-04-03-22:00:04 ping yahoo.com... icmp reply in 149.05ms
2011-04-03-22:00:05 ping yahoo.com... icmp reply in 127.17ms
2011-04-03-22:00:06 ping yahoo.com... icmp reply in 98.12ms
None
Timestamp: Sun Apr 3 23:00:01 2011
2011-04-03-23:00:02 ping yahoo.com... icmp reply in 93.03ms
2011-04-03-23:00:03 ping yahoo.com... icmp reply in 147.68ms
2011-04-03-23:00:04 ping yahoo.com... icmp reply in 137.32ms
2011-04-03-23:00:05 ping yahoo.com... icmp reply in 119.29ms
None
Timestamp: Mon Apr 4 00:00:01 2011
2011-04-04-00:00:02 ping yahoo.com... icmp reply in 120.27ms
2011-04-04-00:00:03 ping yahoo.com... icmp reply in 84.81ms
2011-04-04-00:00:04 ping yahoo.com... icmp reply in 139.73ms
2011-04-04-00:00:05 ping yahoo.com... icmp reply in 137.98ms
None
Timestamp: Mon Apr 4 01:00:01 2011
2011-04-04-01:00:02 ping yahoo.com... icmp reply in 118.49ms
2011-04-04-01:00:03 ping yahoo.com... icmp reply in 86.09ms
2011-04-04-01:00:04 ping yahoo.com... icmp reply in 89.29ms
2011-04-04-01:00:05 ping yahoo.com... icmp reply in 140.78ms
None
Timestamp: Mon Apr 4 02:00:01 2011
2011-04-04-02:00:02 ping yahoo.com... icmp reply in 180.01ms
2011-04-04-02:00:03 ping yahoo.com... icmp reply in 117.87ms
2011-04-04-02:00:04 ping yahoo.com... icmp reply in 86.63ms
2011-04-04-02:00:05 ping yahoo.com... icmp reply in 87.62ms
None
Timestamp: Mon Apr 4 03:00:01 2011
2011-04-04-03:00:02 ping yahoo.com... icmp reply in 175.23ms
2011-04-04-03:00:03 ping yahoo.com... icmp reply in 143.75ms
2011-04-04-03:00:04 ping yahoo.com... icmp reply in 117.98ms
2011-04-04-03:00:05 ping yahoo.com... icmp reply in 86.66ms
None
Timestamp: Mon Apr 4 04:00:01 2011
2011-04-04-04:00:02 ping yahoo.com... icmp reply in 90.07ms
2011-04-04-04:00:03 ping yahoo.com... icmp reply in 142.92ms
2011-04-04-04:00:04 ping yahoo.com... icmp reply in 140.78ms
2011-04-04-04:00:05 ping yahoo.com... icmp reply in 117.31ms
None
Timestamp: Mon Apr 4 05:00:01 2011
2011-04-04-05:00:02 ping yahoo.com... icmp reply in 124.91ms
2011-04-04-05:00:03 ping yahoo.com... icmp reply in 86.79ms
2011-04-04-05:00:04 ping yahoo.com... icmp reply in 139.71ms
2011-04-04-05:00:05 ping yahoo.com... icmp reply in 146.48ms
None
Timestamp: Mon Apr 4 06:00:01 2011
2011-04-04-06:00:02 ping yahoo.com... icmp reply in 157.56ms
2011-04-04-06:00:03 ping yahoo.com... icmp reply in 86.04ms
2011-04-04-06:00:04 ping yahoo.com... icmp reply in 106.22ms
2011-04-04-06:00:05 ping yahoo.com... icmp reply in 139.81ms
None
Timestamp: Mon Apr 4 07:00:01 2011
2011-04-04-07:00:02 ping yahoo.com... icmp reply in 139.26ms
2011-04-04-07:00:03 ping yahoo.com... icmp reply in 141.26ms
2011-04-04-07:00:04 ping yahoo.com... icmp reply in 117.73ms
2011-04-04-07:00:05 ping yahoo.com... icmp reply in 124.73ms
None
Timestamp: Mon Apr 4 08:00:01 2011
2011-04-04-08:00:02 ping yahoo.com... icmp reply in 122.58ms
2011-04-04-08:00:03 ping yahoo.com... icmp reply in 141.88ms
2011-04-04-08:00:04 ping yahoo.com... icmp reply in 142.12ms
2011-04-04-08:00:05 ping yahoo.com... icmp reply in 118.48ms
None
Timestamp: Mon Apr 4 09:00:01 2011
2011-04-04-09:00:02 ping yahoo.com... icmp reply in 114.96ms
2011-04-04-09:00:03 ping yahoo.com... icmp reply in 86.72ms
2011-04-04-09:00:04 ping yahoo.com... icmp reply in 144.48ms
2011-04-04-09:00:05 ping yahoo.com... icmp reply in 137.99ms
None
Timestamp: Mon Apr 4 10:00:01 2011
2011-04-04-10:00:02 ping yahoo.com... icmp reply in 147.32ms
2011-04-04-10:00:03 ping yahoo.com... icmp reply in 87.84ms
2011-04-04-10:00:04 ping yahoo.com... icmp reply in 87.39ms
2011-04-04-10:00:05 ping yahoo.com... icmp reply in 142.70ms
None
Timestamp: Mon Apr 4 11:00:01 2011
2011-04-04-11:00:02 ping yahoo.com... icmp reply in 166.55ms
2011-04-04-11:00:03 ping yahoo.com... icmp reply in 119.20ms
2011-04-04-11:00:04 ping yahoo.com... icmp reply in 88.40ms
2011-04-04-11:00:05 ping yahoo.com... icmp reply in 86.73ms
None
Timestamp: Mon Apr 4 12:00:01 2011
2011-04-04-12:00:02 ping yahoo.com... icmp reply in 162.93ms
2011-04-04-12:00:03 ping yahoo.com... icmp reply in 141.53ms
2011-04-04-12:00:04 ping yahoo.com... icmp reply in 117.73ms
2011-04-04-12:00:05 ping yahoo.com... icmp reply in 87.99ms
None
Timestamp: Mon Apr 4 13:00:01 2011
2011-04-04-13:00:02 ping yahoo.com... icmp reply in 174.09ms
2011-04-04-13:00:03 ping yahoo.com... icmp reply in 135.77ms
2011-04-04-13:00:04 ping yahoo.com... icmp reply in 117.30ms
2011-04-04-13:00:05 ping yahoo.com... icmp reply in 86.16ms
None
Timestamp: Mon Apr 4 14:00:01 2011
2011-04-04-14:00:02 ping yahoo.com... icmp reply in 113.14ms
2011-04-04-14:00:03 ping yahoo.com... icmp reply in 137.85ms
2011-04-04-14:00:04 ping yahoo.com... icmp reply in 143.38ms
2011-04-04-14:00:05 ping yahoo.com... icmp reply in 120.76ms
None
Timestamp: Mon Apr 4 15:00:01 2011
2011-04-04-15:00:02 ping yahoo.com... icmp reply in 89.01ms
2011-04-04-15:00:03 ping yahoo.com... icmp reply in 86.01ms
2011-04-04-15:00:04 ping yahoo.com... icmp reply in 141.73ms
2011-04-04-15:00:05 ping yahoo.com... icmp reply in 139.66ms
None
Timestamp: Mon Apr 4 16:00:01 2011
2011-04-04-16:00:02 ping yahoo.com... icmp reply in 136.14ms
2011-04-04-16:00:03 ping yahoo.com... icmp reply in 87.11ms
2011-04-04-16:00:04 ping yahoo.com... icmp reply in 85.53ms
2011-04-04-16:00:05 ping yahoo.com... icmp reply in 142.10ms
Nonehour_max = timedelta(seconds=3601)
hour_min = timedelta(seconds=3595)
InLog = open('ping.log', 'r')
time_warp = {}
def log_dict():
for line in InLog:
line = line.strip()
if len(line) == 0:
continue
elif 'None' in line:
continue
elif 'Timestamp' in line:
time_value = line[11:]
date_object = datetime.strptime(time_value, '%a %b %d %H:%M:%S %Y')
delay_value = ' '
else:
my_values = line.split(' ')
time_value = str(my_values[0])
delay_str = str(my_values[6])
delay_val = str(delay_str[0:-2])
date_object = datetime.strptime(time_value, '%Y-%m-%d-%H:%M:%S')
if re.search("[a-zA-Z_']", delay_val):
delay_value = 2000
else:
delay_value = round(float(delay_val), 2)
time_warp[date_object] = delay_value
for key in sorted(time_warp.keys()):
print key, time_warp[key]
print '=' * 35
print len(time_warp), 'timestamps evaluated'
print '=' * 35
return time_warp
def find_delays(time_warp):
odds = []
evens = []
delays = {}
index = 0
for key in sorted(time_warp.keys()):
index = index + 1
if index % 2 == 0:
even = key
evens.append(even)
else:
odd = key
odds.append(odd)
i = 0
j = 0
for i, even in enumerate(evens):
for j, odd in enumerate(odds):
if i == j:
delta = even - odd
if delta > hour_max:
delays[delta] = even, odd
index = 0
for key in time_warp.keys():
if time_warp[key] != ' ':
if time_warp[key] > 200:
index = index + 1
time_warp[key] = round(float(time_warp[key]), 2)
fmt_key = key.strftime('%a %b %d %H:%M:%S %Y')
print key, time_warp[key], fmt_key
print '=' * 35
print len(delays), 'time intervals exceeded'
print index, 'delays evaluated'
print '=' * 35
log_dict()
find_delays(time_warp)
I did not study details of your code, yet. Anyway, I believe there is no reason to use a dictionary for the time_warp. Here is the same code modified for using the list instead. This way you can simplify the looping through. Also, the order of the information is the same as in the log -- no need to sort the keys of the dictionary:
c.py
c.py
from datetime import datetime, date, time, timedelta
import re
import os, sys
hour_max = timedelta(seconds=3601)
hour_min = timedelta(seconds=3595)
InLog = open('ping.log', 'r')
time_warp = [] # changed from the dictionary to the list
def log_dict():
for line in InLog:
line = line.strip()
if len(line) == 0:
continue
elif line.startswith('None'):
continue
elif line.startswith('Timestamp'):
time_value = line[11:]
date_object = datetime.strptime(time_value, '%a %b %d %H:%M:%S %Y')
delay_value = ' '
else:
my_values = line.split(' ')
time_value = str(my_values[0])
delay_str = str(my_values[6])
delay_val = str(delay_str[0:-2])
date_object = datetime.strptime(time_value, '%Y-%m-%d-%H:%M:%S')
if re.search("[a-zA-Z_']", delay_val):
delay_value = 2000
else:
delay_value = round(float(delay_val), 2)
time_warp.append( (date_object, delay_value) ) # changed from the dictionary to the list
for date, delay in time_warp: # list of tuples
print date, delay
print '=' * 35
print len(time_warp), 'timestamps evaluated'
print '=' * 35
return time_warp
def find_delays(time_warp):
odds = []
evens = []
delays = {}
index = 0
for date, delay in time_warp:
index = index + 1
if index % 2 == 0:
evens.append(date)
else:
odds.append(date)
i = 0
j = 0
for i, even in enumerate(evens):
for j, odd in enumerate(odds):
if i == j:
delta = even - odd
if delta > hour_max:
delays[delta] = even, odd
cnt = 0
for date, delay in time_warp: # list of tuples
if delay != ' ':
if delay > 200:
cnt = cnt + 1
date_str = date.strftime('%a %b %d %H:%M:%S %Y')
print date, delay, date_str
print '=' * 35
print len(delays), 'time intervals exceeded'
print cnt, 'delays evaluated'
print '=' * 35
log_dict()
find_delays(time_warp)
In my opinion, there is no reason for the round(xxx, 2). The rounding should be done only for output text formatting. I do not understand the intention of using evens and odds lists in the find_delays(). Can you explaint it by words?
The constant 2000 should be changed to 2000.0 to make the delay_value always of the float type if it is assigned a number.
Here is the slightly modified code:
d.py
The constant 2000 should be changed to 2000.0 to make the delay_value always of the float type if it is assigned a number.
Here is the slightly modified code:
d.py
from datetime import datetime, date, time, timedelta
import re
import os, sys
hour_max = timedelta(seconds=3601)
hour_min = timedelta(seconds=3595)
InLog = open('ping.log', 'r')
time_warp = [] # changed from the dictionary to the list
def log_dict():
for line in InLog:
line = line.strip()
if len(line) == 0:
continue
elif line.startswith('None'):
continue
elif line.startswith('Timestamp'):
time_value = line[11:]
date_object = datetime.strptime(time_value, '%a %b %d %H:%M:%S %Y')
delay_value = ' '
else:
my_values = line.split(' ')
time_value = str(my_values[0])
delay_str = str(my_values[6])
delay_val = str(delay_str[0:-2])
date_object = datetime.strptime(time_value, '%Y-%m-%d-%H:%M:%S')
if re.search("[a-zA-Z_']", delay_val):
delay_value = 2000.0
else:
delay_value = float(delay_val)
time_warp.append( (date_object, delay_value) ) # changed from the dictionary to the list
for date, delay in time_warp: # list of tuples
print date, delay
print '=' * 35
print len(time_warp), 'timestamps evaluated'
print '=' * 35
return time_warp
def find_delays(time_warp):
odds = []
evens = []
delays = {}
index = 0
for date, delay in time_warp:
index = index + 1
if index % 2 == 0:
evens.append(date)
else:
odds.append(date)
i = 0
j = 0
for i, even in enumerate(evens):
for j, odd in enumerate(odds):
if i == j:
delta = even - odd
if delta > hour_max:
delays[delta] = even, odd
cnt = 0
for date, delay in time_warp: # list of tuples
if delay != ' ':
if delay > 200:
cnt = cnt + 1
date_str = date.strftime('%a %b %d %H:%M:%S %Y')
print date, delay, date_str
print '=' * 35
print len(delays), 'time intervals exceeded'
print cnt, 'delays evaluated'
print '=' * 35
log_dict()
find_delays(time_warp)ASKER
Thanks pepr!
> I do not understand the intention of using evens and odds lists in the find_delays()
I cannot find another way to measure the time interval between timestamps than to separate out the date objects into two lists. Everything else I have tried results in my subtracting each key or list value from itself.
> I do not understand the intention of using evens and odds lists in the find_delays()
I cannot find another way to measure the time interval between timestamps than to separate out the date objects into two lists. Everything else I have tried results in my subtracting each key or list value from itself.
You can remember the last_date timestamp from the previous loop of input processing and calculate the interval. The value can be added to collected data. (Or it could be calculated later.) I am not sure, what precisely you want to test.
Try the following code:
Try the following code:
from datetime import datetime, date, time, timedelta
import re
import os, sys
hour_max = timedelta(seconds=3601)
hour_min = timedelta(seconds=3595)
InLog = open('ping.log', 'r')
time_warp = [] # changed from the dictionary to the list
def log_dict():
last_date = None
for line in InLog:
line = line.strip()
if len(line) == 0:
continue
elif line.startswith('None'):
continue
elif line.startswith('Timestamp'):
time_value = line[11:]
date_object = datetime.strptime(time_value, '%a %b %d %H:%M:%S %Y')
delay_value = 0.0
if last_date is None:
last_date = date_object
else:
my_values = line.split(' ')
time_value = str(my_values[0])
delay_str = str(my_values[6])
delay_val = str(delay_str[0:-2])
date_object = datetime.strptime(time_value, '%Y-%m-%d-%H:%M:%S')
if re.search("[a-zA-Z_']", delay_val):
delay_value = 2000.0
else:
delay_value = float(delay_val)
time_warp.append( (date_object, delay_value, date_object-last_date) )
last_date = date_object
for date, delay, interval in time_warp:
print date, delay, interval
print '=' * 35
print len(time_warp), 'timestamps evaluated'
print '=' * 35
return time_warp
def find_delays(time_warp):
cnt = 0
intervals = []
for date, delay, interval in time_warp:
if interval > hour_max:
intervals.append( (date, delay, interval) )
print 'time interval exceeded>', date, interval
if delay > 200.0:
cnt = cnt + 1
date_str = date.strftime('%a %b %d %H:%M:%S %Y')
print date, delay, date_str
print '=' * 35
print len(intervals), 'time intervals exceeded'
print cnt, 'delays evaluated'
print '=' * 35
log_dict()
find_delays(time_warp)ASKER
Thanks pepr! I don't quite understand how this works:
>You can remember the last_date timestamp from the previous loop of input processing and calculate the interval.
If I understood how the last_date object works, I could apply this to a different set of circumstances. For example, the interval between the timestamps is not important for the icmp requests, since each one is always 1 second or so later than the previous request.
However the 'Timestamp' lines in the log, which are supposed to be one hour apart, are sometimes many hours apart. There are fewer than 3000 Timestamp lines in my log (from over 13,000 total), so if I store the Timestamp lines in say, a dictionary of date objects and assign interval values to each key, then I should get the same number of 'time interval exceeded>' results as in your code above. But I don't.
I know that I don't need an intervals dict for this - an intervals list would probably be easier - but I wanted to make sure that I wasn't simply copying your code. In any case, mine doesn't work, probably because I don't understand how to use the 'last_date' object.
>You can remember the last_date timestamp from the previous loop of input processing and calculate the interval.
If I understood how the last_date object works, I could apply this to a different set of circumstances. For example, the interval between the timestamps is not important for the icmp requests, since each one is always 1 second or so later than the previous request.
However the 'Timestamp' lines in the log, which are supposed to be one hour apart, are sometimes many hours apart. There are fewer than 3000 Timestamp lines in my log (from over 13,000 total), so if I store the Timestamp lines in say, a dictionary of date objects and assign interval values to each key, then I should get the same number of 'time interval exceeded>' results as in your code above. But I don't.
I know that I don't need an intervals dict for this - an intervals list would probably be easier - but I wanted to make sure that I wasn't simply copying your code. In any case, mine doesn't work, probably because I don't understand how to use the 'last_date' object.
from datetime import datetime, date, time, timedelta
import re
import os, sys
hour_max = timedelta(seconds=3605)
hour_min = timedelta(seconds=3595)
InLog = open('ping.log', 'r')
intervals = {}
def log_dict():
last_date = None
for line in InLog:
line = line.strip()
if len(line) == 0:
continue
elif line.startswith('None'):
continue
elif line.startswith('Timestamp'):
time_value = line[11:]
date_object = datetime.strptime(time_value, '%a %b %d %H:%M:%S %Y')
delay_value = 0.0
intervals[date_object] = {}
if last_date is None:
last_date = date_object
for date_object in intervals.keys():
intervals[date_object] = date_object-last_date
last_date = date_object
print '=' * 35
print len(intervals), 'intervals evaluated'
print '=' * 35
return intervals
def find_delays(intervals):
cnt = 0
for key in sorted(intervals.keys()):
if intervals[key] > hour_max:
cnt = cnt + 1
print 'time interval exceeded >', key, intervals[key]
else:
print key, intervals[key]
print '=' * 35
print cnt, 'time intervals exceeded'
print '=' * 35
log_dict()
find_delays(intervals)
Few comments to the above code, first. The line 22: intervals[date_object] = {}
It seems to me that there is a kind of confusion in initialization. You always reasign that value by something else. When you need "some" value that will be overwritten anyway, the None is good candidate. When using an empty dictionary, it also works, but it is more costly (processor time, memory) which may play the role in complex tasks.
The lines 27 to 29:
for date_object in intervals.keys():
intervals[date_object] = date_object-last_date
last_date = date_object
You probably wanted to do it after the loop that reads the log lines -- i.e. one-level less indentation.
Also, the loop does not make sense when the last_date is not initialized just before the loop (if undented) and if the keys are not sorted. If this is the case then also lines 24 and 25 make no sense.
I have something to do now. I will write more later.
It seems to me that there is a kind of confusion in initialization. You always reasign that value by something else. When you need "some" value that will be overwritten anyway, the None is good candidate. When using an empty dictionary, it also works, but it is more costly (processor time, memory) which may play the role in complex tasks.
The lines 27 to 29:
for date_object in intervals.keys():
intervals[date_object] = date_object-last_date
last_date = date_object
You probably wanted to do it after the loop that reads the log lines -- i.e. one-level less indentation.
Also, the loop does not make sense when the last_date is not initialized just before the loop (if undented) and if the keys are not sorted. If this is the case then also lines 24 and 25 make no sense.
I have something to do now. I will write more later.
I want to share my own experience with programming. Explicitly, do not consider this as any kind of offense against anyone, Sara ;) (I do not think that you are that kind of person, but I wanted this to be emphasized.)
Writing a program means to create the mental picture of the problem being solved. Then the mental picture must be translated into description in the chosen programming language. (Various programming languages may be more or less suitable for capturing/transformation of the mental picture.)
In every case, the capability to create a mental picture of the problem is essential for any programmer. It does not mean only to know what should be solved, but also to know how it should be solved. The mental picture develops in your head when you think about the solution.
Having the solution (unknown to you) written in a programming language, you can rebuild the mental picture in your head. This way, you can copy the mental picture from the author's head to your own via studying the source code. This explains why also the passive knowledge of the programming language--i.e. the ability to read and understand the constructions--may help you to learn how to program.
The key to building the mental picture in your head is imagination, including the dynamics. What I personally consider a really good way to improve your abilities or even to start thinking about the solution is to copy the reality. The mental picture should actually simulate the reality. And the good way to start with simulation of the reality is to think in terms: "How would I solve the problem by hand, with paper and pencil."
Having the printed log and enough time, you probably know what to look for and how to find the important things in it. Actually, starting to do that manually may easily reveal what you must consider in the programmatic solution. The key observation is that you can change the mental picture much faster that the formally written source code. Because of that, one should never star with typing "import...". Always and always... think first.
You also have to use your empathy. Try to become someone else. Try to behave as the (benevolent) boss who formulates what should be done. The key is to find the correct questions and to judge them to discover whether the task is meaninful at the highest level of abstraction. It makes no sense to search for details if the task is one big nonsense.
Now my questions/assuptions about the solved problem:
The two tests are a kind of independent, but it may be good idea to solve them at once. Please, add your comments.
Writing a program means to create the mental picture of the problem being solved. Then the mental picture must be translated into description in the chosen programming language. (Various programming languages may be more or less suitable for capturing/transformation of the mental picture.)
In every case, the capability to create a mental picture of the problem is essential for any programmer. It does not mean only to know what should be solved, but also to know how it should be solved. The mental picture develops in your head when you think about the solution.
Having the solution (unknown to you) written in a programming language, you can rebuild the mental picture in your head. This way, you can copy the mental picture from the author's head to your own via studying the source code. This explains why also the passive knowledge of the programming language--i.e. the ability to read and understand the constructions--may help you to learn how to program.
The key to building the mental picture in your head is imagination, including the dynamics. What I personally consider a really good way to improve your abilities or even to start thinking about the solution is to copy the reality. The mental picture should actually simulate the reality. And the good way to start with simulation of the reality is to think in terms: "How would I solve the problem by hand, with paper and pencil."
Having the printed log and enough time, you probably know what to look for and how to find the important things in it. Actually, starting to do that manually may easily reveal what you must consider in the programmatic solution. The key observation is that you can change the mental picture much faster that the formally written source code. Because of that, one should never star with typing "import...". Always and always... think first.
You also have to use your empathy. Try to become someone else. Try to behave as the (benevolent) boss who formulates what should be done. The key is to find the correct questions and to judge them to discover whether the task is meaninful at the highest level of abstraction. It makes no sense to search for details if the task is one big nonsense.
Now my questions/assuptions about the solved problem:
1
The first part of the task is to find, whether the ping was called each hour as expected. If no, the too long interval should be reported.2
During one test (i.e. for one Timestamp: record), you want to detect if some delays are not too big.3
Is the target always the yahoo.com, or do you want to collect information separately for more machines?The two tests are a kind of independent, but it may be good idea to solve them at once. Please, add your comments.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks very much, that did it! I was able to take your code and make changes that don't extend the size of the script but they still work - I'll end up using yours but I wanted to make sure that I can adapt these functions to dict objects.
The big challenge for me always is to understand what happens inside a loop and what doesn't - as a colleague once said, I have flow control issues. We also have code floating around the office that was written by people long gone who were learning as they wrote it. I've picked up on those scripts so I have some things to unlearn.
I would have answered sooner but my Internet died at home yesterday. It dies every year around break-up when the ice melts and at other times in between; there's no real competition in the ISP business in Interior Alaska. It's not like I live in Deadhorse, but it sure does feel like I'm at the end of the line :(
The big challenge for me always is to understand what happens inside a loop and what doesn't - as a colleague once said, I have flow control issues. We also have code floating around the office that was written by people long gone who were learning as they wrote it. I've picked up on those scripts so I have some things to unlearn.
I would have answered sooner but my Internet died at home yesterday. It dies every year around break-up when the ice melts and at other times in between; there's no real competition in the ISP business in Interior Alaska. It's not like I live in Deadhorse, but it sure does feel like I'm at the end of the line :(
ASKER
That should do, thank again :)
Thanks for your "thanks" ;)
As far as I can say, your usage of the dict instead of the list works. The problem is (in the context of the the above comment related to the general programmer-thinking) that switching to the mental picture of using a dictionary complicates the things and changes the approach to the solution.
KISS: Keep It Small and Stupid (some say Simple instead) -- about the design and programs to be created.
As far as I can say, your usage of the dict instead of the list works. The problem is (in the context of the the above comment related to the general programmer-thinking) that switching to the mental picture of using a dictionary complicates the things and changes the approach to the solution.
KISS: Keep It Small and Stupid (some say Simple instead) -- about the design and programs to be created.
Q: How are "down times" represented?
Q: Are all records in the input file to the same host?