sara_bellum
asked on
python script to read the timestamp of a file and print it
I have a python script that iterates through a list of stations, reads in a raw data file for each station and prints the latest timestamp of the file on a diagnostics page. That works fine, but now we have a station whose raw data file isn't in the default format that the script uses to read in the data.
We could; a) write a script to reformat that station's raw data or b) read in the unix timestamp of the raw data file and print it to the diagnostics page.
It seems that option b would be easier, since we don't need to plot the data as we do for the other stations.
Is there a way to print the unix timestamp of a file with a known path/filename to the table below?
TIA
We could; a) write a script to reformat that station's raw data or b) read in the unix timestamp of the raw data file and print it to the diagnostics page.
It seems that option b would be easier, since we don't need to plot the data as we do for the other stations.
Is there a way to print the unix timestamp of a file with a known path/filename to the table below?
TIA
#excerpts of stationDiag.kid
<html xmlns:py="http://purl.org/kid/ns#">
<?python
import time
currentTime = time.strftime('%Y-%m-%d %H:%M')
?>
<div py:for="baseStation in baseStationList"> //station list is read from a csv file
<tr><td colspan="7">
<div align="center">
${baseStationInfo[baseStation]['baseStationName']}
</div></td></tr>
<tr py:for="station in baseStationInfo[baseStation]['stationOrder']">
<td>
<div py:if="diag[station]['StationName'] == 'MetStation'"><a href="/path/metstation.html"> ${diag[station]['StationName']} - ${diag[station]['StationDescriptiveName']}</a></div>
<?python
# Set the color based on the time diff
if 'timeDiff' in diag[station]:
days_value = diag[station]['timeDiff']
if days_value > 5:
days_color = '#FF0000'
elif days_value > 1:
days_color = '#FF9999'
elif days_value > 0.25:
days_color = '#FFCC00'
else:
days_color = '#00FF00'
days_value = '%.2f' % diag[station]['timeDiff']
else:
days_color = ''
days_value = 'N/A'
?>
<td bgcolor="${days_color}">${days_value}</td>
#The new station is omitted from the station list due to the different data file format
# so this won't work:
<td>${diag[station]['timeStamp']}</td>
<td bgcolor="${days_color}">${days_value}</td>
you can use os.path.getmtime, os.path.getctime, os.path.getatime to get timestamps of files.
The above functions return time in seconds since the epoch. Then you can convert the time to the time.struct_time (http://docs.python.org/library/time.html#time.struct_time) via time.gmtime() (http://docs.python.org/library/time.html#time.gmtime) or time.localtime() (http://docs.python.org/library/time.html#time.localtime). And then use time.strftime() function to format the result into a string (see http://docs.python.org/library/time.html#time.strftime). There also is an example how to form the "standard timestamp".
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Well, you want the '%Y-%m-%d %H:%M' format string.
ASKER
Thanks very much!! That was very helpful, in particular since I was getting runtime errors on my Linux client with a couple of other scripts that ostensibly do the same thing. It's always better to start with a known/proven script - so I ran yours on the server and all was ok.
I tried executing your fname function (enclosed in python brackets) directly in the kid page but got errors:
"exec is not allowed in function '_pull' it contains a nested function with free variables"
I checked the python script that generates the kid page to see what I need to make your script work with it. I copy the lines that concern me below, with comments.
You will see that gp.py reads a config file to iterate through the stations, and as I mentioned, the 2 stations with non-standard data must be omitted from the config file. It may be easier to import your script (which I named timefiles.py) into gp.py than it will be to call it directly from stationDiag.kid. But I still don't know how to call the filestamp function inside the kid page - the variable names ($MTL, $PVM) can't be printed to the html table unless the function is called.
As I mentioned earlier, the diag.kid page calls the functions from gp.py in this way:
<tr py:for="station in baseStationInfo[baseStatio n]['statio nOrder']">
<td><div py:if "diag[station]['StationNam e'] == 'some name">
put something in the table, else don't print anything
I tried executing your fname function (enclosed in python brackets) directly in the kid page but got errors:
"exec is not allowed in function '_pull' it contains a nested function with free variables"
I checked the python script that generates the kid page to see what I need to make your script work with it. I copy the lines that concern me below, with comments.
You will see that gp.py reads a config file to iterate through the stations, and as I mentioned, the 2 stations with non-standard data must be omitted from the config file. It may be easier to import your script (which I named timefiles.py) into gp.py than it will be to call it directly from stationDiag.kid. But I still don't know how to call the filestamp function inside the kid page - the variable names ($MTL, $PVM) can't be printed to the html table unless the function is called.
As I mentioned earlier, the diag.kid page calls the functions from gp.py in this way:
<tr py:for="station in baseStationInfo[baseStatio
<td><div py:if "diag[station]['StationNam
put something in the table, else don't print anything
# the gp.py script begins with the processStation function:
def processStation(station, stationInfo, gpConfig, stationPage, stationInclude):
"""
Takes the station name, its info dict, a ConfigParser object, and the
stationPage and stationInclude kid modules, gets the data for the station, and renders the page(s) for that station. Returns a dict with diagnostic information to be used for the diagnostics page.
"""
import os
import datetime
import time
import pytz
import gpUtil
# rest of function omitted
# I add your function to get timestamps of non-standard data files
def filestamp(fname):
import os
import time
import timefiles
fileTime = os.path.getmtime(fname)
st = time.gmtime(fileTime)
return time.strftime('%Y-%m-%d %H:%M')
MTL = filestamp('/path/storage.dat')
PVM = filestamp('/path/diag.dat')
# several def's omitted...
def main():
import cPickle as cp
import kid
import sys
import ConfigParser
import os
import time
import gpUtil
try:
configFile = sys.argv[1]
except IndexError:
print "Needs Configuration File"
sys.exit(1)
# Have to do this after we read the config file
kid.enable_import(path = gpConfig.get('Main', 'templateDir'))
# And then that allows us to directly import the template
import stationPage
import stationDiag
# Code follows on what gets printed to what kid page, what gets emailed etc...
# gp.py ends with this:
if __name__ == "__main__":
main()
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks very much, I've learned a lot (I hope ;-)
Your script ran perfectly (both timestamp options) from my home folder on the server with the path pointing to files located in my home folder.
I copied the script to the user folder that should run the script (the files that I need timestamps for are also owned by that user) and I got inconsistent results:
if the timestamp of either of the two files was within the same hour as the current date and time, filestamp printed the current time instead - for both files.
if the timestamp of one of the files was within the same hour as the current date and time, strftime printed a future timestamp (midnight 9 April) for that file
if the timestamp of one of the files was several days old, strftime printed the correct date, but the hour was 8 hours ahead of the actual timestamp of the file.
I realize that these are not problems associated with your answer and I'm prepared to close this question and award points, but I'd be curious to know if you have any idea what causes such inconsistencies.
Your script ran perfectly (both timestamp options) from my home folder on the server with the path pointing to files located in my home folder.
I copied the script to the user folder that should run the script (the files that I need timestamps for are also owned by that user) and I got inconsistent results:
if the timestamp of either of the two files was within the same hour as the current date and time, filestamp printed the current time instead - for both files.
if the timestamp of one of the files was within the same hour as the current date and time, strftime printed a future timestamp (midnight 9 April) for that file
if the timestamp of one of the files was several days old, strftime printed the correct date, but the hour was 8 hours ahead of the actual timestamp of the file.
I realize that these are not problems associated with your answer and I'm prepared to close this question and award points, but I'd be curious to know if you have any idea what causes such inconsistencies.
ASKER
I should add that I was running the script as that user (from the user /home/user/bin directory) when the errors appeared, and compared the print statements with ls -l output for the files concerned. There are several other python scripts in that directory so it should have run ok :(
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I've been out of town for the past 10 days - sorry for my delay. Your responses have been very comprehensive, thank you very much!