Link to home
Start Free TrialLog in
Avatar of sara_bellum
sara_bellumFlag for United States of America

asked on

python script to read the timestamp of a file and print it

I have a python script that iterates through a list of stations, reads in a raw data file for each station and prints the latest timestamp of the file on a diagnostics page.  That works fine, but now we have a station whose raw data file isn't in the default format that the script uses to read in the data.  
We could; a) write a script to reformat that station's raw data or b) read in the unix timestamp of the raw data file and print it to the diagnostics page.
It seems that option b would be easier, since we don't need to plot the data as we do for the other stations.

Is there a way to print the unix timestamp of a file with a known path/filename to the table below?

TIA


#excerpts of stationDiag.kid
 
<html xmlns:py="http://purl.org/kid/ns#">
<?python
import time
currentTime = time.strftime('%Y-%m-%d %H:%M')
?>
<div py:for="baseStation in baseStationList">  //station list is read from a csv file
<tr><td colspan="7">
  <div align="center">
  ${baseStationInfo[baseStation]['baseStationName']}
  </div></td></tr>
<tr py:for="station in baseStationInfo[baseStation]['stationOrder']">
 <td> 
 <div py:if="diag[station]['StationName'] == 'MetStation'"><a href="/path/metstation.html"> ${diag[station]['StationName']} -  ${diag[station]['StationDescriptiveName']}</a></div>
 
<?python
# Set the color based on the time diff
if 'timeDiff' in diag[station]:
    days_value = diag[station]['timeDiff']
    if days_value > 5:
        days_color = '#FF0000'
    elif days_value > 1:
        days_color = '#FF9999'
    elif days_value > 0.25:
        days_color = '#FFCC00'
    else:
        days_color = '#00FF00'
    days_value = '%.2f' % diag[station]['timeDiff']
else:
    days_color = ''
    days_value = 'N/A'
?>
<td bgcolor="${days_color}">${days_value}</td>
 
#The new station is omitted from the station list due to the different data file format 
# so this won't work:
<td>${diag[station]['timeStamp']}</td>
<td bgcolor="${days_color}">${days_value}</td>

Open in new window

Avatar of ghostdog74
ghostdog74

you can use os.path.getmtime, os.path.getctime, os.path.getatime to get timestamps of files.
Avatar of pepr
The above functions return time in seconds since the epoch. Then you can convert the time to the time.struct_time (http://docs.python.org/library/time.html#time.struct_time) via time.gmtime() (http://docs.python.org/library/time.html#time.gmtime) or time.localtime() (http://docs.python.org/library/time.html#time.localtime). And then use time.strftime() function to format the result into a string (see http://docs.python.org/library/time.html#time.strftime). There also is an example how to form the "standard timestamp".
ASKER CERTIFIED SOLUTION
Avatar of pepr
pepr

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Well, you want the '%Y-%m-%d %H:%M' format string.
Avatar of sara_bellum

ASKER

Thanks very much!! That was very helpful, in particular since I was getting runtime errors on my Linux client with a couple of other scripts that ostensibly do the same thing.  It's always better to start with a known/proven script - so I ran yours on the server and all was ok.

I tried executing your fname function (enclosed in python brackets) directly in the kid page but got errors:
"exec is not allowed in function '_pull' it contains a nested function with free variables"

I checked the python script that generates the kid page to see what I need to make your script work with it.  I copy the lines that concern me below, with comments.

You will see that gp.py reads a config file to iterate through the stations, and as I mentioned, the 2 stations with non-standard data must be omitted from the config file.  It may be easier to import your script (which I named timefiles.py) into gp.py than it will be to call it directly from stationDiag.kid.  But I still don't know how to call the filestamp function  inside the kid page - the variable names ($MTL, $PVM) can't be printed to the html table unless the function is called.

As I mentioned earlier, the diag.kid page calls the functions from gp.py in this way:
<tr py:for="station in baseStationInfo[baseStation]['stationOrder']">
<td><div py:if "diag[station]['StationName'] == 'some name">
put something in the table, else don't print anything








# the gp.py script begins with the processStation function: 
 
def processStation(station, stationInfo, gpConfig, stationPage, stationInclude):
    """
    Takes the station name, its info dict, a ConfigParser object, and the
    stationPage and stationInclude kid modules, gets the data for the station, and renders the page(s) for that station.  Returns a dict with diagnostic information to be used for the diagnostics page.
    """
    import os
    import datetime
    import time
    import pytz
    import gpUtil
# rest of function omitted
 
# I add your function to get timestamps of non-standard data files
 
def filestamp(fname):
 
    import os
    import time
    import timefiles
 
    fileTime = os.path.getmtime(fname)
    st = time.gmtime(fileTime)
    return time.strftime('%Y-%m-%d %H:%M')
 
    MTL = filestamp('/path/storage.dat')
    PVM = filestamp('/path/diag.dat')
 
# several def's omitted...
 
def main():
 
    import cPickle as cp
    import kid
    import sys
    import ConfigParser
    import os
    import time
    import gpUtil
 
try:
        configFile = sys.argv[1]
    except IndexError:
        print "Needs Configuration File"
        sys.exit(1)
 
# Have to do this after we read the config file
    kid.enable_import(path = gpConfig.get('Main', 'templateDir'))
 
    # And then that allows us to directly import the template
    import stationPage
    import stationDiag
 
# Code follows on what gets printed to what kid page, what gets emailed etc...
 
# gp.py ends with this:
 
if __name__ == "__main__":
    main()

Open in new window

SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks very much, I've learned a lot (I hope ;-)
Your script ran perfectly (both timestamp options) from my home folder on the server with the path pointing to files located in my home folder.

I copied the script to the user folder that should run the script (the files that I need timestamps for are also owned by that user) and I got inconsistent results:
if the timestamp of either of the two files was within the same hour as the current date and time, filestamp printed the current time instead - for both files.
if the timestamp of one of the files was within the same hour as the current date and time, strftime printed a future timestamp (midnight 9 April) for that file
if the timestamp of one of the files was several days old, strftime printed the correct date, but the hour was 8 hours ahead of the actual timestamp of the file.

I realize that these are not problems associated with your answer and I'm prepared to close this question and award points, but I'd be curious to know if you have any idea what causes such inconsistencies.  
I should add that I was running the script as that user (from the user /home/user/bin directory) when the errors appeared, and compared the print statements with ls -l output for the files concerned.  There are several other python scripts in that directory so it should have run ok :(
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I've been out of town for the past 10 days - sorry for my delay.  Your responses have been very comprehensive, thank you very much!