Solved

python script to read the timestamp of a file and print it

Posted on 2009-04-06
10
2,261 Views
Last Modified: 2012-05-06
I have a python script that iterates through a list of stations, reads in a raw data file for each station and prints the latest timestamp of the file on a diagnostics page.  That works fine, but now we have a station whose raw data file isn't in the default format that the script uses to read in the data.  
We could; a) write a script to reformat that station's raw data or b) read in the unix timestamp of the raw data file and print it to the diagnostics page.
It seems that option b would be easier, since we don't need to plot the data as we do for the other stations.

Is there a way to print the unix timestamp of a file with a known path/filename to the table below?

TIA


#excerpts of stationDiag.kid
 
<html xmlns:py="http://purl.org/kid/ns#">
<?python
import time
currentTime = time.strftime('%Y-%m-%d %H:%M')
?>
<div py:for="baseStation in baseStationList">  //station list is read from a csv file
<tr><td colspan="7">
  <div align="center">
  ${baseStationInfo[baseStation]['baseStationName']}
  </div></td></tr>
<tr py:for="station in baseStationInfo[baseStation]['stationOrder']">
 <td> 
 <div py:if="diag[station]['StationName'] == 'MetStation'"><a href="/path/metstation.html"> ${diag[station]['StationName']} -  ${diag[station]['StationDescriptiveName']}</a></div>
 
<?python
# Set the color based on the time diff
if 'timeDiff' in diag[station]:
    days_value = diag[station]['timeDiff']
    if days_value > 5:
        days_color = '#FF0000'
    elif days_value > 1:
        days_color = '#FF9999'
    elif days_value > 0.25:
        days_color = '#FFCC00'
    else:
        days_color = '#00FF00'
    days_value = '%.2f' % diag[station]['timeDiff']
else:
    days_color = ''
    days_value = 'N/A'
?>
<td bgcolor="${days_color}">${days_value}</td>
 
#The new station is omitted from the station list due to the different data file format 
# so this won't work:
<td>${diag[station]['timeStamp']}</td>
<td bgcolor="${days_color}">${days_value}</td>

Open in new window

0
Comment
Question by:sara_bellum
  • 5
  • 4
10 Comments
 
LVL 9

Expert Comment

by:ghostdog74
ID: 24084132
you can use os.path.getmtime, os.path.getctime, os.path.getatime to get timestamps of files.
0
 
LVL 28

Expert Comment

by:pepr
ID: 24085070
The above functions return time in seconds since the epoch. Then you can convert the time to the time.struct_time (http://docs.python.org/library/time.html#time.struct_time) via time.gmtime() (http://docs.python.org/library/time.html#time.gmtime) or time.localtime() (http://docs.python.org/library/time.html#time.localtime). And then use time.strftime() function to format the result into a string (see http://docs.python.org/library/time.html#time.strftime). There also is an example how to form the "standard timestamp".
0
 
LVL 28

Accepted Solution

by:
pepr earned 500 total points
ID: 24085505
To be more concrete...
import os
import time
 
def filestamp(fname):
    fileTime = os.path.getmtime(fname)
    st = time.gmtime(fileTime)
    return time.strftime('%a, %d %b %Y %H:%M:%S +0000', st)
    
 
if __name__ == '__main__':
    print filestamp('a.txt')    
    
    # or the same one-liner
    print time.strftime('%a, %d %b %Y %H:%M:%S +0000', time.gmtime(os.path.getmtime('a.txt')))
 
    

Open in new window

0
Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

 
LVL 28

Expert Comment

by:pepr
ID: 24085521
Well, you want the '%Y-%m-%d %H:%M' format string.
0
 

Author Comment

by:sara_bellum
ID: 24093035
Thanks very much!! That was very helpful, in particular since I was getting runtime errors on my Linux client with a couple of other scripts that ostensibly do the same thing.  It's always better to start with a known/proven script - so I ran yours on the server and all was ok.

I tried executing your fname function (enclosed in python brackets) directly in the kid page but got errors:
"exec is not allowed in function '_pull' it contains a nested function with free variables"

I checked the python script that generates the kid page to see what I need to make your script work with it.  I copy the lines that concern me below, with comments.

You will see that gp.py reads a config file to iterate through the stations, and as I mentioned, the 2 stations with non-standard data must be omitted from the config file.  It may be easier to import your script (which I named timefiles.py) into gp.py than it will be to call it directly from stationDiag.kid.  But I still don't know how to call the filestamp function  inside the kid page - the variable names ($MTL, $PVM) can't be printed to the html table unless the function is called.

As I mentioned earlier, the diag.kid page calls the functions from gp.py in this way:
<tr py:for="station in baseStationInfo[baseStation]['stationOrder']">
<td><div py:if "diag[station]['StationName'] == 'some name">
put something in the table, else don't print anything








# the gp.py script begins with the processStation function: 
 
def processStation(station, stationInfo, gpConfig, stationPage, stationInclude):
    """
    Takes the station name, its info dict, a ConfigParser object, and the
    stationPage and stationInclude kid modules, gets the data for the station, and renders the page(s) for that station.  Returns a dict with diagnostic information to be used for the diagnostics page.
    """
    import os
    import datetime
    import time
    import pytz
    import gpUtil
# rest of function omitted
 
# I add your function to get timestamps of non-standard data files
 
def filestamp(fname):
 
    import os
    import time
    import timefiles
 
    fileTime = os.path.getmtime(fname)
    st = time.gmtime(fileTime)
    return time.strftime('%Y-%m-%d %H:%M')
 
    MTL = filestamp('/path/storage.dat')
    PVM = filestamp('/path/diag.dat')
 
# several def's omitted...
 
def main():
 
    import cPickle as cp
    import kid
    import sys
    import ConfigParser
    import os
    import time
    import gpUtil
 
try:
        configFile = sys.argv[1]
    except IndexError:
        print "Needs Configuration File"
        sys.exit(1)
 
# Have to do this after we read the config file
    kid.enable_import(path = gpConfig.get('Main', 'templateDir'))
 
    # And then that allows us to directly import the template
    import stationPage
    import stationDiag
 
# Code follows on what gets printed to what kid page, what gets emailed etc...
 
# gp.py ends with this:
 
if __name__ == "__main__":
    main()

Open in new window

0
 
LVL 28

Assisted Solution

by:pepr
pepr earned 500 total points
ID: 24094830
I am not sure if I understand details. Anyway, it is usual to put the import commands to the beginning of the module. It works also when you place them into a function definition body; however, it is always imported only when it was not imported earlier. This way I suggest to put normal imports to the beginning (sorted by alphabet is good sometimes, or grouping by meaning is also the choice). I am not sure about

    kid.enable_import(path = gpConfig.get('Main', 'templateDir'))
 
    # And then that allows us to directly import the template
    import stationPage
    import stationDiag

Anyway, if you cannot put the definition of filestamp() function directly to the template, you can "import timefiles" and call timefiles.filestamp().

I do not know your details. The "free variable" means that there is a variable that is used inside the code block but is not defined in the block -- see http://docs.python.org/reference/executionmodel.html#naming-and-binding for the details. You can avoid having free variables if you pass them via function arguments.
# the gp.py script begins with the processStation function: 
 
import ConfigParser
import cPickle as cp
import datetime
import gpUtil
import kid
import os
import pytz
import sys
import time
 
 
def processStation(station, stationInfo, gpConfig, stationPage, stationInclude):
    """
    Takes the station name, its info dict, a ConfigParser object, and the
    stationPage and stationInclude kid modules, gets the data for the station, and renders the page(s) for that station.  Returns a dict with diagnostic information to be used for the diagnostics page.
    """
# rest of function omitted
 
# I add your function to get timestamps of non-standard data files
 
def filestamp(fname):
 
    fileTime = os.path.getmtime(fname)
    st = time.gmtime(fileTime)
    return time.strftime('%Y-%m-%d %H:%M')
 
 
# several def's omitted...
 
def main():
 
    MTL = filestamp('/path/storage.dat')
    PVM = filestamp('/path/diag.dat')
    # or you can 
    import timefiles
    MTL = timefiles.filestamp('/path/storage.dat')
    PVM = timefiles.filestamp('/path/diag.dat')
 
    try:
        configFile = sys.argv[1]
    except IndexError:
        print "Needs Configuration File"
        sys.exit(1)
 
# Have to do this after we read the config file
    kid.enable_import(path = gpConfig.get('Main', 'templateDir'))
 
    # And then that allows us to directly import the template
    import stationPage
    import stationDiag
 
# Code follows on what gets printed to what kid page, what gets emailed etc...
 
# gp.py ends with this:
 
if __name__ == "__main__":
    main()

Open in new window

0
 

Author Comment

by:sara_bellum
ID: 24103408
Thanks very much, I've learned a lot (I hope ;-)
Your script ran perfectly (both timestamp options) from my home folder on the server with the path pointing to files located in my home folder.

I copied the script to the user folder that should run the script (the files that I need timestamps for are also owned by that user) and I got inconsistent results:
if the timestamp of either of the two files was within the same hour as the current date and time, filestamp printed the current time instead - for both files.
if the timestamp of one of the files was within the same hour as the current date and time, strftime printed a future timestamp (midnight 9 April) for that file
if the timestamp of one of the files was several days old, strftime printed the correct date, but the hour was 8 hours ahead of the actual timestamp of the file.

I realize that these are not problems associated with your answer and I'm prepared to close this question and award points, but I'd be curious to know if you have any idea what causes such inconsistencies.  
0
 

Author Comment

by:sara_bellum
ID: 24103424
I should add that I was running the script as that user (from the user /home/user/bin directory) when the errors appeared, and compared the print statements with ls -l output for the files concerned.  There are several other python scripts in that directory so it should have run ok :(
0
 
LVL 28

Assisted Solution

by:pepr
pepr earned 500 total points
ID: 24113793
Because of the time gap, I am not an expert for Unix-like system any more. Try also to use os.path.getctime() instead of os.path.getmtime(). The doc says (http://docs.python.org/library/os.path.html?highlight=os.path#os.path.getctime):

    "Return the systems ctime which, on some systems (like Unix) is the time of the last change,
     and, on others (like Windows), is the creation time for path. [...]"
0
 

Author Closing Comment

by:sara_bellum
ID: 31567337
I've been out of town for the past 10 days - sorry for my delay.  Your responses have been very comprehensive, thank you very much!
0

Featured Post

Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The really strange introduction Once upon a time there were individuals who intentionally put the grass seeds to the soil with anticipation of solving their nutrition problems. Or they maybe only played with seeds and noticed what happened... Som…
"The time has come," the Walrus said, "To talk of many things: Of sets--and lists--and dictionaries-- Of variable kinks-- And why you see it changing not-- And why so strange are strings." This part describes how variables and references (see …
Learn the basics of lists in Python. Lists, as their name suggests, are a means for ordering and storing values. : Lists are declared using brackets; for example: t = [1, 2, 3]: Lists may contain a mix of data types; for example: t = ['string', 1, T…
Learn the basics of modules and packages in Python. Every Python file is a module, ending in the suffix: .py: Modules are a collection of functions and variables.: Packages are a collection of modules.: Module functions and variables are accessed us…

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now