File Archiving by Unix Timestamp

Oh, and for the sake of completeness, you can add that as a shell script to your crontab to run every month as described under http://en.wikipedia.org/wiki/Crontab like

0 0 1 * * /Monitor/gsmbackup.sh

if you save

#!/bin/bash
find /Monitor -name "*.gsm" -mtime 180 -exec mv {} /OldMonitor \;

as /Monitor/gsmbackup.sh

Does mtime just show you the time when it was written/changed/accessed therefore will not do what I need as I access these files on a daily basis.

As I said the key is in the name of the file. The name of the file has its unix timestamp. As when a new file is written it is called (unixtimestamp).gsm the written/changed/accessed data and time would be different would it not if I accessed a file today writting on the 10th or Dec 2006 it would not have todays date as I have accessed it.

If this is not the case let me know but I think for this to be reliable it needs to be done by the file name which is also the unix timestamp.

Hm, IMO it is easier for find to work with that. 'Last acc4essed' here means 'last written to'.

Here is a little script in python which would hopefully accomplish what you are looking for. It basically reads in all the files in the directory, compare first part of the filename with a calculated 6 months old epoch timestamp and moves files older than that. Very basic but does at least do some basic checking like if it has permission to perform operations on files and such.

---

#! /usr/bin/env python
#

import sys, os, time, calendar, shutil

CheckDir='/Monitor'
MoveDir='/OldMonitor'

# Calculate the time threshold

year,month,day,hour,min,sec,wday,yday,isdst = time.localtime()

if month < 7:
    month = 12 + (month - 6)
    year  = year - 1
else:
    month = month - 6

timetreshold = int(calendar.timegm((year,month,day,hour,min,sec,wday,yday,isdst)))

# Scan dir for files

if os.access(CheckDir,os.W_OK) & os.access(MoveDir,os.W_OK):
    moves = 0
    for root,dirs,files in os.walk(CheckDir):
        for f in files:
            stamp = f[:10]
            try:
                if int(stamp) <= timetreshold:
                    shutil.move(os.path.join(root,f),os.path.join(MoveDir,f))
                    moves = moves + 1
            except ValueError:
                continue      
else:
    print 'error: No write access to directories'
    sys.exit(1)

print 'Archiving done, ' + str(moves) + ' file(s) moved'

---

Well, I was bored and since there's always room for improvement, I made some modifications to the script. The old did have some issues with hardcoded paths and the fact that the timestamp had to be exactly 10 numbers in length, so this new version should work better. When run, you need to specify the input directory (-i option) and output directory (-o option) to indicate where to look for files and where to move them, for instance:

# ./gsmarchiver -i /Monitor -o /OLDMonitor

Use -q to disable output completely or -v to have detailed listings of what is happening. You can also use the -p option to simulate the program running but without actually performing any moves.

If running is successfully, it will exit with an exitcode of 0. Any errors would produce an exitcode of 1

---

#! /usr/bin/env python
#

import sys, os, time, calendar, shutil, optparse

def ProcessCmdopts():
    """ Process command options and check for errors """

    cmdparser = optparse.OptionParser(usage='%prog [options] -i DIR -o DIR')
    cmdparser.set_defaults(pretending=False)
    cmdparser.add_option('-p', '--pretend', dest='pretending', action='store_true', help='Only simulate actions taken')
    cmdparser.add_option('-v', '--verbose', dest='verbosity', action='store_const', const=2, help='Make more output noise')
    cmdparser.add_option('-q', '--quiet', dest='verbosity', action='store_const', const=0, help='Disable output')
    cmdparser.add_option('-i', '--indir', dest='indir', metavar='DIR', help='Process files in DIR')
    cmdparser.add_option('-o', '--outdir', dest='outdir', metavar='DIR', help="Move files to DIR")
    (options, args) = cmdparser.parse_args()

    if not options.verbosity:
        options.verbosity = 1

    return options
   

def GetThreshold():
    """ Calculate POSIX epoch timestamp 6 months back in time """

    (dyear,dmonth,dday,dhour,dmin,dsec,dwday,dyday,disdst) = time.localtime()

    if dmonth < 7:
        dmonth = 12 + (dmonth - 6)
        dyear  = dyear - 1
    else:
        dmonth = dmonth - 6

    threshold = int(calendar.timegm((dyear,dmonth,dday,dhour,dmin,dsec,dwday,dyday,disdst)))

    return threshold
   

def CheckFiles(indir, outdir, files, verbosity, pretending, threshold):
    """
    Check if files are below threshold and moves them from indir to
    outdir.
    """

    processed=0
    skipped=0
    moved=0
    print '\n'
   
    for f in files:
        processed += 1
        try:
            stamp = int(f[:f.find('.')])
        except ValueError:
            if verbosity == 2:
                print 'Skipping : %s - not a valid timestamp' % (os.path.join(indir,f))
            skipped += 1
        else:
            if stamp <= threshold:
                try:
                    if not pretending:
                        shutil.move(os.path.join(indir,f),os.path.join(outdir,f))
                except IOError:
                    if verbosity:
                        print 'Error    : Unable to move %s, aborting' % (os.path.join(indir,f))
                    sys.exit(1)
                else:
                    if verbosity == 2:
                        print 'Moving   : %s => %s' % (os.path.join(indir,f),os.path.join(outdir,f))
                    moved += 1
            else:
                if verbosity == 2:
                    print 'Skipping : %s - not old enough' % (os.path.join(indir,f))
                skipped += 1

    print '\nArchiving completed'

    if not pretending:
        print '%i file(s) in directory, %i moved and %i skipped' % (processed, moved, skipped)
    else:
        print '%i file(s) in directory, %i would be moved and %i would be skipped' % (processed, moved, skipped)
       

def Main():
    options = ProcessCmdopts()

    if os.access(options.indir,os.W_OK) & os.access(options.outdir,os.W_OK):
        for root,dirs,files in os.walk(options.indir):
            CheckFiles(options.indir, options.outdir, files, options.verbosity,
                        options.pretending, GetThreshold())
    else:
        if options.verbosity:
            print 'Error    : No write access to directories'
        sys.exit(1)
   

if __name__ == '__main__':
    Main()

---