Automated Archive and Purge Tool for a data logging application.

I need to be able to archive all records older than a year, but be able to restore the archived records if required for inspection at a later date.
(The records are a history of alarms, and plant values form a production process)
The logging application uses MS SQL as the database.

Aged 'archived blocks' should be organised in 1 year chunks for easy selection.
What would be the best (and easiest way to implement this). This includes any special applications already available for this type of task.

Thanks fo now,

Who is Participating?
Guy Hengel [angelIII / a3]Connect With a Mentor Billing EngineerCommented:
I would suggest the following:
For each Year, create a table (maybe in another dedicated database), with the name of the (new) table having the year postfixed. If you later decided to have chunks of 1 month, the process will not change that much

To "archive" to that table, you simply insert all the relevant records to that table, and then delete them in the actual table, both instructions in a proper transaction with error-checking/handling...

Once you have the records in a table, you can do with that table whatever you want, for example DTS as indicated already above. The advantage of having 1 dedicated table is the fact that you can playback the records to this table instead directly to the main table, thus avoiding to clutter the main table unnecessary.

To automate this process, it will be a bit more tricky, but if you are used to DTS, you know for sure how to work with dynamic properties. The years, tablenames and filenames should be configured dynamically, so you can call the DTS package(s) with the Year as parameter and don't need to reconfigure it every year...

I think the easiet way to do this is as follows
Use a DTS Package to export all records from the tables into flat  files (text file) comma delimeted. using the win zip command line utility zip the text files and move them into an archive folder.

to re import them just use another DTS package to import the unziped comma delimited files back into the database

This way the archive files are portable and not part of sql server and can be moved off site for safe keeping

Hope this helps
All Courses

From novice to tech pro — start learning today.