rsync: Daily, Weekly and Monthly backups

I want to use rsync to store daily, weekly and monthly backups of my /home/ directory on my production server to a backup server.

I want seven daily backups to be kept.  So every day, the oldest daily backup should be deleted and a new backup added.

I want five weekly backups to be kept.  So every week, the oldest weekly backup should be deleted and the newest daily backup should be kept as a weekly backup.

I want twelve monthly backups to be kept.  So every month, the oldest monthly backup should be deleted and the newest daily backup should be kept as a weekly monthly backup.

LVL 16
hankknightAsked:
Who is Participating?
 
arober11Commented:
Following on from _robthewolf_ suggestion:

First you need to generate a pass phrase less a [DR]SA key pair, to allow password less ssh / rsync access to the remote host.  e.g. type:

ssh-keygen -t rsa

and just keep hitting <enter>.

THen on the remote server create the following directory structure somewhere:
/backup/Mon/
/backup/Tue/
/backup/Wed/
...
/backup/Sun/

/backup/Week1/
/backup/Week2/
/backup/Week3/
/backup/Week4/
/backup/Week5/

/backup/Jan/
/backup/Feb/
/backup/Mar/
...
/backup/Dec/

Then on the local server add something like the following to your users crontab e.g.  

# BAckup $HOME to a remote server
# Daily backup
05 02 * * * /xxx/xxx/bin/rsync -optlgvr --delete-after --rsh=/usr/bin/ssh --rsync-path=/REMOTE_RSYNC_BIN_DIR/rsync --force "$HOME/"  "user@remote.host.com:/xxx/backup/`date '+\%b'`" > /tmp/backup_daily 2>&1
#
# Weekly backup
05 03 * * 1  /xxx/xxx/bin/rsync -optlgvr --delete-after --rsh=/usr/bin/ssh --rsync-path=/REMOTE_RSYNC_BIN_DIR/rsync --force "$HOME/"  "user@remote.host.com:/xxx/backup/Week$((`date '+\%V'` % 5))" > /tmp/backup_weekly 2>&1
#
# An alternative approach to do the same weekl backup:
#05 03 * * 1  dailyBackup="/xxx/backup/`date '+\%b'`";newBackup="/xxx/backup/Week$((`date '+\%V'` \% 5))"; ssh user@remote.host.com "rm -Rf $newBackup && cp -Rp $dailyBackup $newBackup"  > /tmp/backup_weekly 2>&1
#
# Monthly backup
05 04 1 * * /xxx/xxx/bin/rsync -optlgvr --delete-after --rsh=/usr/bin/ssh --rsync-path=/REMOTE_RSYNC_BIN_DIR/rsync --force "$HOME/"  "user@remote.host.com:/xxx/backup/`date '+\%a'`" > /tmp/backup_monthly 2>&1
0
 
IanThCommented:
there are a number of linux backup tools available are you sure you want to use rsync
0
 
hankknightAuthor Commented:
I am using an online service that only supports rsync.
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

 
IanThCommented:
the usual way is to use cron for scheduling
0
 
robthewolfCommented:
you put a line in cron for each different backup that you want.
one for each day of the week, one for the week and one for the month.
each line in the cron will write to the specific backup that you want.
For example you will have a line that backs up on a monday.  It will always back up to the same place, which will be designated at the monday backup.  This will happen once a week and the following week will update the previous monday backup
0
 
bman21Commented:
just to expand on robthewolf and IanTh's suggestion, the cron daemon would have to look something like this:
####################################################
# Daily backups: This will run rsync every night at                      #
#  1:00 a.m. (you can change the time to whatever  you want)  #
####################################################
#Sunday
0 1 * * 0 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/SUNDAY/

#Monday
0 1 * * 1 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/MONDAY/

#Tuesday
0 1 * * 2 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/TUESDAY/

#Wednesday
0 1 * * 3 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/WEDNESDAY/

#Thursday
0 1 * * 4 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/THURSDAY/

#Friday
0 1 * * 5 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/FRIDAY/

#Saturday
0 1 * * 6 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/SATURDAY/

##############################
# Keep 4 WEEKLY "snapshots"        #
# runs at 2 am Saturdays if the day #
# of the month falls on the given       #
# day range                                       #
##############################

#Week 1
0 2 1-7 * 6 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/WEEK_1/

#Week 2
0 2 8-14 * 6 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/WEEK_2/

#Week 3
0 2 15-21 * 6 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/WEEK_3/

#Week 4
0 2 22-31 * 6 rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/WEEK_4/

#################
# Monthly backups  #
#################

#January
0 3 * 1 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/January/

#February
0 3 * 2 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/February/

#March
0 3 * 3 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/March/

#April
0 3 * 4 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/April/

#May
0 3 * 5 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/May/

#June
0 3 * 6 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/June/

#July
0 3 * 7 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/July/

#August
0 3 * 8 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/August/

#September
0 3 * 9 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/September/

#October
0 3 * 10 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/October/

#November
0 3 * 11 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/November/

#December
0 3 * 12 * rsync -avu --delete /LOCAL/DIRECTORY/ -e ssh remote_server:/REMOTE/DIRECTORY/December/

##################
# End of Crontab file #
##################

**NOTE** you can use the --dry-run option to ensure it is doing what you want to do without actually copying anything over.  

**NOTE** just for clarity, the * * * * * in front of the rsync command tells cron when to run rsync.  Each * stands for MINUTES, HOUR, DAY_OF_MONTH, MONTH, DAY_OF_WEEK.  

**NOTE**  I am not to sure if the weekly scripts will work.  Technicaly, the Daily scropts we created is the way to do weekly cron jobs, but since you are wanting to keep a weekly backup, that is the only way that i know how to do it short of actually creating a shell script.

The Daily and Monthly scripts will work though.  Notice that I added the "-u" and "--delete" commands.  The -u, or update, switch saves time on your backups after the first one has been created.  The --delete will delete any files off the remote directory that are no longer existent on the local filesystem.

I know that was a lot to dump on you.  If you have any questions please feel free to ask.  The above written code should be a "plug and play" minus a few alterations to the actually directory and the ssh connection to your backup site.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.