Rsync Snapshot - Help understand shell script

Hi Experts,

Can you please help me understand the following shell script that I'd found from  Mike Rubel's website?

To be consistent with Mike's setup on his main web page, shouldn't line 36 be:

[ -d $dst/$1.1 ] && cp -al $dst/$1.0 $dst/$1.1

i.e. copy from version 0 over version 1? I understand it's only copying across the hard link but shouldn't the order matter?

Thanks in advance
Ricky

#!/bin/bash

# This script makes a snapshot of several directories using rsync.
# It's based on Rob Bos's snapshot script found at mikerubel.org.

# Version 0.1, Geordy Kitchen, November 26, 2002
# This script is released under the terms of the GNU General Public License,
# version 2.0.

# Your crontab will typically look something like this:
# 0 */4 * * * /bin/this_script hourly
# 59 3 * * * /bin/this_script daily
# 58 3 * * 0 /bin/this_script weekly
# 57 3 1 * * /bin/this_script monthly

# Configuration

  # Destination directory.
dst=/backup

srcroot="user1@hostname"

  # Use a trailing slash for directories, just the filename for files.
src="/var/mail/user1 /home/user1/"

  # Number of times to run per day.
hourly=6

# Constants
daily=7
weekly=4
monthly=12

case $1 in 
    hourly)
        [ -d $dst/$1.1 ] && cp -al $dst/$1.1 $dst/$1.0
        for i in $src; do
	    dirdst=`echo $i | sed 's|^/||; s|[^/]*$||'`
	    [ -d $dst/$1.0/$dirdst ] || mkdir -p $dst/$1.0/$dirdst
	    rsync -a --delete $srcroot:$i $dst/$1.0/$dirdst
	done
	;;
    daily)
        [ -d $dst/hourly.$hourly ] && mv $dst/hourly.$hourly $dst/daily.0
	;;
    weekly)
	[ -d $dst/daily.$daily ] && mv $dst/daily.$daily $dst/weekly.0
	;;
    monthly)
	[ -d $dst/weekly.$weekly ] && mv $dst/weekly.$weekly $dst/monthly.0
	;;
    *)
	echo "syntax: $0 <type>"
        exit 1
	;;
esac

# Rotate the current list of backups, if we can.
if [ -d $dst/$1.0 ]; then
	oldest=`ls -d $dst/$1.* | tail -n 1 | sed 's/^.*\.//'`
	for i in `seq $oldest 0`; do
	    mv $dst/$1.$i $dst/$1.$((i+1))
	done
fi

# if we've rotated the last backup off the stack, remove it.
[ -d $dst/$1.$((${!1}+1)) ] && rm -rf $dst/$1.$((${!1}+1))

# finally, let's make the new snapshot reflect the current date.
[ -d $dst/$1.1 ] && touch $dst/$1.1

Open in new window

Ricky NguyenITAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ozoCommented:
Order does matter, but it looks like the rotate at line 62 already moves version 0 to version 1, so the cp seems to be copying things back to 0
0
Ricky NguyenITAuthor Commented:
From line 36 to line 40 this is how I'm seeing it.
Line 36: If directory for version 1 exist then copy version 1 over version 0.
Line 38: Determine the sub-branch string.
Line 39: Check if required version 0 directory exist, if not create it.
Line 40: Rsync source to version 0.

Wouldn't line 36 be redundant? Is this a bug? Should the copy be from version 0 to version 1?
0
ozoCommented:
I can't be sure whether it is a bug without knowing the intention of the author, but it does not seem redundant to me.
Rather, it may avoid redundant rsync copying.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

Ricky NguyenITAuthor Commented:
Thanks. That's probably it.
0
Ricky NguyenITAuthor Commented:
The purpose of line 36 was to recreate directory structure with hard links. When I run the script the first time. Version 1 was created. When I ran the script the second time, version 1 was copy under version 0 directory tree. I don't think that was the intention.
0
ozoCommented:
If it was not the intention, then I would call it a bug,
On the other hand, it's possible to imagine that the author could have preferred a hard linked copy under version 0 rather than a rsync copy under version 0

But if the intention was to copy version 0 to version 1, then it would seem that it was meant  to cover the case where a previous run failed before the Rotate could be completed, since I don't see how else $dst/$1.0 could exist.
In that case, one would have to explain the intention to copy the failed version 0 over the previous version 1
0
Ricky NguyenITAuthor Commented:
How would line 36 create directory structure with hardlinks? I'm really confused now.
0
Ricky NguyenITAuthor Commented:
the hourly rotation isn't keeping 6 copies. I'm only seeing version 0 and version 1.
0
ozoCommented:
recreateing directory structure with hard links is what you claimed in http:#a39529985 was  the purpose of line 36, and that interpretation seems constant with the man cp documentation of the -al switches.
0
ozoCommented:
Have you run the script 6 times?
Did the
# Rotate the current list of backups, if we can.
section work?
0
Ricky NguyenITAuthor Commented:
I ran it a dozen times. It's not rotating. It works when I change line 61 to:
 for ((i=$oldest; i=1; i--)); do
0
Ricky NguyenITAuthor Commented:
Thanks for sticking around ozo. it's getting pretty late here so I'll just leave it for now. I think the problem is the gap in my knowledge. I might just come back to this when I build up a little more command line experience.

Many thanks again.
Ricky
0
Ricky NguyenITAuthor Commented:
In case you're interested in the original code below from Mike's website.:

#!/bin/bash
# ----------------------------------------------------------------------
# mikes handy rotating-filesystem-snapshot utility
# ----------------------------------------------------------------------
# this needs to be a lot more general, but the basic idea is it makes
# rotating backup-snapshots of /home whenever called
# ----------------------------------------------------------------------

unset PATH	# suggestion from H. Milz: avoid accidental use of $PATH

# ------------- system commands used by this script --------------------
ID=/usr/bin/id;
ECHO=/bin/echo;

MOUNT=/bin/mount;
RM=/bin/rm;
MV=/bin/mv;
CP=/bin/cp;
TOUCH=/bin/touch;

RSYNC=/usr/bin/rsync;


# ------------- file locations -----------------------------------------

MOUNT_DEVICE=/dev/hdb1;
SNAPSHOT_RW=/root/snapshot;
EXCLUDES=/usr/local/etc/backup_exclude;


# ------------- the script itself --------------------------------------

# make sure we're running as root
if (( `$ID -u` != 0 )); then { $ECHO "Sorry, must be root.  Exiting..."; exit; } fi

# attempt to remount the RW mount point as RW; else abort
$MOUNT -o remount,rw $MOUNT_DEVICE $SNAPSHOT_RW ;
if (( $? )); then
{
	$ECHO "snapshot: could not remount $SNAPSHOT_RW readwrite";
	exit;
}
fi;


# rotating snapshots of /home (fixme: this should be more general)

# step 1: delete the oldest snapshot, if it exists:
if [ -d $SNAPSHOT_RW/home/hourly.3 ] ; then			\
$RM -rf $SNAPSHOT_RW/home/hourly.3 ;				\
fi ;

# step 2: shift the middle snapshots(s) back by one, if they exist
if [ -d $SNAPSHOT_RW/home/hourly.2 ] ; then			\
$MV $SNAPSHOT_RW/home/hourly.2 $SNAPSHOT_RW/home/hourly.3 ;	\
fi;
if [ -d $SNAPSHOT_RW/home/hourly.1 ] ; then			\
$MV $SNAPSHOT_RW/home/hourly.1 $SNAPSHOT_RW/home/hourly.2 ;	\
fi;

# step 3: make a hard-link-only (except for dirs) copy of the latest snapshot,
# if that exists
if [ -d $SNAPSHOT_RW/home/hourly.0 ] ; then			\
$CP -al $SNAPSHOT_RW/home/hourly.0 $SNAPSHOT_RW/home/hourly.1 ;	\
fi;

# step 4: rsync from the system into the latest snapshot (notice that
# rsync behaves like cp --remove-destination by default, so the destination
# is unlinked first.  If it were not so, this would copy over the other
# snapshot(s) too!
$RSYNC								\
	-va --delete --delete-excluded				\
	--exclude-from="$EXCLUDES"				\
	/home/ $SNAPSHOT_RW/home/hourly.0 ;

# step 5: update the mtime of hourly.0 to reflect the snapshot time
$TOUCH $SNAPSHOT_RW/home/hourly.0 ;

# and thats it for home.

# now remount the RW snapshot mountpoint as readonly

$MOUNT -o remount,ro $MOUNT_DEVICE $SNAPSHOT_RW ;
if (( $? )); then
{
	$ECHO "snapshot: could not remount $SNAPSHOT_RW readonly";
	exit;
} fi;

Open in new window

0
ozoCommented:
what does
seq 5 0
do on your system?
0
Gerwin Jansen, EE MVETopic Advisor Commented:
@ozo - seq 5 0 should produce no output at all.
0
Ricky NguyenITAuthor Commented:
yeah seq 50 doesn't do anything for me either.
0
ozoCommented:
on some systems
seq 5 0
produces
5
4
3
2
1
0
the script appears to have been written for such systems
0
Ricky NguyenITAuthor Commented:
yep your spot on. thats why i was having issues with line 36. Once looping on line 61 was updated with for ((i=$oldest; i>=1; i--)); do, my hourly snapshot works perfectly. I no longer have nested branches. The loops would move all folders up one version. That's why on line 36, there would be no version 0 when the copy occurs.
0
ozoCommented:
Does
seq $oldest  -1 0
work on your system?
0
Ricky NguyenITAuthor Commented:
no seq $oldest -1 0 doesn't work. I think the seq command is no longer supported.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.