Solved

Script for LVM and rsync backup

Posted on 2010-08-17
9
1,033 Views
Last Modified: 2012-06-27
I need to backup my email server.  I want to use rsync, but because files are in a constant state of change (email coming in and out, logs changing, etc), I think it would be much cleaner to take an LVM snapshot first, then rsync it.

Can someone recommend a script that would create the LVM snap and then rsync it to the secondary server?  I have made the private and public keys (I can rsync just fine right now, except for all the files and the database that are in states of chaneg, it's a postgre db if that helps any).  

Thanks anyone!
0
Comment
Question by:dougp23
9 Comments
 
LVL 16

Expert Comment

by:santoshmotwani
ID: 33453863
0
 
LVL 5

Expert Comment

by:stermeau
ID: 33453896
If it it a postgres database, why not doing a dump of the database itself?
0
 
LVL 35

Expert Comment

by:torimar
ID: 33453995
0
 
LVL 25

Accepted Solution

by:
madunix earned 250 total points
ID: 33455261
rsync is a good tool to build a relation between master and slave, for example my qmail server master and slave are replucated by
copying  these directories:
- - /home/vpopmail/
- - /var/qmail/control/
- - /var/qmail/alias
and other directories that exists in both the servers and needs to be replicated, using rsync command:
rsync -avz -e ssh -R /home/vpopmail user@host:/home/rsync/backupdir

on the otherhand it depends on you application, If the files and file systems are too heavey I/O _very_ busy or locked by applications copy/ftp/rsync... commands are not useful specially for DB and it is better to use mirror/shadow/standby features of database.

regarding rsync i have created my own script to move data, feel free to change it
the script i have tested on multiple linux server RHEL, Centos
#!/bin/bash
#  This script uses Rsync the backup files
#  to a remote server. To use this script
#  you first have to setup ssh to use keys.
#  This elimates the need for a password but
#  still provides security I have ssh on
#  a non standard port so I had to specify
#  the port (ssh -p 2998).
##########################################
#  Simply change the variables list below
#  to match your specifications.
##########################################
#  The Script also logs the transfer time.
#  For automated backup add script to crontab.
# Setup Variables
SOURCEPATH="/../source/"
DESTPATH="/../target/"
DESTHOST="10.6.40.81"
#The user you created with ssh_keys
DESTUSER="root"
LOCKFILE="/var/lock/rsyncbackup.lock"
LOGFILE="/var/log/rsyncbackup.log"
STARTTIME=$(date +"%s")
# Verfiy the last rsync is not still running
if [ -f $LOCKFILE ]; then
ERRORTIME=$(date +"%d/%b/%Y:%H:%M:%S %Z")
echo "[$ERRORTIME] WARNING rsyncbackup.lock exists, aborting..." >> $LOGFILE
exit 1
fi
# If it finished, create a new lock file
touch $LOCKFILE
# The rsync command (for explanation type rsync --help)
rsync -aqrpogtu --delete --bwlimit=512 --rsh='ssh -p 22' $SOURCEPATH $DESTUSER@$DESTHOST:$DESTPATH 2>&1 >> $LOGFILE
# Log when the script was run
DONE=$(date +"%d/%b/%Y:%H:%M:%S %Z")
# Log how long it took the script to run
STOPTIME=$(date +"%s")
SECONDS=$(($STOPTIME - $STARTTIME))
echo "[$DONE]finished: - Tranfer took:$(((SECONDS/60)/60)) hours $(((SECONDS/60)%60)) minutes $((SECONDS%60)) seconds" >> $LOGFILE
# Remove lock file
rm -f $LOCKFILE
# END
#

Open in new window

0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 3

Assisted Solution

by:captainmish
captainmish earned 250 total points
ID: 33462845
A simple bash script could do this for you, along the lines of:

lvcreate -s -n mysnapshot /dev/vgname/lvname
mount /dev/vgname/mysnapshot /mnt/snapshot
rsync -avzx /mnt/snapshot user@destination:/destination
umount /mnt/snapshot
lvremove -f /dev/vgname/mysnapshot

of course, you will probably want to add some dates, destination tricks etc as well - also check out --link-dest which will make hard links where possible and save you some disk space on the destination
0
 
LVL 1

Author Comment

by:dougp23
ID: 33514208
Thank you, I think I'm pretty close!  How do I display any LV I might already have?  Is it possible none are configured? Where would they be under /dev if they were configured?

Thanks.
0
 
LVL 3

Expert Comment

by:captainmish
ID: 33515983
To display existing logical volumes, use lvdisplay (need to be root or use sudo), on ubuntu, the device files can be found under /dev/mapper/vgname-lvname or /dev/vgname/lvname

To make a whole volume group available to the kernel, use vgchange -a y (this will more than likely be done for you on boot though)
0
 
LVL 3

Expert Comment

by:captainmish
ID: 33516023
Snapshots will have a "LV snapshot status" line in the output of lvdisplay, if its all working well, it will be something like "active destination for /dev/vgname/lvname"
0
 
LVL 1

Author Comment

by:dougp23
ID: 33521536
lvdisplay returns "No volume groups found".  Is it easy to enable LVM?  It won't format the disks or anything I hope if I try it?
0

Featured Post

Get up to 2TB FREE CLOUD per backup license!

An exclusive Black Friday offer just for Expert Exchange audience! Buy any of our top-rated backup solutions & get up to 2TB free cloud per system! Perform local & cloud backup in the same step, and restore instantly—anytime, anywhere. Grab this deal now before it disappears!

Join & Write a Comment

rdate is a Linux command and the network time protocol for immediate date and time setup from another machine. The clocks are synchronized by entering rdate with the -s switch (command without switch just checks the time but does not set anything). …
I am a long time windows user and for me it is normal to have spaces in directory and file names. Changing to Linux I found myself frustrated when I moved my windows data over to my new Linux computer. The problem occurs when at the command line.…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now