Solved

Script for LVM and rsync backup

Posted on 2010-08-17
9
1,057 Views
Last Modified: 2012-06-27
I need to backup my email server.  I want to use rsync, but because files are in a constant state of change (email coming in and out, logs changing, etc), I think it would be much cleaner to take an LVM snapshot first, then rsync it.

Can someone recommend a script that would create the LVM snap and then rsync it to the secondary server?  I have made the private and public keys (I can rsync just fine right now, except for all the files and the database that are in states of chaneg, it's a postgre db if that helps any).  

Thanks anyone!
0
Comment
Question by:dougp23
9 Comments
 
LVL 16

Expert Comment

by:santoshmotwani
ID: 33453863
0
 
LVL 5

Expert Comment

by:stermeau
ID: 33453896
If it it a postgres database, why not doing a dump of the database itself?
0
 
LVL 35

Expert Comment

by:torimar
ID: 33453995
0
Simplifying Server Workload Migrations

This use case outlines the migration challenges that organizations face and how the Acronis AnyData Engine supports physical-to-physical (P2P), physical-to-virtual (P2V), virtual to physical (V2P), and cross-virtual (V2V) migration scenarios to address these challenges.

 
LVL 25

Accepted Solution

by:
madunix earned 250 total points
ID: 33455261
rsync is a good tool to build a relation between master and slave, for example my qmail server master and slave are replucated by
copying  these directories:
- - /home/vpopmail/
- - /var/qmail/control/
- - /var/qmail/alias
and other directories that exists in both the servers and needs to be replicated, using rsync command:
rsync -avz -e ssh -R /home/vpopmail user@host:/home/rsync/backupdir

on the otherhand it depends on you application, If the files and file systems are too heavey I/O _very_ busy or locked by applications copy/ftp/rsync... commands are not useful specially for DB and it is better to use mirror/shadow/standby features of database.

regarding rsync i have created my own script to move data, feel free to change it
the script i have tested on multiple linux server RHEL, Centos
#!/bin/bash
#  This script uses Rsync the backup files
#  to a remote server. To use this script
#  you first have to setup ssh to use keys.
#  This elimates the need for a password but
#  still provides security I have ssh on
#  a non standard port so I had to specify
#  the port (ssh -p 2998).
##########################################
#  Simply change the variables list below
#  to match your specifications.
##########################################
#  The Script also logs the transfer time.
#  For automated backup add script to crontab.
# Setup Variables
SOURCEPATH="/../source/"
DESTPATH="/../target/"
DESTHOST="10.6.40.81"
#The user you created with ssh_keys
DESTUSER="root"
LOCKFILE="/var/lock/rsyncbackup.lock"
LOGFILE="/var/log/rsyncbackup.log"
STARTTIME=$(date +"%s")
# Verfiy the last rsync is not still running
if [ -f $LOCKFILE ]; then
ERRORTIME=$(date +"%d/%b/%Y:%H:%M:%S %Z")
echo "[$ERRORTIME] WARNING rsyncbackup.lock exists, aborting..." >> $LOGFILE
exit 1
fi
# If it finished, create a new lock file
touch $LOCKFILE
# The rsync command (for explanation type rsync --help)
rsync -aqrpogtu --delete --bwlimit=512 --rsh='ssh -p 22' $SOURCEPATH $DESTUSER@$DESTHOST:$DESTPATH 2>&1 >> $LOGFILE
# Log when the script was run
DONE=$(date +"%d/%b/%Y:%H:%M:%S %Z")
# Log how long it took the script to run
STOPTIME=$(date +"%s")
SECONDS=$(($STOPTIME - $STARTTIME))
echo "[$DONE]finished: - Tranfer took:$(((SECONDS/60)/60)) hours $(((SECONDS/60)%60)) minutes $((SECONDS%60)) seconds" >> $LOGFILE
# Remove lock file
rm -f $LOCKFILE
# END
#

Open in new window

0
 
LVL 3

Assisted Solution

by:captainmish
captainmish earned 250 total points
ID: 33462845
A simple bash script could do this for you, along the lines of:

lvcreate -s -n mysnapshot /dev/vgname/lvname
mount /dev/vgname/mysnapshot /mnt/snapshot
rsync -avzx /mnt/snapshot user@destination:/destination
umount /mnt/snapshot
lvremove -f /dev/vgname/mysnapshot

of course, you will probably want to add some dates, destination tricks etc as well - also check out --link-dest which will make hard links where possible and save you some disk space on the destination
0
 
LVL 1

Author Comment

by:dougp23
ID: 33514208
Thank you, I think I'm pretty close!  How do I display any LV I might already have?  Is it possible none are configured? Where would they be under /dev if they were configured?

Thanks.
0
 
LVL 3

Expert Comment

by:captainmish
ID: 33515983
To display existing logical volumes, use lvdisplay (need to be root or use sudo), on ubuntu, the device files can be found under /dev/mapper/vgname-lvname or /dev/vgname/lvname

To make a whole volume group available to the kernel, use vgchange -a y (this will more than likely be done for you on boot though)
0
 
LVL 3

Expert Comment

by:captainmish
ID: 33516023
Snapshots will have a "LV snapshot status" line in the output of lvdisplay, if its all working well, it will be something like "active destination for /dev/vgname/lvname"
0
 
LVL 1

Author Comment

by:dougp23
ID: 33521536
lvdisplay returns "No volume groups found".  Is it easy to enable LVM?  It won't format the disks or anything I hope if I try it?
0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Over the last ten+ years I have seen Linux configuration tools come and go. In the early days there was the tried-and-true, all-powerful linuxconf that many thought would remain the one and only Linux configuration tool until the end of times. Well,…
Using 'screen' for session sharing, The Simple Edition Step 1: user starts session with command: screen Step 2: other user (logged in with same user account) connects with command: screen -x Done. Both users are connected to the same CLI sessio…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

792 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question