Solved

Script for LVM and rsync backup

Posted on 2010-08-17
9
1,070 Views
Last Modified: 2012-06-27
I need to backup my email server.  I want to use rsync, but because files are in a constant state of change (email coming in and out, logs changing, etc), I think it would be much cleaner to take an LVM snapshot first, then rsync it.

Can someone recommend a script that would create the LVM snap and then rsync it to the secondary server?  I have made the private and public keys (I can rsync just fine right now, except for all the files and the database that are in states of chaneg, it's a postgre db if that helps any).  

Thanks anyone!
0
Comment
Question by:dougp23
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
9 Comments
 
LVL 16

Expert Comment

by:santoshmotwani
ID: 33453863
0
 
LVL 5

Expert Comment

by:stermeau
ID: 33453896
If it it a postgres database, why not doing a dump of the database itself?
0
 
LVL 35

Expert Comment

by:torimar
ID: 33453995
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 25

Accepted Solution

by:
madunix earned 250 total points
ID: 33455261
rsync is a good tool to build a relation between master and slave, for example my qmail server master and slave are replucated by
copying  these directories:
- - /home/vpopmail/
- - /var/qmail/control/
- - /var/qmail/alias
and other directories that exists in both the servers and needs to be replicated, using rsync command:
rsync -avz -e ssh -R /home/vpopmail user@host:/home/rsync/backupdir

on the otherhand it depends on you application, If the files and file systems are too heavey I/O _very_ busy or locked by applications copy/ftp/rsync... commands are not useful specially for DB and it is better to use mirror/shadow/standby features of database.

regarding rsync i have created my own script to move data, feel free to change it
the script i have tested on multiple linux server RHEL, Centos
#!/bin/bash
#  This script uses Rsync the backup files
#  to a remote server. To use this script
#  you first have to setup ssh to use keys.
#  This elimates the need for a password but
#  still provides security I have ssh on
#  a non standard port so I had to specify
#  the port (ssh -p 2998).
##########################################
#  Simply change the variables list below
#  to match your specifications.
##########################################
#  The Script also logs the transfer time.
#  For automated backup add script to crontab.
# Setup Variables
SOURCEPATH="/../source/"
DESTPATH="/../target/"
DESTHOST="10.6.40.81"
#The user you created with ssh_keys
DESTUSER="root"
LOCKFILE="/var/lock/rsyncbackup.lock"
LOGFILE="/var/log/rsyncbackup.log"
STARTTIME=$(date +"%s")
# Verfiy the last rsync is not still running
if [ -f $LOCKFILE ]; then
ERRORTIME=$(date +"%d/%b/%Y:%H:%M:%S %Z")
echo "[$ERRORTIME] WARNING rsyncbackup.lock exists, aborting..." >> $LOGFILE
exit 1
fi
# If it finished, create a new lock file
touch $LOCKFILE
# The rsync command (for explanation type rsync --help)
rsync -aqrpogtu --delete --bwlimit=512 --rsh='ssh -p 22' $SOURCEPATH $DESTUSER@$DESTHOST:$DESTPATH 2>&1 >> $LOGFILE
# Log when the script was run
DONE=$(date +"%d/%b/%Y:%H:%M:%S %Z")
# Log how long it took the script to run
STOPTIME=$(date +"%s")
SECONDS=$(($STOPTIME - $STARTTIME))
echo "[$DONE]finished: - Tranfer took:$(((SECONDS/60)/60)) hours $(((SECONDS/60)%60)) minutes $((SECONDS%60)) seconds" >> $LOGFILE
# Remove lock file
rm -f $LOCKFILE
# END
#

Open in new window

0
 
LVL 3

Assisted Solution

by:captainmish
captainmish earned 250 total points
ID: 33462845
A simple bash script could do this for you, along the lines of:

lvcreate -s -n mysnapshot /dev/vgname/lvname
mount /dev/vgname/mysnapshot /mnt/snapshot
rsync -avzx /mnt/snapshot user@destination:/destination
umount /mnt/snapshot
lvremove -f /dev/vgname/mysnapshot

of course, you will probably want to add some dates, destination tricks etc as well - also check out --link-dest which will make hard links where possible and save you some disk space on the destination
0
 
LVL 1

Author Comment

by:dougp23
ID: 33514208
Thank you, I think I'm pretty close!  How do I display any LV I might already have?  Is it possible none are configured? Where would they be under /dev if they were configured?

Thanks.
0
 
LVL 3

Expert Comment

by:captainmish
ID: 33515983
To display existing logical volumes, use lvdisplay (need to be root or use sudo), on ubuntu, the device files can be found under /dev/mapper/vgname-lvname or /dev/vgname/lvname

To make a whole volume group available to the kernel, use vgchange -a y (this will more than likely be done for you on boot though)
0
 
LVL 3

Expert Comment

by:captainmish
ID: 33516023
Snapshots will have a "LV snapshot status" line in the output of lvdisplay, if its all working well, it will be something like "active destination for /dev/vgname/lvname"
0
 
LVL 1

Author Comment

by:dougp23
ID: 33521536
lvdisplay returns "No volume groups found".  Is it easy to enable LVM?  It won't format the disks or anything I hope if I try it?
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Weird Samba Connectivity Issue... 7 64
Connect to CentOS (in a local VMWare VM) using Putty 7 139
SSH in linux 9 65
CentOs root password/fsck issue 7 31
Network Interface Card (NIC) bonding, also known as link aggregation, NIC teaming and trunking, is an important concept to understand and implement in any environment where high availability is of concern. Using this feature, a server administrator …
Fine Tune your automatic Updates for Ubuntu / Debian
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

740 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question