Solved

Script for LVM and rsync backup

Posted on 2010-08-17
9
1,097 Views
Last Modified: 2012-06-27
I need to backup my email server.  I want to use rsync, but because files are in a constant state of change (email coming in and out, logs changing, etc), I think it would be much cleaner to take an LVM snapshot first, then rsync it.

Can someone recommend a script that would create the LVM snap and then rsync it to the secondary server?  I have made the private and public keys (I can rsync just fine right now, except for all the files and the database that are in states of chaneg, it's a postgre db if that helps any).  

Thanks anyone!
0
Comment
Question by:dougp23
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
9 Comments
 
LVL 5

Expert Comment

by:stermeau
ID: 33453896
If it it a postgres database, why not doing a dump of the database itself?
0
NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.

 
LVL 25

Accepted Solution

by:
madunix earned 250 total points
ID: 33455261
rsync is a good tool to build a relation between master and slave, for example my qmail server master and slave are replucated by
copying  these directories:
- - /home/vpopmail/
- - /var/qmail/control/
- - /var/qmail/alias
and other directories that exists in both the servers and needs to be replicated, using rsync command:
rsync -avz -e ssh -R /home/vpopmail user@host:/home/rsync/backupdir

on the otherhand it depends on you application, If the files and file systems are too heavey I/O _very_ busy or locked by applications copy/ftp/rsync... commands are not useful specially for DB and it is better to use mirror/shadow/standby features of database.

regarding rsync i have created my own script to move data, feel free to change it
the script i have tested on multiple linux server RHEL, Centos
#!/bin/bash
#  This script uses Rsync the backup files
#  to a remote server. To use this script
#  you first have to setup ssh to use keys.
#  This elimates the need for a password but
#  still provides security I have ssh on
#  a non standard port so I had to specify
#  the port (ssh -p 2998).
##########################################
#  Simply change the variables list below
#  to match your specifications.
##########################################
#  The Script also logs the transfer time.
#  For automated backup add script to crontab.
# Setup Variables
SOURCEPATH="/../source/"
DESTPATH="/../target/"
DESTHOST="10.6.40.81"
#The user you created with ssh_keys
DESTUSER="root"
LOCKFILE="/var/lock/rsyncbackup.lock"
LOGFILE="/var/log/rsyncbackup.log"
STARTTIME=$(date +"%s")
# Verfiy the last rsync is not still running
if [ -f $LOCKFILE ]; then
ERRORTIME=$(date +"%d/%b/%Y:%H:%M:%S %Z")
echo "[$ERRORTIME] WARNING rsyncbackup.lock exists, aborting..." >> $LOGFILE
exit 1
fi
# If it finished, create a new lock file
touch $LOCKFILE
# The rsync command (for explanation type rsync --help)
rsync -aqrpogtu --delete --bwlimit=512 --rsh='ssh -p 22' $SOURCEPATH $DESTUSER@$DESTHOST:$DESTPATH 2>&1 >> $LOGFILE
# Log when the script was run
DONE=$(date +"%d/%b/%Y:%H:%M:%S %Z")
# Log how long it took the script to run
STOPTIME=$(date +"%s")
SECONDS=$(($STOPTIME - $STARTTIME))
echo "[$DONE]finished: - Tranfer took:$(((SECONDS/60)/60)) hours $(((SECONDS/60)%60)) minutes $((SECONDS%60)) seconds" >> $LOGFILE
# Remove lock file
rm -f $LOCKFILE
# END
#

Open in new window

0
 
LVL 3

Assisted Solution

by:captainmish
captainmish earned 250 total points
ID: 33462845
A simple bash script could do this for you, along the lines of:

lvcreate -s -n mysnapshot /dev/vgname/lvname
mount /dev/vgname/mysnapshot /mnt/snapshot
rsync -avzx /mnt/snapshot user@destination:/destination
umount /mnt/snapshot
lvremove -f /dev/vgname/mysnapshot

of course, you will probably want to add some dates, destination tricks etc as well - also check out --link-dest which will make hard links where possible and save you some disk space on the destination
0
 
LVL 1

Author Comment

by:dougp23
ID: 33514208
Thank you, I think I'm pretty close!  How do I display any LV I might already have?  Is it possible none are configured? Where would they be under /dev if they were configured?

Thanks.
0
 
LVL 3

Expert Comment

by:captainmish
ID: 33515983
To display existing logical volumes, use lvdisplay (need to be root or use sudo), on ubuntu, the device files can be found under /dev/mapper/vgname-lvname or /dev/vgname/lvname

To make a whole volume group available to the kernel, use vgchange -a y (this will more than likely be done for you on boot though)
0
 
LVL 3

Expert Comment

by:captainmish
ID: 33516023
Snapshots will have a "LV snapshot status" line in the output of lvdisplay, if its all working well, it will be something like "active destination for /dev/vgname/lvname"
0
 
LVL 1

Author Comment

by:dougp23
ID: 33521536
lvdisplay returns "No volume groups found".  Is it easy to enable LVM?  It won't format the disks or anything I hope if I try it?
0

Featured Post

Do you have a plan for Continuity?

It's inevitable. People leave organizations creating a gap in your service. That's where Percona comes in.

See how Pepper.com relies on Percona to:
-Manage their database
-Guarantee data safety and protection
-Provide database expertise that is available for any situation

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you have a server on collocation with the super-fast CPU, that doesn't mean that you get it running at full power. Here is a preamble. When doing inventory of Linux servers, that I'm administering, I've found that some of them are running on l…
Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial
Suggested Courses

631 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question