Can rsync access/copy/backup open files on Ubuntu?

harrowc
harrowc used Ask the Experts™
on
Hi,

I would like to know whether rsync can be relied upon as a full system backup.

For example could I backup the entire root partition, and restore it to a freshly formatted new disk and that would work?

This hinges on being able to backup or copy files that are in use or 'open' files.

My searching on google for this seems to come up with the answer of 'sometimes' and 'it depends on the operating system' and such.

Anybody found the answer and are using rsync for system backup? or just data backup?

Thank you.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Commented:
In most cases, it performs a backup sufficient for recovery.   Although I prefer to use 'rdiff-backup'  or   'rsnapshot'  for something closer to a true incremental backup.

However, files that are in use may not be in a crash-consistent state.
You have to ask the question -- what files are routinely being written to on the system?

There are a few ways to handle this.

(1) Shut down the application,  for example, if a MySQL server is running on the machine,   the backup script could start the service before rsync'ing  /var/lib/mysql

(2) Tell the application it's being backed up, or use application-specific tools,  for example   mysqlhotcopy.    PostgreSQL's    pg_start_backup(),  pg_end_backup()

(3) Use an  app-specific  data dump / data export procedure for the special apps,  e.g.  mysqldump  all databases  to a backup file  that is copied by rsync,  or pg_dumpall  your PostgreSQL cluster.


(4)  Use a volume or filesystem snapshot facility.
If your files are stored on a NAS or SAN,  the  SAN or NAS may have the ability to create a point-in-time snapshot of your disk contents.

Use 'sync'  to commit all pending writes to disk,  create the snapshot,   then mount the snapshot  in read-only mode,  and copy files _from the snapshot_  to the backup server.

This produces a crash-consistent backup.


Current Linux filesystems do not support any 'filesystem level' snapshot mechanism,  on OpenSolaris systems using ZFS as root filesystem, there is a snapshot mechanism available.


However, if your Linux filesystem exists on top of a LVM volume, and if not _all disk space_  has been assigned to volumes   (E.g.  the LVM volume group has free disk space)

You can create a 'LVM Snapshot'  based on your existing filesystem.
* Mount the LVM snapshot in read-only mode.
* RSYNC files from the read-only volume to the backup server
* Unmount the read-only snapshot
* Delete the snapshot

You have a crash-consistent backup.



Top Expert 2015

Commented:
I do not see any practical purpose of makeing backups of changing files.
There is nothing in rsync to prevent copying them.
Indeed - I do use rsync for file transfer, not backup.
Top Expert 2007

Commented:
gheist, what do you use for doing backups?
11/26 Forrester Webinar: Savings for Enterprise

How can your organization benefit from savings just by replacing your legacy backup solutions with Acronis' #CyberProtection? Join Forrester's Joe Branca and Ryan Davis from Acronis live as they explain how you can too.

Author

Commented:
Thanks Mysidia, good answer.
It is interesting that a LVM snapshot sounds exactly like volume shadow copy  on Windows, perhaps that were Microsoft/Veritas got the idea.

Thanks,

Commented:
* One downside of LVM snapshots that should be noted;  when a snapshot exists,  write performance on the original volume can be impacted a bit.   Using backups for snapshots  is most suitable to be done during a period of low load.

Linux LVM was first written in 1998, and was based on the concepts in the HP-UX  volume manager.

The HP Logical Volume Manager in HP-UX got snapshot capabilities in version 9.0, which was released in 1995.

The Veritas filesystem  and storage facilities were originally developed under an OEM agreement,  and  it's the  standard  filesystem used by HP-UX.

Microsoft licenses various storage technology from Veritas for use in Windows.
If you open 'Disk Management' on a windows XP or server 2003 machine,
And choose "Help >  About Disk Management Extension"
You will see "Dynamic disk and volume management provided to Microsoft by VERITAS Software corporation"



Top Expert 2015

Commented:
I do use DVD-R for backups, My employer uses TSM and SQL Backtrack.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial