Solved

Automated Backups in Linux (Duplicity)

Posted on 2008-10-25
7
1,387 Views
Last Modified: 2013-12-16
Hi!

I'm using Duplicity ( http://duplicity.nongnu.org/ ) to automatically backup my files.

The following command will backup all of my users' website files and preferences (e.g. Spamassassin prefs) to a separate drive:

duplicity -v8 /var/www file:///backup/duplicity/var/www

What I need is to exclude all the temporary files. Temporary files are all inside a directory called "phptmp" and there is one of these directories per user, e.g.

/var/www/web1/phptmp/
/var/www/web2/phptmp/
/var/www/web3/phptmp/
/var/www/web4/phptmp/

My question is: what is the syntax I need to exclude all these temp folders?

0
Comment
Question by:Julian Matz
  • 3
  • 3
7 Comments
 
LVL 16

Accepted Solution

by:
ai_ja_nai earned 500 total points
ID: 22805274
--exclude /destination/dir option does the job. You may use wildcards to generalize your choice.
Also, --exclude-filelist /path/to/file.txt lets you define a plain text file with directories listed inside, separed by returns.
Look for the manual file on duplicity website
0
 
LVL 21

Author Comment

by:Julian Matz
ID: 22806430
Thanks. I have looked through the manual numerous times but couldn't find the answer. Do you know how a wildcard would be applied exactly?
This is where the wildcard would need to be placed (*):

--exclude /var/www/web*/phptmp

Or would it have to be? :

--exclude /web*/phptmp

I didn't know that you need to take the destination path when using --exclude ...
0
 
LVL 16

Expert Comment

by:ai_ja_nai
ID: 22806904
well, you have to use absolute paths so --exclude /var/www/web*/phptmp is a correct approach, in my opinion. Wildcards may be used also to specify depth of exclusion: --exclude /var/www/** <-- this will exclude everything in two level deep recursive search; *** will exclude in three levels deep and so on.
You finally may use such things like --exclude ignorecase:/usr/[a-z0-9]foo/*/**.py to take advantage of full reg exp matching power. Here we told him to exclude case insensitively any folder in /usr/*foo/*/**.py. Note how we told it to match any alphanumeric chars in foo via [a-z0-9].

But if you just need to exclude specific dirs I belive that specifying  a filelist would be the most easy and effective way
0
Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

 
LVL 21

Author Comment

by:Julian Matz
ID: 22862817
Thanks. Being as these temp directories might change (if users are added or deleted), I wouldn't be able to use a static filelist.

Can I use standard regular expressions?

Would this work for me?

duplicity -v8 /var/www --exclude /var/www/web[0-9]+/phptmp file:///backup/duplicity/var/www
0
 
LVL 16

Expert Comment

by:ai_ja_nai
ID: 22865613
Mhh. Actually does exist the regex support, but in a separate rsync patch under development...
With the options we have now, In a scenario where only the folder "webN" changes, try
 --exclude /var/www/*/phptmp/   <---include always the slash, if it's not a file
This will exclude any folder called phptmp/ two levels below a base directory called /var/www/

We have to use this because pattern expansion with + doesn't match. Only * can match any numer of charachters.
Is this ok?
0
 
LVL 21

Author Comment

by:Julian Matz
ID: 22965930
Thanks! I'm giving this a try now and will let you know how I get on.
0

Featured Post

Microsoft Certification Exam 74-409

Veeam® is happy to provide the Microsoft community with a study guide prepared by MVP and MCT, Orin Thomas. This guide will take you through each of the exam objectives, helping you to prepare for and pass the examination.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
This tutorial will walk an individual through the steps necessary to install and configure the Windows Server Backup Utility. Directly connect an external storage device such as a USB drive, or CD\DVD burner: If the device is a USB drive, ensure i…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

813 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

8 Experts available now in Live!

Get 1:1 Help Now