Solved

Automated Backups in Linux (Duplicity)

Posted on 2008-10-25
7
1,374 Views
Last Modified: 2013-12-16
Hi!

I'm using Duplicity ( http://duplicity.nongnu.org/ ) to automatically backup my files.

The following command will backup all of my users' website files and preferences (e.g. Spamassassin prefs) to a separate drive:

duplicity -v8 /var/www file:///backup/duplicity/var/www

What I need is to exclude all the temporary files. Temporary files are all inside a directory called "phptmp" and there is one of these directories per user, e.g.

/var/www/web1/phptmp/
/var/www/web2/phptmp/
/var/www/web3/phptmp/
/var/www/web4/phptmp/

My question is: what is the syntax I need to exclude all these temp folders?

0
Comment
Question by:Julian Matz
  • 3
  • 3
7 Comments
 
LVL 16

Accepted Solution

by:
ai_ja_nai earned 500 total points
ID: 22805274
--exclude /destination/dir option does the job. You may use wildcards to generalize your choice.
Also, --exclude-filelist /path/to/file.txt lets you define a plain text file with directories listed inside, separed by returns.
Look for the manual file on duplicity website
0
 
LVL 21

Author Comment

by:Julian Matz
ID: 22806430
Thanks. I have looked through the manual numerous times but couldn't find the answer. Do you know how a wildcard would be applied exactly?
This is where the wildcard would need to be placed (*):

--exclude /var/www/web*/phptmp

Or would it have to be? :

--exclude /web*/phptmp

I didn't know that you need to take the destination path when using --exclude ...
0
 
LVL 16

Expert Comment

by:ai_ja_nai
ID: 22806904
well, you have to use absolute paths so --exclude /var/www/web*/phptmp is a correct approach, in my opinion. Wildcards may be used also to specify depth of exclusion: --exclude /var/www/** <-- this will exclude everything in two level deep recursive search; *** will exclude in three levels deep and so on.
You finally may use such things like --exclude ignorecase:/usr/[a-z0-9]foo/*/**.py to take advantage of full reg exp matching power. Here we told him to exclude case insensitively any folder in /usr/*foo/*/**.py. Note how we told it to match any alphanumeric chars in foo via [a-z0-9].

But if you just need to exclude specific dirs I belive that specifying  a filelist would be the most easy and effective way
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 
LVL 21

Author Comment

by:Julian Matz
ID: 22862817
Thanks. Being as these temp directories might change (if users are added or deleted), I wouldn't be able to use a static filelist.

Can I use standard regular expressions?

Would this work for me?

duplicity -v8 /var/www --exclude /var/www/web[0-9]+/phptmp file:///backup/duplicity/var/www
0
 
LVL 16

Expert Comment

by:ai_ja_nai
ID: 22865613
Mhh. Actually does exist the regex support, but in a separate rsync patch under development...
With the options we have now, In a scenario where only the folder "webN" changes, try
 --exclude /var/www/*/phptmp/   <---include always the slash, if it's not a file
This will exclude any folder called phptmp/ two levels below a base directory called /var/www/

We have to use this because pattern expansion with + doesn't match. Only * can match any numer of charachters.
Is this ok?
0
 
LVL 21

Author Comment

by:Julian Matz
ID: 22965930
Thanks! I'm giving this a try now and will let you know how I get on.
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
VM backup deduplication is a method of reducing the amount of storage space needed to save VM backups. In most organizations, VMs contain many duplicate copies of data, such as VMs deployed from the same template, VMs with the same OS, or VMs that h…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This tutorial will walk an individual through configuring a drive on a Windows Server 2008 to perform shadow copies in order to quickly recover deleted files and folders. Click on Start and then select Computer to view the available drives on the se…

706 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now