We help IT Professionals succeed at work.

Can't get cwrsync over ssh directory exclusions to work

Medium Priority
883 Views
Last Modified: 2013-12-01
Hello all-

Here's my situation-

I have remote users with laptops who are backing up data to a central location server. The laptop clients are Windows XP, a few Vista. The backup server is Windows Server 2003 standard.

On both server and client I have installed CopSSH and cwrsync from www.itefix.no

I have constructed batch files to run either manually initiated backup transfers, or to run as scheduled tasks. This all works really well. Transfer rates are great, etc. I've set up SSH key pairs so users do not have to enter passwords, and I have an exclusion list text file that lives on the server which is pulled down and referenced anew for every transfer.

The rsync command is backing up users' My Documents folder, and in the command, I have a path exclusion to leave out /My Documents/Exclude*

I have an exclusion for file size limit, one for excluding types from a text file, and one to exclude a directory within My Docs. This directory path exclusion will not work.

I've tried many many forms of the syntax, and with ones that are obviously wrong, I get syntax errors like I expect, but versions that are correct or should work, I get no errors or failures, but the exclusion simply doesn't work. It's like the command runs properly, but then ignores itself and sends the excluded directory anyway.

I have been searching the web for several weeks now to find a solution to no avail. I've seen several others online with similar issues who've gotten it to work, but none of the applied fixes worked in my setup.............

Any thoughts or ideas? Any help is greatly appreciated.
Batch File Example, commented line is the one that fails...
 
@ECHO OFF
 
SET CYGWIN=nontsec
SET CWRSYNCHOME=c:\unix\
chdir C:\unix\Bin
c:
"C:\unix\Bin\rsync" -avzr --delete username@xxx.xxx.xxx.xxx:/cygdrive/c/exclude.txt /cygdrive/c/exclude.txt
THIS LINE FAILS TO EXCLUDE THE DIRECTORY>> "C:\unix\Bin\rsync" -avzr --progress --exclude="/cygdrive/c/Documents and Settings/username/My Documents/Exclude/*" --exclude-from="/cygdrive/c/exclude.txt" --max-size=1.5MB-1 "/cygdrive/c/Documents and Settings/username/My Documents/" -e SSH username@xxx.xxx.xxx.xxx:/cygdrive/c/backup_folder
 
PAUSE
 
EXIT

Open in new window

Comment
Watch Question

Add \ before spaces so that this

/cygdrive/c/Documents and Settings/username/My Documents/Exclude/*

becomes this

/cygdrive/c/Documents\ and\ Settings/username/My\ Documents/Exclude/*

If this doesn't work, remove the asterisk at the end of the path

Author

Commented:
Hi-

Thanks for the ideas-

I had tried already with and without the asterisk.

I did try the escape character in the path as you suggested, but that still did not work. It didn't give any errors at all - the command appears to run successfully, but the exclusion just doesn't work...

Any other thoughts?

Thanks again.
what if you simply use --exclude-from option alone?

Author

Commented:
Just using the --exclude-from=exclude.txt works for files. Any file types that are listed there are not transferred- *.exe, *.msi, etc.

I tried adding the directory path to exclude to that text file, and removed the

--exclude=/directory/path/

 line from the batch file.

This didn't work either....

the entries in your file exclusion file (awful word play) shoud look like the snippet attached.

anyway, this

--exclude=/directory/path/

should be this

--exclude='/directory/path/'

and not this

--exclude="/directory/path/"
somefile
somedir/file.*
dir/dir/*

Open in new window

Author

Commented:
I definitely appreciate your help.

But that didn't work either...

This is why I posted to EE. I've been doing as much reading about it as I could find, and it <i>should</i> work. But it doesn't. I get no error codes or anything.

Anyway, the exclusion file types in the text file DO work. I've tested that successfully. Using File Size exclusions works, etc.

Just excluding the directory by path path refuses to work.
Could it have anything to do with the -avzr switches? I didn't think any of those would prevent a child directory exclusion...

Thanks again for your help.
Don't think so. By the way, -r is useless because -a already includes it (actually, -a is -rlptgoD)
Have a look at this if it can help you, particularly the comments (http://articles.slicehost.com/2007/10/10/rsync-exclude-files-and-folders): it seems is a bug

Not the solution you were looking for? Getting a personalized solution is easy.

Ask the Experts

Author

Commented:
Hi-

Yeah, I realized through enough reading of man pages and articles that 'r' was inherent in 'a', I just hadn't taken it out yet as I was mired in this exlusions thing...

In any event, this worked!

What was messing me up was this part: "Note: the directory path is relative to the folder you are backing up."

So since I was backing up the My Docs folder, and the excluded directory was there, all I had to do was add the directory name to the exclude.txt file rsync was reading from.

So now, all exclusion can live in that one text file, which is pulled newly from the server for each backup. Now allows a lot of control over what we are backing up.

Great news!

Thanks again for your help. This really gets me out of a bind.
Access more of Experts Exchange with a free account
Thanks for using Experts Exchange.

Create a free account to continue.

Limited access with a free account allows you to:

  • View three pieces of content (articles, solutions, posts, and videos)
  • Ask the experts questions (counted toward content limit)
  • Customize your dashboard and profile

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.