Solved

rsync giving 'file exists' errors

Posted on 2013-12-13
19
1,670 Views
Last Modified: 2013-12-22
We're using backuppc for backups, and it's been working fine, more or less. About a month ago, one of the hosts started giving errors during backup. The host is a Windows 7 machine running rsyncd. Here's what the log shows (the first line is the arguments to rsync):

Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive . .
Sent exclude: System?Volume?Information
Sent exclude: $RECYCLE.BIN
Sent exclude: hiberfil.sys
Sent exclude: pagefile.sys
Remote[1]: rsync: opendir "Appx/client/lib" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Dbg" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Em" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Resource" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Struct" (in cDrive) failed: File exists (17)
... etc ..

Farther down in the log I see more errors like:

Remote[1]: rsync: send_files failed to open "some file name" (in cDrive): File exists (17)

I've googled around, and I can't figure out what rsync is unhappy about. It seems both 'opendir' and 'send_files' are complaining that 'file exists', but what file? Where?  And why now? It's been running for months without this error.

Any help appreciated!

Thanks.

Jean
0
Comment
Question by:JPNeron
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 8
  • 3
  • 3
  • +4
19 Comments
 
LVL 28

Expert Comment

by:masnrock
ID: 39718684
Trying adding the --update flag... which this won't get rid of files that have been deleted, it might help get around the issue of files that already exist. Try it and let us know if you still have trouble. Also, what version of rsync are you using?

Each error is telling you the names of directories that already exist, at least based on what you've shared.
0
 
LVL 21

Expert Comment

by:Mazdajai
ID: 39718786
Can you provide the full source / destination / command?

Have you check if 'Appx/client/lib' and others are files or directories?
0
 
LVL 21

Expert Comment

by:wyliecoyoteuk
ID: 39718801
Might be a compressed or encrypted file.
We had some files copied over from a  Mac  give that problem
0
Resolve Critical IT Incidents Fast

If your data, services or processes become compromised, your organization can suffer damage in just minutes and how fast you communicate during a major IT incident is everything. Learn how to immediately identify incidents & best practices to resolve them quickly and effectively.

 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 39719064
You may be running into a case sensitive filesystem problem or a filename with a / in it. Always fun times with osx
0
 

Author Comment

by:JPNeron
ID: 39719071
mansnrock, I'll try adding the -update flag, although the command has been working fine for months as is, but let's see what happens. Not sure on the version of rsync, the log shows:

incr backup started back to 2013-12-11 20:00:01 (backup #849) for directory cDrive
Connected to lisa-vostro3500:873, remote version 30
Negotiated protocol version 28
Connected to module cDrive
Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive . .

Is 'remote version 30' the rsync version?

Maxdajai - I can't provide the command, only the parameters I have configured it to send, which is the 'Sending args:' line in the log. The log entries are referring to directories. Farther down in the log I get the same error except it says 'send_file' instead of 'opendir', and it refers to actual files.

Wyliecoyoteuk - Nope, not compressed or encrypted files, that hasn't changed in months.
0
 

Author Comment

by:JPNeron
ID: 39719074
aarontomosky - This isn't a Mac, I'm backing up from a Win 7 machine, using 'backuppc' on Linux. The backup is kept on the Linux filesystem, and has been working fine for months.
0
 
LVL 39

Expert Comment

by:Aaron Tomosky
ID: 39719089
Any chance the Linux system is out of room?
0
 
LVL 16

Expert Comment

by:gurutc
ID: 39721233
All good suggestions.  I'd also try a test rsync  to a new destination location.  That would really nail down if something's funny with the current destination filespec.

- gurutc
0
 

Author Comment

by:JPNeron
ID: 39721967
aarontomosky - lots of space, >300 gig available. The backup software is able to back up other machines to the same location, it's only this one that has trouble.

gurutc - I don't know how to use rsync, I just filled in the parameters using the recommended settings. What sort of test command could I use?

Thanks.
0
 
LVL 16

Expert Comment

by:gurutc
ID: 39723775
Try just using a different destination folder for the backup.  Can you change it from cDrive easlily in the interface?

- gurutc
0
 
LVL 16

Expert Comment

by:gurutc
ID: 39723781
Also, as masnrock recommended, can you add -update to the list of sending args?

- gurutc
0
 

Accepted Solution

by:
JPNeron earned 0 total points
ID: 39724058
Well, this is interesting... today the errors stopped and the backup was taken normally. The only change I made was to reset the password the backup software uses to connect to the source machine. I don't understand how that got changed in the first place.

I'm starting to think it's a bug in backuppc, maybe it wasn't sending the correct arguments to rsync, in spite of my settings and the log.

Regardless, since it's working again I'll just close this.

Thanks anyway.
0
 
LVL 27

Expert Comment

by:skullnobrains
ID: 39726656
most likely the problem is that you don't make sure your backup runs only once at a time.

rsync first computes all the operations it has to do and then executes them, so it may determine that a file needs to be created and actually transfer and create the file hours or even days afterwards depending on how long the backup task needs to run.

a frequent problem is that 2 or more instances run at the same time so one of them created the file while the other had listed the files to copy but not performed the actual copy

most likely you where in such a situation, possibly with MANY rsync instances at the same time (since rsync is hard on resources, the more instances running, the longer the backup, and the more instances you get). resetting the password probably killed the running instances so the first backup was able to work properly.

if the above is correct, the same probem will arise again at some point
0
 

Author Comment

by:JPNeron
ID: 39726861
Interesting theory, but we rebooted both the backup server and the source machine when this first occurred, so any backups/rsync in progress would have been cancelled. The problem still occurred after that.
0
 
LVL 27

Expert Comment

by:skullnobrains
ID: 39726889
if you had the problem on the first run, it does rule out this theory. if it happened after a few hours/days, it may just have happened again.

---

this may also help you since it seems to explain while it would not happen every time you run the program

http://stackoverflow.com/questions/10581537/rsync-file-exists-17-error-when-syncing-with-an-encfs-partition
0
 

Author Comment

by:JPNeron
ID: 39726908
It happened on the first run after the reboots.

I saw that link, but it's not a encfs partition. And, remember that this was working fine for months, then the error suddenly started appearing, and now just as mysteriously, it's gone.
0
 
LVL 27

Expert Comment

by:skullnobrains
ID: 39727181
sorry, i'm kind of out of reasonably plausible ideas that are easy to check

for the record, you'd have similar issues with fat32 extended in order to handle file permissions and timestamping as with encfs, but i assume this is not your case

i'd advise you to provide the rsyncd log files (server side) if you have them or possibly enable rsyncd detailed logging to troubleshoot this next time you have the same problems
0
 

Author Comment

by:JPNeron
ID: 39727197
Will do, thanks.
0
 

Author Closing Comment

by:JPNeron
ID: 39734436
Problem seemed to go away on it's own.
0

Featured Post

Free eBook: Backup on AWS

Everything you need to know about backup and disaster recovery with AWS, for FREE!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

ADCs have gained traction within the last decade, largely due to increased demand for legacy load balancing appliances to handle more advanced application delivery requirements and improve application performance.
Fine Tune your automatic Updates for Ubuntu / Debian
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
Viewers will learn how to connect to a wireless network using the network security key. They will also learn how to access the IP address and DNS server for connections that must be done manually. After setting up a router, find the network security…

737 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question