Solved

rsync giving 'file exists' errors

Posted on 2013-12-13
19
1,480 Views
Last Modified: 2013-12-22
We're using backuppc for backups, and it's been working fine, more or less. About a month ago, one of the hosts started giving errors during backup. The host is a Windows 7 machine running rsyncd. Here's what the log shows (the first line is the arguments to rsync):

Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive . .
Sent exclude: System?Volume?Information
Sent exclude: $RECYCLE.BIN
Sent exclude: hiberfil.sys
Sent exclude: pagefile.sys
Remote[1]: rsync: opendir "Appx/client/lib" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Dbg" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Em" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Resource" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Struct" (in cDrive) failed: File exists (17)
... etc ..

Farther down in the log I see more errors like:

Remote[1]: rsync: send_files failed to open "some file name" (in cDrive): File exists (17)

I've googled around, and I can't figure out what rsync is unhappy about. It seems both 'opendir' and 'send_files' are complaining that 'file exists', but what file? Where?  And why now? It's been running for months without this error.

Any help appreciated!

Thanks.

Jean
0
Comment
Question by:JPNeron
  • 8
  • 3
  • 3
  • +4
19 Comments
 
LVL 20

Expert Comment

by:masnrock
ID: 39718684
Trying adding the --update flag... which this won't get rid of files that have been deleted, it might help get around the issue of files that already exist. Try it and let us know if you still have trouble. Also, what version of rsync are you using?

Each error is telling you the names of directories that already exist, at least based on what you've shared.
0
 
LVL 21

Expert Comment

by:Mazdajai
ID: 39718786
Can you provide the full source / destination / command?

Have you check if 'Appx/client/lib' and others are files or directories?
0
 
LVL 21

Expert Comment

by:wyliecoyoteuk
ID: 39718801
Might be a compressed or encrypted file.
We had some files copied over from a  Mac  give that problem
0
 
LVL 38

Expert Comment

by:Aaron Tomosky
ID: 39719064
You may be running into a case sensitive filesystem problem or a filename with a / in it. Always fun times with osx
0
 

Author Comment

by:JPNeron
ID: 39719071
mansnrock, I'll try adding the -update flag, although the command has been working fine for months as is, but let's see what happens. Not sure on the version of rsync, the log shows:

incr backup started back to 2013-12-11 20:00:01 (backup #849) for directory cDrive
Connected to lisa-vostro3500:873, remote version 30
Negotiated protocol version 28
Connected to module cDrive
Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive . .

Is 'remote version 30' the rsync version?

Maxdajai - I can't provide the command, only the parameters I have configured it to send, which is the 'Sending args:' line in the log. The log entries are referring to directories. Farther down in the log I get the same error except it says 'send_file' instead of 'opendir', and it refers to actual files.

Wyliecoyoteuk - Nope, not compressed or encrypted files, that hasn't changed in months.
0
 

Author Comment

by:JPNeron
ID: 39719074
aarontomosky - This isn't a Mac, I'm backing up from a Win 7 machine, using 'backuppc' on Linux. The backup is kept on the Linux filesystem, and has been working fine for months.
0
 
LVL 38

Expert Comment

by:Aaron Tomosky
ID: 39719089
Any chance the Linux system is out of room?
0
 
LVL 16

Expert Comment

by:gurutc
ID: 39721233
All good suggestions.  I'd also try a test rsync  to a new destination location.  That would really nail down if something's funny with the current destination filespec.

- gurutc
0
 

Author Comment

by:JPNeron
ID: 39721967
aarontomosky - lots of space, >300 gig available. The backup software is able to back up other machines to the same location, it's only this one that has trouble.

gurutc - I don't know how to use rsync, I just filled in the parameters using the recommended settings. What sort of test command could I use?

Thanks.
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 
LVL 16

Expert Comment

by:gurutc
ID: 39723775
Try just using a different destination folder for the backup.  Can you change it from cDrive easlily in the interface?

- gurutc
0
 
LVL 16

Expert Comment

by:gurutc
ID: 39723781
Also, as masnrock recommended, can you add -update to the list of sending args?

- gurutc
0
 

Accepted Solution

by:
JPNeron earned 0 total points
ID: 39724058
Well, this is interesting... today the errors stopped and the backup was taken normally. The only change I made was to reset the password the backup software uses to connect to the source machine. I don't understand how that got changed in the first place.

I'm starting to think it's a bug in backuppc, maybe it wasn't sending the correct arguments to rsync, in spite of my settings and the log.

Regardless, since it's working again I'll just close this.

Thanks anyway.
0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 39726656
most likely the problem is that you don't make sure your backup runs only once at a time.

rsync first computes all the operations it has to do and then executes them, so it may determine that a file needs to be created and actually transfer and create the file hours or even days afterwards depending on how long the backup task needs to run.

a frequent problem is that 2 or more instances run at the same time so one of them created the file while the other had listed the files to copy but not performed the actual copy

most likely you where in such a situation, possibly with MANY rsync instances at the same time (since rsync is hard on resources, the more instances running, the longer the backup, and the more instances you get). resetting the password probably killed the running instances so the first backup was able to work properly.

if the above is correct, the same probem will arise again at some point
0
 

Author Comment

by:JPNeron
ID: 39726861
Interesting theory, but we rebooted both the backup server and the source machine when this first occurred, so any backups/rsync in progress would have been cancelled. The problem still occurred after that.
0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 39726889
if you had the problem on the first run, it does rule out this theory. if it happened after a few hours/days, it may just have happened again.

---

this may also help you since it seems to explain while it would not happen every time you run the program

http://stackoverflow.com/questions/10581537/rsync-file-exists-17-error-when-syncing-with-an-encfs-partition
0
 

Author Comment

by:JPNeron
ID: 39726908
It happened on the first run after the reboots.

I saw that link, but it's not a encfs partition. And, remember that this was working fine for months, then the error suddenly started appearing, and now just as mysteriously, it's gone.
0
 
LVL 26

Expert Comment

by:skullnobrains
ID: 39727181
sorry, i'm kind of out of reasonably plausible ideas that are easy to check

for the record, you'd have similar issues with fat32 extended in order to handle file permissions and timestamping as with encfs, but i assume this is not your case

i'd advise you to provide the rsyncd log files (server side) if you have them or possibly enable rsyncd detailed logging to troubleshoot this next time you have the same problems
0
 

Author Comment

by:JPNeron
ID: 39727197
Will do, thanks.
0
 

Author Closing Comment

by:JPNeron
ID: 39734436
Problem seemed to go away on it's own.
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Don’t let your business fall victim to the coming apocalypse – use our Survival Guide for the Fax Apocalypse to identify the risks and signs of zombie fax activities at your business.
ADCs have gained traction within the last decade, largely due to increased demand for legacy load balancing appliances to handle more advanced application delivery requirements and improve application performance.
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…
This tutorial will show how to configure a new Backup Exec 2012 server and move an existing database to that server with the use of the BEUtility. Install Backup Exec 2012 on the new server and apply all of the latest hotfixes and service packs. The…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now