JPNeron
asked on
rsync giving 'file exists' errors
We're using backuppc for backups, and it's been working fine, more or less. About a month ago, one of the hosts started giving errors during backup. The host is a Windows 7 machine running rsyncd. Here's what the log shows (the first line is the arguments to rsync):
Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive . .
Sent exclude: System?Volume?Information
Sent exclude: $RECYCLE.BIN
Sent exclude: hiberfil.sys
Sent exclude: pagefile.sys
Remote[1]: rsync: opendir "Appx/client/lib" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Dbg" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Em" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Resource
Remote[1]: rsync: opendir "Appx/data/00/0AD/Struct" (in cDrive) failed: File exists (17)
... etc ..
Farther down in the log I see more errors like:
Remote[1]: rsync: send_files failed to open "some file name" (in cDrive): File exists (17)
I've googled around, and I can't figure out what rsync is unhappy about. It seems both 'opendir' and 'send_files' are complaining that 'file exists', but what file? Where? And why now? It's been running for months without this error.
Any help appreciated!
Thanks.
Jean
Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive . .
Sent exclude: System?Volume?Information
Sent exclude: $RECYCLE.BIN
Sent exclude: hiberfil.sys
Sent exclude: pagefile.sys
Remote[1]: rsync: opendir "Appx/client/lib" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Dbg" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Em" (in cDrive) failed: File exists (17)
Remote[1]: rsync: opendir "Appx/data/00/0AD/Resource
Remote[1]: rsync: opendir "Appx/data/00/0AD/Struct" (in cDrive) failed: File exists (17)
... etc ..
Farther down in the log I see more errors like:
Remote[1]: rsync: send_files failed to open "some file name" (in cDrive): File exists (17)
I've googled around, and I can't figure out what rsync is unhappy about. It seems both 'opendir' and 'send_files' are complaining that 'file exists', but what file? Where? And why now? It's been running for months without this error.
Any help appreciated!
Thanks.
Jean
Can you provide the full source / destination / command?
Have you check if 'Appx/client/lib' and others are files or directories?
Have you check if 'Appx/client/lib' and others are files or directories?
Might be a compressed or encrypted file.
We had some files copied over from a Mac give that problem
We had some files copied over from a Mac give that problem
You may be running into a case sensitive filesystem problem or a filename with a / in it. Always fun times with osx
ASKER
mansnrock, I'll try adding the -update flag, although the command has been working fine for months as is, but let's see what happens. Not sure on the version of rsync, the log shows:
incr backup started back to 2013-12-11 20:00:01 (backup #849) for directory cDrive
Connected to lisa-vostro3500:873, remote version 30
Negotiated protocol version 28
Connected to module cDrive
Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive . .
Is 'remote version 30' the rsync version?
Maxdajai - I can't provide the command, only the parameters I have configured it to send, which is the 'Sending args:' line in the log. The log entries are referring to directories. Farther down in the log I get the same error except it says 'send_file' instead of 'opendir', and it refers to actual files.
Wyliecoyoteuk - Nope, not compressed or encrypted files, that hasn't changed in months.
incr backup started back to 2013-12-11 20:00:01 (backup #849) for directory cDrive
Connected to lisa-vostro3500:873, remote version 30
Negotiated protocol version 28
Connected to module cDrive
Sending args: --server --sender --numeric-ids --perms --owner --group -D --links --hard-links --times --block-size=2048 --recursive . .
Is 'remote version 30' the rsync version?
Maxdajai - I can't provide the command, only the parameters I have configured it to send, which is the 'Sending args:' line in the log. The log entries are referring to directories. Farther down in the log I get the same error except it says 'send_file' instead of 'opendir', and it refers to actual files.
Wyliecoyoteuk - Nope, not compressed or encrypted files, that hasn't changed in months.
ASKER
aarontomosky - This isn't a Mac, I'm backing up from a Win 7 machine, using 'backuppc' on Linux. The backup is kept on the Linux filesystem, and has been working fine for months.
Any chance the Linux system is out of room?
All good suggestions. I'd also try a test rsync to a new destination location. That would really nail down if something's funny with the current destination filespec.
- gurutc
- gurutc
ASKER
aarontomosky - lots of space, >300 gig available. The backup software is able to back up other machines to the same location, it's only this one that has trouble.
gurutc - I don't know how to use rsync, I just filled in the parameters using the recommended settings. What sort of test command could I use?
Thanks.
gurutc - I don't know how to use rsync, I just filled in the parameters using the recommended settings. What sort of test command could I use?
Thanks.
Try just using a different destination folder for the backup. Can you change it from cDrive easlily in the interface?
- gurutc
- gurutc
Also, as masnrock recommended, can you add -update to the list of sending args?
- gurutc
- gurutc
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
most likely the problem is that you don't make sure your backup runs only once at a time.
rsync first computes all the operations it has to do and then executes them, so it may determine that a file needs to be created and actually transfer and create the file hours or even days afterwards depending on how long the backup task needs to run.
a frequent problem is that 2 or more instances run at the same time so one of them created the file while the other had listed the files to copy but not performed the actual copy
most likely you where in such a situation, possibly with MANY rsync instances at the same time (since rsync is hard on resources, the more instances running, the longer the backup, and the more instances you get). resetting the password probably killed the running instances so the first backup was able to work properly.
if the above is correct, the same probem will arise again at some point
rsync first computes all the operations it has to do and then executes them, so it may determine that a file needs to be created and actually transfer and create the file hours or even days afterwards depending on how long the backup task needs to run.
a frequent problem is that 2 or more instances run at the same time so one of them created the file while the other had listed the files to copy but not performed the actual copy
most likely you where in such a situation, possibly with MANY rsync instances at the same time (since rsync is hard on resources, the more instances running, the longer the backup, and the more instances you get). resetting the password probably killed the running instances so the first backup was able to work properly.
if the above is correct, the same probem will arise again at some point
ASKER
Interesting theory, but we rebooted both the backup server and the source machine when this first occurred, so any backups/rsync in progress would have been cancelled. The problem still occurred after that.
if you had the problem on the first run, it does rule out this theory. if it happened after a few hours/days, it may just have happened again.
---
this may also help you since it seems to explain while it would not happen every time you run the program
http://stackoverflow.com/questions/10581537/rsync-file-exists-17-error-when-syncing-with-an-encfs-partition
---
this may also help you since it seems to explain while it would not happen every time you run the program
http://stackoverflow.com/questions/10581537/rsync-file-exists-17-error-when-syncing-with-an-encfs-partition
ASKER
It happened on the first run after the reboots.
I saw that link, but it's not a encfs partition. And, remember that this was working fine for months, then the error suddenly started appearing, and now just as mysteriously, it's gone.
I saw that link, but it's not a encfs partition. And, remember that this was working fine for months, then the error suddenly started appearing, and now just as mysteriously, it's gone.
sorry, i'm kind of out of reasonably plausible ideas that are easy to check
for the record, you'd have similar issues with fat32 extended in order to handle file permissions and timestamping as with encfs, but i assume this is not your case
i'd advise you to provide the rsyncd log files (server side) if you have them or possibly enable rsyncd detailed logging to troubleshoot this next time you have the same problems
for the record, you'd have similar issues with fat32 extended in order to handle file permissions and timestamping as with encfs, but i assume this is not your case
i'd advise you to provide the rsyncd log files (server side) if you have them or possibly enable rsyncd detailed logging to troubleshoot this next time you have the same problems
ASKER
Will do, thanks.
ASKER
Problem seemed to go away on it's own.
Each error is telling you the names of directories that already exist, at least based on what you've shared.