Rsync to list files that did not get copied to destination directory

rsync -avzh path/to/directory/A/ path/to/directory/B

The above commands copies everything under directory A to directory B. Sometimes during the copy the files in destination B are locked by some other process which makes the copy incomplete. Is there a way in Rsync to list all such locked files that did not get copied over to the destination.
gaurav sharmaAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

woolmilkporcCommented:
Add "--dry-run" to the options of your command.

rsync will tell you which files are still to be copied without actually transferring anything.
0
gaurav sharmaAuthor Commented:
Dry run gives me the list of all the objects that get copied to the destination.
I only need the list of files that are not transferred from source.
0
woolmilkporcCommented:
Since rsync (the way you use it) will copy only files which do not yet exist (or are newer than those) on the destination "--dry-run" should do what you desire. In other words, "--dry-run" will list only files or file versions that have not been transferred from source yet.
If you don't want to see files which have been created or modified in the meantime (after the failed run but before "--dry-run") on the source I fear you will have to rely on the error log of the previous (failed) run in question to find out the non-copied files.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Gerwin Jansen, EE MVETopic Advisor Commented:
Try adding a log file and check that you've run the rsync command:

--log-file=FILE
0
SandyCommented:
Gaurav.. but i would suggest doing the rsync on open files won't be done.. i tried it numerous times but always it has a need of downtime to get the migration done.

I would suggest take the downtime with app owner and do the rsync, else open or locked files will always be acting like this only.

Ty/SA
0
gaurav sharmaAuthor Commented:
Thank you for all the comments. I have a folder of size 4 gb that needs to be copied to a location. It should overwrite everything each time on the destination. The copy fails many a times since the files in the destination are locked. I need something that would tell me which files could not be copied due to that encountered lock.
0
gaurav sharmaAuthor Commented:
The copy can be done using cygwin or windows.
0
woolmilkporcCommented:
Consult the logfile, I keep telling you.
0
serialbandCommented:
I concur with woolmilkporc.  Consult the log file.  You'll have to generate it.

It sounds like you're running rsync on a Windows system.  In the Linux/Unix world, files do not get locked that way.  They are locked by inodes and file names are overwritten with new files with different inodes.  The old files may still be running, but the file names are replaced.  You only need to restart the process for it to disappear and load the new file.


On Windows, you can't ever overwrite the locked files because those are locked by the file names.  No matter which method you use to copy to a locked file, you can't overwrite it.  You also can't load it to read.  The only workaround for reading a locked file for copying is to enable VSS.  You can then use an open sourced 3rd party tool called hobocopy to make the copy.  http://hobocopy.sourceforge.net/ https://candera.github.io/hobocopy/

Unless you're copying from a linux or unix box to your windows box, it's better to use robocopy or hobocopy if you want to presevere windows ACLs.  rsync is better for unix copies.  Even with the windows tools, you need to consult the log file.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.