?
Solved

Using robocopy or xcopy to copy only missing (extra) files

Posted on 2009-05-18
16
Medium Priority
?
7,541 Views
Last Modified: 2012-05-07
Hello experts

I am using robocopy with the /mir switch to mirror my data onto a NAS drive.  It works beautifully, but if I accidentally delete a file in the source and then run my robocopy script then this file is lost for good.  What I would like to do is identify all extra files (ie all files that exist in the destination but not in the source) and run a first sweep to copy these files somewhere before running the mirror.  In this way my mirror is nice and neat, but if I have accidentally deleted a file then I have it backed up.  Any ideas...I'm starting to think this is not possible and it would seem a fairly obvious thing to want to do.

Many thanks for your help
0
Comment
Question by:RichardGrieveson
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
  • 3
  • +2
16 Comments
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24411970
Hi

I've used a synchronization program called Foldermatch when it comes to synchronization issues like this.  Here's the link:

http://www.foldermatch.com/

I'm sure you'd like to have something that you can schedule.  I'll look at the options and get back.

Check it out in the mean time.
0
 

Author Comment

by:RichardGrieveson
ID: 24412139
Thanks for the link...I guess I'll go down this route if its not possible to script
0
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24412178
Hey ..

I'm also looking at the options for Robocopy and others.  I'll get back with a response on the scripting question.

You may want to install FolderMatch and check it out.  I believe there is a 30 day trial available.  
0
Prepare for your VMware VCP6-DCV exam.

Josh Coen and Jason Langer have prepared the latest edition of VCP study guide. Both authors have been working in the IT field for more than a decade, and both hold VMware certifications. This 163-page guide covers all 10 of the exam blueprint sections.

 
LVL 14

Expert Comment

by:uucknaaa
ID: 24412406
Hi

Looking at some of the command line options, I note this example about 3/4 of the way down the page:

http://www.ss64.com/nt/robocopy.html

where it mentions automatically skipping files that already exist.  I also see the capability to log the skipped files.  Am I reading the example right?

Let me know what you think.
0
 
LVL 14

Expert Comment

by:igor-1965
ID: 24412598
If you run the following command it will list all extra files in destination:
/L parameter just list the files so you could use it to identify the files that missing in source but present in destination.
 
Hope it helps

robocopy <sourse> <destination> /MIR /NP /L | find /I "extra "

Open in new window

0
 

Author Comment

by:RichardGrieveson
ID: 24412695

Great..thanks...I would then somehow have to parse this list and iterate through copying to a folder?
0
 
LVL 14

Assisted Solution

by:uucknaaa
uucknaaa earned 300 total points
ID: 24412911
Hey ..

Here's a link to a script/batch to copy existing files to a folder:

http://www.computing.net/answers/dos/batch-help-copy-files-with-rename/16874.html

You could run a script like this before executing the backup.

Let me know if not acceptable.
0
 
LVL 70

Accepted Solution

by:
Qlemo earned 600 total points
ID: 24413692
How about this. The used folders for are in the first three set's, and if you omit the robocopy /mir at the end, no harm can be done.


@echo off
setlocal EnableDelayedExpansion
 
set srcdir=c:\source
set mirror=d:\mirror\recent
set copydir=d:\mirror\deleted
 
for /F "tokens=3*" %%F in ('robocopy /xc /xn /xo /NP /NS /NJH /NJS /FP /L /s /ndl %srcdir% %mirror% ^| find /v "New"') do (
  set file=%%F%%G
  xcopy !file! !file:%mirror%=%copydir%!
)
 
robocopy /mir ...

Open in new window

0
 
LVL 14

Expert Comment

by:igor-1965
ID: 24414398
I created a script very close to above.
You could combine all pieces with your existing command to a single batch.

Cheers

1st. Run the command to identify extra files and save it to EXTRAS.LST:
 
robocopy <sourse> <destination> /MIR /NP /L | find /I "extra " > EXTRAS.LST
 
2nd. Run the following batch that parses EXTRAS.LST and copy extra file aside to C:\EXTRAS (you could change where it must be copied to):
 
@echo off
setlocal enabledelayedexpansion
(for /f "tokens=*" %%a in ('type EXTRAS.LST') do (
   set line=%%a
   XCOPY /S "!line:~23,254!" C:\EXTRAS
))
#echo on
 
3rd. You run your usual ROBOCOPY /MIR batch

Open in new window

0
 
LVL 70

Expert Comment

by:Qlemo
ID: 24414429
igor,

you don't need a type. Just use   for /F "tokens=*" %%a  in (extras.lst) do
The lines contain the words "Extra File", so I assume your xcopy is not working.

0
 
LVL 14

Assisted Solution

by:igor-1965
igor-1965 earned 600 total points
ID: 24414769
Nope, it's fully tested and it is working fine.
You are absolutely right - "type" is not needed so I reposting the amended script.

1st. Run the command to identify extra files and save it to EXTRAS.LST:
 
robocopy <sourse> <destination> /MIR /NP /L | find /I "extra " > EXTRAS.LST
 
2nd. Run the following batch that parses EXTRAS.LST and copy extra file aside to C:\EXTRAS (you could change where it must be copied to):
 
@echo off
setlocal enabledelayedexpansion
(for /f "tokens=*" %%a in (EXTRAS.LST) do (
   set line=%%a
   XCOPY /S "!line:~23,254!" C:\EXTRAS
))
@echo on
 
3rd. You run your usual ROBOCOPY /MIR batch

Open in new window

0
 
LVL 70

Expert Comment

by:Qlemo
ID: 24414869
Ah, sorry, didn't see the "!line:~23,254!". But this lilne is giving me two issues:
1) The file size is included (it start at pos. 24). The file name starts at pos. 49 (48 in ~ syntax).
2) the summary line is included, resulting in a file name
   "   Copied   Skipped  Mismatch    FAILED    Extras"

0
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24415444
Richard,

What do you think about this.  Have you got enough information to come up with a single script that will accomplish your goal?  The input from Igor and Olemo is great.  If so, could I ask to see your final script?  Your initial question is something I've had to deal with many times before when manageing backups.

Thanks
0
 

Author Closing Comment

by:RichardGrieveson
ID: 31582584
Hello Guys

Thanks very much...great stuff
0
 
LVL 70

Expert Comment

by:Qlemo
ID: 24417545
For future grading:
"A"   means  "fits exactly", or "fits almost"
"B"  means "was helpful, but I had some work to do myself"
"C"  means "gave me some info, but was only helpful only in small parts"

In this case, this would be a grade "A" for all.

If you would like to adjust the grading here, which would be welcome but not necessary, press "Request attention" in your question, provide reason like "Pleae reopen, want to give better grades"  or the like, and after a Admin reopened the question you can Accept again.
0
 

Expert Comment

by:DrJarmin
ID: 24644443
This looks like a great solution to a similar issue I am having.

Can you tell me - how do you execute the script above?  I have just got myself a copy of Robocopy but haven't used it yet.

Thanks!
0

Featured Post

Efficient way to get backups off site to Azure

This user guide provides instructions on how to deploy and configure both a StoneFly Scale Out NAS Enterprise Cloud Drive virtual machine and Veeam Cloud Connect in the Microsoft Azure Cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

VALIDATING DATES One method of validating dates is to jam the date into the DATE command and see if it accepts it by examining the system's errorlevel value. A non-zero result indicates failure. A typical example might look something like the fol…
YESTERDAY YESTERDAY.BAT is inspired by a previous article I wrote entitled: TOMORROW.BAT (http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/MS_DOS/A_4196-Advanced-Batch-File-Programming-TOMORROW-BAT.html). The crux of this batch f…
Michael from AdRem Software explains how to view the most utilized and worst performing nodes in your network, by accessing the Top Charts view in NetCrunch network monitor (https://www.adremsoft.com/). Top Charts is a view in which you can set seve…
Add bar graphs to Access queries using Unicode block characters. Graphs appear on every record in the color you want. Give life to numbers. Hopes this gives you ideas on visualizing your data in new ways ~ Create a calculated field in a query: …

741 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question