Solved

Using robocopy or xcopy to copy only missing (extra) files

Posted on 2009-05-18
16
6,532 Views
Last Modified: 2012-05-07
Hello experts

I am using robocopy with the /mir switch to mirror my data onto a NAS drive.  It works beautifully, but if I accidentally delete a file in the source and then run my robocopy script then this file is lost for good.  What I would like to do is identify all extra files (ie all files that exist in the destination but not in the source) and run a first sweep to copy these files somewhere before running the mirror.  In this way my mirror is nice and neat, but if I have accidentally deleted a file then I have it backed up.  Any ideas...I'm starting to think this is not possible and it would seem a fairly obvious thing to want to do.

Many thanks for your help
0
Comment
Question by:RichardGrieveson
  • 5
  • 4
  • 3
  • +2
16 Comments
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24411970
Hi

I've used a synchronization program called Foldermatch when it comes to synchronization issues like this.  Here's the link:

http://www.foldermatch.com/

I'm sure you'd like to have something that you can schedule.  I'll look at the options and get back.

Check it out in the mean time.
0
 

Author Comment

by:RichardGrieveson
ID: 24412139
Thanks for the link...I guess I'll go down this route if its not possible to script
0
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24412178
Hey ..

I'm also looking at the options for Robocopy and others.  I'll get back with a response on the scripting question.

You may want to install FolderMatch and check it out.  I believe there is a 30 day trial available.  
0
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24412406
Hi

Looking at some of the command line options, I note this example about 3/4 of the way down the page:

http://www.ss64.com/nt/robocopy.html

where it mentions automatically skipping files that already exist.  I also see the capability to log the skipped files.  Am I reading the example right?

Let me know what you think.
0
 
LVL 14

Expert Comment

by:igor-1965
ID: 24412598
If you run the following command it will list all extra files in destination:
/L parameter just list the files so you could use it to identify the files that missing in source but present in destination.
 
Hope it helps

robocopy <sourse> <destination> /MIR /NP /L | find /I "extra "

Open in new window

0
 

Author Comment

by:RichardGrieveson
ID: 24412695

Great..thanks...I would then somehow have to parse this list and iterate through copying to a folder?
0
 
LVL 14

Assisted Solution

by:uucknaaa
uucknaaa earned 100 total points
ID: 24412911
Hey ..

Here's a link to a script/batch to copy existing files to a folder:

http://www.computing.net/answers/dos/batch-help-copy-files-with-rename/16874.html

You could run a script like this before executing the backup.

Let me know if not acceptable.
0
 
LVL 68

Accepted Solution

by:
Qlemo earned 200 total points
ID: 24413692
How about this. The used folders for are in the first three set's, and if you omit the robocopy /mir at the end, no harm can be done.


@echo off

setlocal EnableDelayedExpansion
 

set srcdir=c:\source

set mirror=d:\mirror\recent

set copydir=d:\mirror\deleted
 

for /F "tokens=3*" %%F in ('robocopy /xc /xn /xo /NP /NS /NJH /NJS /FP /L /s /ndl %srcdir% %mirror% ^| find /v "New"') do (

  set file=%%F%%G

  xcopy !file! !file:%mirror%=%copydir%!

)
 

robocopy /mir ...

Open in new window

0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 
LVL 14

Expert Comment

by:igor-1965
ID: 24414398
I created a script very close to above.
You could combine all pieces with your existing command to a single batch.

Cheers

1st. Run the command to identify extra files and save it to EXTRAS.LST:
 

robocopy <sourse> <destination> /MIR /NP /L | find /I "extra " > EXTRAS.LST
 

2nd. Run the following batch that parses EXTRAS.LST and copy extra file aside to C:\EXTRAS (you could change where it must be copied to):
 

@echo off

setlocal enabledelayedexpansion

(for /f "tokens=*" %%a in ('type EXTRAS.LST') do (

   set line=%%a

   XCOPY /S "!line:~23,254!" C:\EXTRAS

))

#echo on
 

3rd. You run your usual ROBOCOPY /MIR batch

Open in new window

0
 
LVL 68

Expert Comment

by:Qlemo
ID: 24414429
igor,

you don't need a type. Just use   for /F "tokens=*" %%a  in (extras.lst) do
The lines contain the words "Extra File", so I assume your xcopy is not working.

0
 
LVL 14

Assisted Solution

by:igor-1965
igor-1965 earned 200 total points
ID: 24414769
Nope, it's fully tested and it is working fine.
You are absolutely right - "type" is not needed so I reposting the amended script.

1st. Run the command to identify extra files and save it to EXTRAS.LST:

 

robocopy <sourse> <destination> /MIR /NP /L | find /I "extra " > EXTRAS.LST

 

2nd. Run the following batch that parses EXTRAS.LST and copy extra file aside to C:\EXTRAS (you could change where it must be copied to):

 

@echo off

setlocal enabledelayedexpansion

(for /f "tokens=*" %%a in (EXTRAS.LST) do (

   set line=%%a

   XCOPY /S "!line:~23,254!" C:\EXTRAS

))

@echo on

 

3rd. You run your usual ROBOCOPY /MIR batch

Open in new window

0
 
LVL 68

Expert Comment

by:Qlemo
ID: 24414869
Ah, sorry, didn't see the "!line:~23,254!". But this lilne is giving me two issues:
1) The file size is included (it start at pos. 24). The file name starts at pos. 49 (48 in ~ syntax).
2) the summary line is included, resulting in a file name
   "   Copied   Skipped  Mismatch    FAILED    Extras"

0
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24415444
Richard,

What do you think about this.  Have you got enough information to come up with a single script that will accomplish your goal?  The input from Igor and Olemo is great.  If so, could I ask to see your final script?  Your initial question is something I've had to deal with many times before when manageing backups.

Thanks
0
 

Author Closing Comment

by:RichardGrieveson
ID: 31582584
Hello Guys

Thanks very much...great stuff
0
 
LVL 68

Expert Comment

by:Qlemo
ID: 24417545
For future grading:
"A"   means  "fits exactly", or "fits almost"
"B"  means "was helpful, but I had some work to do myself"
"C"  means "gave me some info, but was only helpful only in small parts"

In this case, this would be a grade "A" for all.

If you would like to adjust the grading here, which would be welcome but not necessary, press "Request attention" in your question, provide reason like "Pleae reopen, want to give better grades"  or the like, and after a Admin reopened the question you can Accept again.
0
 

Expert Comment

by:DrJarmin
ID: 24644443
This looks like a great solution to a similar issue I am having.

Can you tell me - how do you execute the script above?  I have just got myself a copy of Robocopy but haven't used it yet.

Thanks!
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Using dates in 'DOS' batch files has always been tricky as it has no built in ways of extracting date information.  There are many tricks using string manipulation to pull out parts of the %date% variable or output of the date /t command but these r…
YESTERDAY YESTERDAY.BAT is inspired by a previous article I wrote entitled: TOMORROW.BAT (http://www.experts-exchange.com/OS/Microsoft_Operating_Systems/MS_DOS/A_4196-Advanced-Batch-File-Programming-TOMORROW-BAT.html). The crux of this batch f…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now