Solved

Using robocopy or xcopy to copy only missing (extra) files

Posted on 2009-05-18
16
6,771 Views
Last Modified: 2012-05-07
Hello experts

I am using robocopy with the /mir switch to mirror my data onto a NAS drive.  It works beautifully, but if I accidentally delete a file in the source and then run my robocopy script then this file is lost for good.  What I would like to do is identify all extra files (ie all files that exist in the destination but not in the source) and run a first sweep to copy these files somewhere before running the mirror.  In this way my mirror is nice and neat, but if I have accidentally deleted a file then I have it backed up.  Any ideas...I'm starting to think this is not possible and it would seem a fairly obvious thing to want to do.

Many thanks for your help
0
Comment
Question by:RichardGrieveson
  • 5
  • 4
  • 3
  • +2
16 Comments
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24411970
Hi

I've used a synchronization program called Foldermatch when it comes to synchronization issues like this.  Here's the link:

http://www.foldermatch.com/

I'm sure you'd like to have something that you can schedule.  I'll look at the options and get back.

Check it out in the mean time.
0
 

Author Comment

by:RichardGrieveson
ID: 24412139
Thanks for the link...I guess I'll go down this route if its not possible to script
0
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24412178
Hey ..

I'm also looking at the options for Robocopy and others.  I'll get back with a response on the scripting question.

You may want to install FolderMatch and check it out.  I believe there is a 30 day trial available.  
0
Courses: Start Training Online With Pros, Today

Brush up on the basics or master the advanced techniques required to earn essential industry certifications, with Courses. Enroll in a course and start learning today. Training topics range from Android App Dev to the Xen Virtualization Platform.

 
LVL 14

Expert Comment

by:uucknaaa
ID: 24412406
Hi

Looking at some of the command line options, I note this example about 3/4 of the way down the page:

http://www.ss64.com/nt/robocopy.html

where it mentions automatically skipping files that already exist.  I also see the capability to log the skipped files.  Am I reading the example right?

Let me know what you think.
0
 
LVL 14

Expert Comment

by:igor-1965
ID: 24412598
If you run the following command it will list all extra files in destination:
/L parameter just list the files so you could use it to identify the files that missing in source but present in destination.
 
Hope it helps

robocopy <sourse> <destination> /MIR /NP /L | find /I "extra "

Open in new window

0
 

Author Comment

by:RichardGrieveson
ID: 24412695

Great..thanks...I would then somehow have to parse this list and iterate through copying to a folder?
0
 
LVL 14

Assisted Solution

by:uucknaaa
uucknaaa earned 100 total points
ID: 24412911
Hey ..

Here's a link to a script/batch to copy existing files to a folder:

http://www.computing.net/answers/dos/batch-help-copy-files-with-rename/16874.html

You could run a script like this before executing the backup.

Let me know if not acceptable.
0
 
LVL 69

Accepted Solution

by:
Qlemo earned 200 total points
ID: 24413692
How about this. The used folders for are in the first three set's, and if you omit the robocopy /mir at the end, no harm can be done.


@echo off
setlocal EnableDelayedExpansion
 
set srcdir=c:\source
set mirror=d:\mirror\recent
set copydir=d:\mirror\deleted
 
for /F "tokens=3*" %%F in ('robocopy /xc /xn /xo /NP /NS /NJH /NJS /FP /L /s /ndl %srcdir% %mirror% ^| find /v "New"') do (
  set file=%%F%%G
  xcopy !file! !file:%mirror%=%copydir%!
)
 
robocopy /mir ...

Open in new window

0
 
LVL 14

Expert Comment

by:igor-1965
ID: 24414398
I created a script very close to above.
You could combine all pieces with your existing command to a single batch.

Cheers

1st. Run the command to identify extra files and save it to EXTRAS.LST:
 
robocopy <sourse> <destination> /MIR /NP /L | find /I "extra " > EXTRAS.LST
 
2nd. Run the following batch that parses EXTRAS.LST and copy extra file aside to C:\EXTRAS (you could change where it must be copied to):
 
@echo off
setlocal enabledelayedexpansion
(for /f "tokens=*" %%a in ('type EXTRAS.LST') do (
   set line=%%a
   XCOPY /S "!line:~23,254!" C:\EXTRAS
))
#echo on
 
3rd. You run your usual ROBOCOPY /MIR batch

Open in new window

0
 
LVL 69

Expert Comment

by:Qlemo
ID: 24414429
igor,

you don't need a type. Just use   for /F "tokens=*" %%a  in (extras.lst) do
The lines contain the words "Extra File", so I assume your xcopy is not working.

0
 
LVL 14

Assisted Solution

by:igor-1965
igor-1965 earned 200 total points
ID: 24414769
Nope, it's fully tested and it is working fine.
You are absolutely right - "type" is not needed so I reposting the amended script.

1st. Run the command to identify extra files and save it to EXTRAS.LST:
 
robocopy <sourse> <destination> /MIR /NP /L | find /I "extra " > EXTRAS.LST
 
2nd. Run the following batch that parses EXTRAS.LST and copy extra file aside to C:\EXTRAS (you could change where it must be copied to):
 
@echo off
setlocal enabledelayedexpansion
(for /f "tokens=*" %%a in (EXTRAS.LST) do (
   set line=%%a
   XCOPY /S "!line:~23,254!" C:\EXTRAS
))
@echo on
 
3rd. You run your usual ROBOCOPY /MIR batch

Open in new window

0
 
LVL 69

Expert Comment

by:Qlemo
ID: 24414869
Ah, sorry, didn't see the "!line:~23,254!". But this lilne is giving me two issues:
1) The file size is included (it start at pos. 24). The file name starts at pos. 49 (48 in ~ syntax).
2) the summary line is included, resulting in a file name
   "   Copied   Skipped  Mismatch    FAILED    Extras"

0
 
LVL 14

Expert Comment

by:uucknaaa
ID: 24415444
Richard,

What do you think about this.  Have you got enough information to come up with a single script that will accomplish your goal?  The input from Igor and Olemo is great.  If so, could I ask to see your final script?  Your initial question is something I've had to deal with many times before when manageing backups.

Thanks
0
 

Author Closing Comment

by:RichardGrieveson
ID: 31582584
Hello Guys

Thanks very much...great stuff
0
 
LVL 69

Expert Comment

by:Qlemo
ID: 24417545
For future grading:
"A"   means  "fits exactly", or "fits almost"
"B"  means "was helpful, but I had some work to do myself"
"C"  means "gave me some info, but was only helpful only in small parts"

In this case, this would be a grade "A" for all.

If you would like to adjust the grading here, which would be welcome but not necessary, press "Request attention" in your question, provide reason like "Pleae reopen, want to give better grades"  or the like, and after a Admin reopened the question you can Accept again.
0
 

Expert Comment

by:DrJarmin
ID: 24644443
This looks like a great solution to a similar issue I am having.

Can you tell me - how do you execute the script above?  I have just got myself a copy of Robocopy but haven't used it yet.

Thanks!
0

Featured Post

Live: Real-Time Solutions, Start Here

Receive instant 1:1 support from technology experts, using our real-time conversation and whiteboard interface. Your first 5 minutes are always free.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Using dates in 'DOS' batch files has always been tricky as it has no built in ways of extracting date information.  There are many tricks using string manipulation to pull out parts of the %date% variable or output of the date /t command but these r…
One of my most closely kept secrets is revealed in this discussion How to output text on the same line This question was recently posted in EE by Simon336697 (http://www.experts-exchange.com/Programming/Languages/Scripting/Shell/Batch/Q_2459…
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …
Nobody understands Phishing better than an anti-spam company. That’s why we are providing Phishing Awareness Training to our customers. According to a report by Verizon, only 3% of targeted users report malicious emails to management. With compan…

776 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question