Robocopy to copy only Changed / New Files ?

I'm afraid you can't do that. ROBOCOPY is an utility for mantaining a "mirrored" copy of a directory structure. The command structure is <source> ---> <destination>. Everything is happening between these two parameters (check between them, copy/move/or delete between them).

u can integrate a validation in your script to check if file has changed and then perform the robocopy as usual.

sedgwick ? any hint on how to do that ?

XXCOPY can do that easily for you
www.xxcopy.com/

for sync operation of updated files
http://www.xxcopy.com/xxgroup/m03/msg03254.htm

hope this helps

>>I have around 300Gb of Data on my Server that I need to backup and keep daily copies for a 7 days (Company's Policy).

do u need to delete them after 7 days?

nope. just re-update

can you tell me if this is what you are looking for.

you have 300GB data in one location and you want to back it up in another location to cycle the backup over the 7 days of week? is that means you will have 300GB backup copy in 7 archives based on Weekday Name? (eg. SAT, SUN, MON)

just simply add /MIR switch to copy ONLY the changes.

http://ss64.com/nt/robocopy.html

nbass668, the thing is as follows :
suppose I have 10 files to backup file1....file10

Now day0 I will backup up all files file1...file10 in a folder <ALL>
Now day1 (monday) I will backup the files that changed on monday to a folder <MON>
so if only 1 file was changed , the MON folder will have 1 file only

Now day2 (tuesday) I will backup the files that changed on tuesday to a folder <TUE>
so if only 3 file was changed , the TUE folder will have 3 files only

so on wednesday my whole data would be the <ALL> folder + <WED> folder.

etc..

find below a batch file which simply does the following.

Check which day it is Today! example MON or TUE or SUN etc.
Check for new modified files for the source directory and copy only the new modified to a folder corresponding to the day of week

if today is Monday... the copy folder will be C:\backup\MON\

I used the windows command XCOPY, you can use instead any similar copy commnads

I have tested the batch file it worked....

let me know your thoughts

@ECHO OFF
:: Find which day it is today (eg. Mon, Tue, Wed etc.)
for /F %%A in ('date /t') do set DAY=%%A

:: Use XCOPY command to find files modified today and copy them to the specified date folder
for /F "tokens=2-4* delims=/ " %%A in ('DATE/T') do set MMDDYY=%%A-%%B-%%C
XCOPY c:\test\*.* c:\backup\%DAY% /D:%MMDDYY% /S /C
PAUSE

Select allOpen in new window

nbass668 thanks for the script but it doesn't do exactly what I need mainly because I can't rely on the modified date !!!
What if the user gets an old file from somewhere else and copies it to his folder ?
that file would never be backed up !!

@ammounpierre;

yeah indeed you are correct. and how I had the big picture of what you facing.

however, it seems your request is kind of complicated.

but, I have two solutions. let me know which one is more close to your request.

Solution 1.
- In order to achieve your request, you will need 8 folders/locations for backup.
- an <ALL> folder that will maintain a clone/sync copy of the destination.. coz this is the only way to compare and find new changed/updated/added files from two directories.
- once compare is complete... the batch file will move the new files from the source to the <ALL> folder to keep it updated and on the same time... move those same specific files to a <DAY> folder. such as MON, TUE, SUN etc.
- this way... the batch file will always sync the destination folder with <ALL> folder and maintain the file changes over 7 folders representing days of week. and cycle them weekly.

Solution 2.
if you dont wish to maintain <ALL> folder daily.
instead of comparing the destination folder with <ALL> folder.. you can compare the destination folder with a database file that gets updated daily with all the folder structure, file sizes, modified date etc. to find out new changes and updates.. and those changes will be copied to the 7 folders representing days of week. and cycle them weekly. and perhaps maintain a full copy only on Mondays if you prefer.

however "Solution 2" does not seem achievable with a simple batch file.. and perhaps you will need a much more advanced scripting... this is my humble opinion.

as for "solution 1".. I think we can easily do it in a batch file

please advice...

nbass668 thank you solution 1 would be great.
So that we don't get confused... here what would happen.

we have the folder <SOURCE> that we need to sync with <DESTINATION>
the batch would do the following :

1- SYNC between <SOURCE> and <DESTINATION> (always 1 way from Source to Destination)
2-MOVE or COPY the new files to <MON> on monday  ....<TUE> on Tuesday...etc...

Thanks

the trick is, to log all new/changed files in to a log file then step into the log file and individually copy each new file to the DAY folder.

however, I was not able to achieve this with ROBOCOPY. as its LOG file or screen output is filled with so many unwanted spaces and characters.

in details;
so I have used XCOPY which output for me only the new and changed files between the source and destination in to a LOG file... XCOPY will not do any copy but to report new changes... I used your own ROBOCOPY command to take care of updating the destination as it is more robust and effective.

once ROBOCOPY finish updating the destination and source folder... XCOPY will step again into the log file and individually copy the new changed files to MON folder or SUN folder (which ever the day falls into)

I have tested it and it worked perfectly.

this method is not bullet proof, try to test it and let me know your concerns.

if you get the idea... you can further customize it or someone can shed more light and make the batch file much more effective

@ECHO OFF

REM Set to current working directory
CD /d %~dp0

REM Find which day it is today (eg. Mon, Tue, Wed etc.)
for /F %%A in ('date /t') do set DAY=%%A

REM Compare the two destination dirctory and [ALL] folder (this command will ONLY output changed/new files to a log file)
XCOPY "C:\DestinationFolder" "C:\backup\[ALL]" /D /E /L /Y /C > logme.txt

REM Copy new files
ROBOCOPY "C:\DestinationFolder"  "C:\backup\[ALL]" *.* /TEE /E /R:5 /W:15 /FFT /NP 

REM Read successive lines from the log file and copy the new files into specific DAY folder
for /f "delims=] tokens=1*" %%a in ('find /v /n "" ^<logme.txt') do (
   IF EXIST "%%b" XCOPY "%%b" "C:\backup\%DAY%" /C /Y
)

PAUSE
EXIT

Select allOpen in new window

nbass668 sorry for the late comments... but it is not working...

here is what I did to test it

I created 2 folders <SOURCE> and <DESTINATION>


Now in the <DESTINATION> folder I Created the following subfolders
<ALL> <Mon> <Tue> <Wed> <Thu> <Fri> <Sat> <Sun>

Now in the source I put 5 files...

Now I run the script but modified it a bit
**this line is giving me an error...So I removed it and put the batch file in the <SOURCE> folder
CD /d %~dp0


HERE IS THE SCRIPT I RUN
**************************
@ECHO OFF

REM Set to current working directory


REM Find which day it is today (eg. Mon, Tue, Wed etc.)
for /F %%A in ('date /t') do set DAY=%%A

REM Compare the two destination dirctory and [ALL] folder (this command will ONLY output changed/new files to a log file)
XCOPY "C:\SOURCE" "C:\DESTINATION\ALL" /D /E /L /Y /C > logme.txt

REM Copy new files
ROBOCOPY "C:\SOURCE"  "C:\DESTINATION\ALL" *.* /TEE /E /R:5 /W:15 /FFT /NP

REM Read successive lines from the log file and copy the new files into specific DAY folder
for /f "delims=] tokens=1*" %%a in ('find /v /n "" ^<logme.txt') do (
   IF EXIST "%%b" XCOPY "%%b" "C:\DESTINATION\%DAY%" /C /Y
)

PAUSE
EXIT

******************************************************


MY PROBLEMS ARE AS FOLLOWS:
1-it is xcopying... instead of robocopy ...so it is much much slower
2-what if I run it more than once in the same day ? It will re-copy again all files for today ?


thanks

The reason I used xcopy because I had issues logging the new changes to a logfile. however, with a bit of asking I found the solution and there is no need to use Xcopy

I have used multiple FOR loops to read the log file then to clean the log file from unwanted spaces and finally to parse the filenames to ROBOCOPY again to copy the new changed files to the respective DAY folder

you can run the batch file multiple times and if ROBOCOPY did not find any changes then it will simply skip.

please note the extra switches for ROBOCOPY and the log file (logtxt.txt) should always be stored in the same folder the batch file running or else this process

I have tested this batch and works..

please test from your end

@ECHO OFF

REM Find which day it is today (eg. Mon, Tue, Wed etc.)
for /F %%A in ('date /t') do set DAY=%%A

REM Copy new files
ROBOCOPY "C:\SOURCE"  "C:\DESTINATION\ALL" *.* /TEE /E /R:5 /W:15 /FFT /NP /LOG:logtxt.txt /NS /NC /NJH /NJS /NDL

REM Clean up the log file and Read successive lines from the log file and copy the new files into specific DAY folder
for /f "delims=] tokens=1*" %%a in ('find /v /n "" ^<logtxt.txt') do (
for /f "tokens=*" %%s in ("%%b") do (
for %%a in ("%%s") do ROBOCOPY "C:\SOURCE" "C:\DESTINATION\%DAY%" "%%~nxa" /TEE /R:5 /W:15 /NP /NS /NC /NJH /NJS /NDL
)
)
PAUSE
EXIT

Select allOpen in new window

please ignore the above batch as I found a small bug that if you have multiple sub folders the files inside them may not get copied to the DAY folder

@ECHO OFF

REM Find which day it is today (eg. Mon, Tue, Wed etc.)
for /F %%A in ('date /t') do set DAY=%%A

REM Copy new files
ROBOCOPY "C:\SOURCE"  "C:\DESTINATION\ALL" *.* /TEE /E /R:5 /W:15 /FFT /NP /LOG:logtxt.txt /NS /NC /NJH /NJS /NDL

REM Clean up the log file and Read successive lines from the log file and copy the new files into specific DAY folder
for /f "delims=] tokens=1*" %%a in ('find /v /n "" ^<logtxt.txt') do (
for /f "tokens=*" %%s in ("%%b") do (
for %%a in ("%%s") do ROBOCOPY %%~da%%~pa "C:\DESTINATION\%DAY%" "%%~nxa" /TEE /R:5 /W:15 /NP /NS /NC /NJH /NJS /NDL
)
)
PAUSE
EXIT

Select allOpen in new window

this is another final corrected version.

I am actually correcting line 12.

and after few live backups I performed with this batch.. i found another bug that subfolders are not created inside the DAY folder...

below is the final version please discard previous

@ECHO OFF

REM Find which day it is today (eg. Mon, Tue, Wed etc.)
for /F %%A in ('date /t') do set DAY=%%A

REM Copy new files
ROBOCOPY "C:\SOURCE"  "C:\DESTINATION\ALL" *.* /TEE /E /R:5 /W:15 /FFT /NP /LOG:logtxt.txt /NS /NC /NJH /NJS /NDL

REM Clean up the log file and Read successive lines from the log file and copy the new files into specific DAY folder
for /f "delims=] tokens=1*" %%a in ('find /v /n "" ^<logtxt.txt') do (
for /f "tokens=*" %%s in ("%%b") do (
for %%a in ("%%s") do ROBOCOPY "%%~da%%~pa\" "C:\DESTINATION\%DAY%%%~pa\" "%%~nxa" /TEE /R:5 /W:15 /NP /NS /NC /NJH /NJS /NDL
)
)
PAUSE
EXIT

Select allOpen in new window

nbass668 I did try it and we are getting close...

there is another problem now.. it does copy also the destination folder ?

I don't know why but I have a sub-folder called "DESTINATION" in the DESTINATION folder
I think it needs some more fine tuning...

thanks

dear ammounpierre

any update? I hope the code is working fine with you :)

appreciate your response