Using xcopy command to copy folders and files

Hi,
Just add /e to the xcoy to copy subdirectories and empty subdirectories to the destination

xcopy /? gives list of other such options.

If you need more control over what is copied have a look at robocopy amonst others (google it)

hth

Steve

To make an exact copy, including permissions and auditing info:

xcopy c:\folder_a e:\folder_a_backup /e /y /c /h /r /o /x

Would you please inform me what each /* means?

xcopy /? shows you.  I doubt you ned most of the above unless you have permissions set, read only files in there etc. that you want to preserve ... but they cover all the options

Id post the list but typing from mobile ...
Steve

Cool so that worked I do have another question apparently there is over 5000 files that got copied over, however; when I look in the cmd prompt (I put a pause at the end of the batch) I can only go so far up what do I need to do to be able to see the first file that got copied over to the last?

I'm assuming I would need to go to the command history section but what number would I need to put at what part to go that far up?

You should pipe the output of the xcopy to a text file and review the text file later.

xcopy c:\folder_a e:\folder_a_backup /e /y /c /h /r /o /x >c:\FilesCopied.txt

AmazingTech....I am assuming the >c:\FilesCopied.txt would be written in the C:\ drive.... so in essence I can do

xcopy c:\folder_a e:\folder_a_backup /e /y /c /h /r /o /x >e:\folder_a_backup\filescopied.txt

Actually would this work:

xcopy /y c:\folder_a e:\Backups\folder_a_backups /e

xcopy /y c:\folder_b e:\Backups\folder_b_backups /e

xcopy /y d:\folder_c e:\Backups\folder_c_backups /e

xcopy /y d:\folder_d e:\Backups\folder_d_backups /e

>e:\Backups\filescopied.txt

pause

And on the "filescopied.txt" what do I need to put in order to place a date time stamp in it??

Nope but you can do

@echo off
set log=e:\Backups\filescopied.txt
xcopy /y c:\folder_a e:\Backups\folder_a_backups /e > %log% 2>&1
xcopy /y c:\folder_b e:\Backups\folder_b_backups /e >>%log% 2>>&1
xcopy /y d:\folder_c e:\Backups\folder_c_backups /e >>%log% 2>>&1
xcopy /y d:\folder_d e:\Backups\folder_d_backups /e >>%log% 2>>&1
pause

The first creates a new file with > the others append to the same file.  The 2> bits capture errors too (considering that is probably what you want to see).

echo %date% %time% >> %log% will give you a date/time stamp.

I don't get what the first two lines do though and where exactly would I put the
echo %date% %time% >> %log%

and what does that "%log%" mean?

I was running a test from above and this is what I put into my batch file:  

@echo off

REM Set log destination
set log=d:\test\test.txt

REM Set destination for backups
set dest=d:\test

REM Copy 1 folder to backup drive keeping log in above filename
echo Backup running on %computername% by %username% > %log% 2>&1

echo Backup A started at %date% %time%  >> %log% 2>>&1
xcopy /y C:\NEW EDITS %dest%\d:\test /e >> %log% 2>>&1


echo Backups completed at %date% %time%  >> %log%

REM Show the log file
notepad %log%
xcopy /y "C:\NEW EDITS" "d:\test" /e >d:\test\test.txt

pause

However, this is what came into the text file
Backup running on MYCOMPUTER by ME
Backup A started at Tue 12/09/2008  9:27:12.94  
Invalid number of parameters
Backups completed at Tue 12/09/2008  9:27:12.97

Now what did I do wrong?  

You have said

xcopy /y C:\NEW EDITS %dest%\d:\test /e >> %log% 2>>&1

and %dest% gets replaced by d:\test so effectively you have written

xcopy /y c:\new edits d:\testD:\test /e .....

Also your source directory has a space in so you need to use " " around it

try

xcopy /y "c:\NEW EDITS" %dest% /e >> %log% 2>>&1

much like you have in your line after the notepad one....

dragon you have been a great help I do have a question though what does 2>>&1 mean

I'll take a step back and explain..

> creates a file from standard output
>> appends to a file
2> writes standard error output to file
2>> appends standard error output to file

some executables will send errors to the stderr (2>) stream others to the normal STDOUT > or 1>

2>&1 sends the STDERR stream into the STDOUT stream which is then being redirected into a file.


Steve

and the &1

See the last line above.  "&1" means the STDOUT stream .... = stream 1 used by 1> which is what you get when you just do >.

i.e.  program outputs

This is normal stuff   to STDOUT (stream 1)
oh know error occured to STDERR (stream 2)

If you do program > x.txt you only get the first line
if you do program 2>x.txt you only get the second line
if you do program >x.txt 2>&1 then the error gets directed into the first stream too so it all goes to x.txt...

Sorry seem to have gone off on one again.... if not clear, easy once you know how :-)

Steve

Just to let you know I will probably keep this open until Friday because on Thursday is when I will be running this.  I will let you know if this is exactly what I wanted thanks again.

Oh yeah one more thing I will be running this batch file monthly so two things:
1.  This batch file will it write over the previous files?
2.  Will it create a new log file everytime I run the batch or will it create a new one?

It will overwrite the previous files.
It will overwrite the log file unless you move it as it stands.  To append change the first > entries to >> instead.  Alternatively use a date based log file:

REM Set log destination.  Take date and replace and / with - characters
set log="d:\test\logfile_%date:/=-%.txt"

I kind of ran into a problem where we have
set dest=e:\backups

The E: drive itself is only about 30GB large and these four folders combined is about 50GB large so it will not all fit.  What I was wondering is I have a network drive connect on the local machine basically it would be the same path except now it would be on drive J which is the networked drive letter, I thought once it is mapped all I would need to do is replace the E: with a J: and it would be fine, however I got a "The system cannot fine the path specified" I am really hoping this would work what am I now doing wrong.  I even did

set dest=\\computername\D$\Backups\

that didn't work either.

It should be fine if there is an J: drive say ... as long as the J: drive is already mapped and accessible --> e.g. if you are scheduling it then the user it is scheduled as needs to have rights to it.

For testing purposes if you map a drive to J:, say, then run from cmd prompt and check you can do a dir J: first then run the script it should be OK.

If not sure post the script as it is here and will check it with you.

You could also use the unc path as above... except you may find you have two \\ characters in the middle there, try taking the \ off the end of the set dest= line.


Steve

Steve,

Sorry I know what I did wrong it probably would've helped if I created the folder Folder_A on the network drive.

easy mistake to make :-)

I'm sorry I keep on saying last thing but this should be the last thing (I hope) when the log file is created is it going to be one huge log file or will it be separate of each folder being copied?  From what I can see it will be one long one if that is the case how would I make separate ones?

Meaning Folder_A will get its own "files copied .txt"
Folder_B will get its own and so on

To use a different log file you could perhaps add an A B C etc, at the beginning i.e.

... existing bit

@echo off

REM Set log destination
set logAll=e:\Backups\filescopied.txt
set log=e:\Backups\filescopied

REM Set destination for backups
set dest=e:\backups


REM Copy 4 folders to backup drive keeping log in above filename
echo Backups running on %computername% by %username% > %logAll% 2>&1

echo Backup A started at %date% %time%  > %log%_A.txt 2>&1
xcopy /y c:\folder_a %dest%\folder_a_backups /e >> %log%_A.txt 2>>&1
echo Backup A ended at %date% %time%  > %log%_A.txt 2>&1
type %log%_A.txt >> %logAll%

echo Backup B started at %date% %time%  > %log%_B.txt 2>&1
xcopy /y c:\folder_b %dest%\folder_b_backups /e >>%log%_B.txt 2>>&1
echo Backup B ended at %date% %time%  > %log%_B.txt 2>&1
type %log%_B.txt >> %logAll%

etc.

echo Backups completed at %date% %time%  >> %log%

REM Show the log file
notepad %logAll%

That way you end up with filescopied_A.txt, filescopied_B.txt and filescopied.txt which contains all of them.

You may want to look at using /D on the xcopy line to copy only changed files too.

Hey Steve,
I tried what you said above and well it almost worked but the .txt files do not show what files have been copied over.  This is what I have put into my batch file (I know I keep changing drive letters, but now I am in my test environment and understanding what you have said in above areas getting this)

@echo off

REM Set log destination
set logAll=d:\test\filescopied.txt
set log=d:\test\filescopied

REM Set destination for backups
set dest=d:\test

REM Copy 2 folders to backup drive keeping log in above filename
echo Backups running on %computername% by %username% > %logAll% 2>&1

echo Backup A started at %date% %time%  > %log%_reports.txt 2>&1
xcopy /y "C:\Documents and Settings\Me\Desktop\Reports" %dest%\Reports_Backups /e >> %log%_reports.txt 2>>&1
echo Backup A ended at %date% %time%  > %log%_reports.txt 2>&1
type %log%_reports.txt >> %logAll%

echo Backup B started at %date% %time%  > %log%_Essays.txt 2>&1
xcopy /y "C:\Documents and Settings\Me\Desktop\Essays" %dest%\Essays_Backups /e >> %log%_Essays.txt 2>>&1
echo Backup B ended at %date% %time%  > %log%_Essays.txt 2>&1
type %log%_Essays.txt >> %logAll%

echo Backups completed at %date% %time%  >> %log%

REM Show the log file
notepad %log%

My filescopied.txt shows this:
Backups running on MyComputer by Mel
Backup A ended at Wed 12/10/2008 13:27:54.15  
Backup B ended at Wed 12/10/2008 13:27:54.22  

The filescopied_Essays.txt show:
Backup B ended at Wed 12/10/2008 13:27:54.22  

The filescopied_reports.txt show:
Backup A ended at Wed 12/10/2008 13:27:54.15  

And the attachment shows what is in the folder overall, mind you the files themselves did copy over





untitled.bmp

Steve,

I got it.  It should have with >> instead of > for the started and end lines

Correct... my fault there copy and pasting.
Have a look at using /d btw too as I think I suggested above on the xcopy if any of the files stay the same it will leave them alone.

Steve

When using the /d function, I am under then impression that I would place it either before or after the /e is that correct?  You said the fuction itself would look at the file and if it is the same it would leave it alone, so I guess an added period (.) at an end of a sentence would constitue as a change so it would overwrite it correct?

Yes that will work either way.  It works on date.time stamp of the file so any change to the file, even if re-saved exactly the same and it would copy.

There are other ways to do it with the robocopy command etc. but xcopy /d /e /y is a good start.

From xcopy /?

 /D:m-d-y     Copies files changed on or after the specified date.
              If no date is given, copies only those files whose
              source time is newer than the destination time.


The other way of doing just changed files is to use

/M           Copies only files with the archive attribute set,
             turns off the archive attribute.


It can be worth doing this too:

/EXCLUDE:file1[+file2][+file3]...
             Specifies a list of files containing strings.  When any of the
             strings match any part of the absolute path of the file to be
             copied, that file will be excluded from being copied.  For
             example, specifying a string like \obj\ or .obj will exclude
             all files underneath the directory obj or all files with the
             .obj extension respectively.

which means you could exclude all .bak or .tmp files for instance by creating a text file called exclude.txt and adding /exclude:exclude.txt etc.

 /C           Continues copying even if errors occur.

Can be useful depending upon what you want in your log files:

/Q           Does not display file names while copying.
/F           Displays full source and destination file names while copying.

If you have any special files in the dirs or need to keep permissions etc:

/H           Copies hidden and system files also.
/R           Overwrites read-only files.
/K           Copies attributes. Normal Xcopy will reset read-only attributes.
/O           Copies file ownership and ACL information.
/X           Copies file audit settings (implies /O).

Dont need to use any of those for most situations but worth knowing of.  xcopy /? for a list.

Steve

Steve,

Sorry not getting back sooner but the big script I executed last week, and everything worked pretty well except for one part of the script:

@echo off

REM Set log destination
set logAll=\\6ces00301447g\D$\Backups\nvzramcgeob\filescopied.txt
set log=\\6ces00301447g\D$\Backups\nvzramcgeob\filescopied

REM Set destination for backups
set dest=\\6ces00301447g\D$\Backups\nvzramcgeob

REM Copy 5 folders to backup drive keeping log in above filename
echo Backup running on %computername% by %username% > %logAll% 2>&1

...............................

echo AppServer_Backup started at %date% %time%  >> %log%_AppServer.txt 2>>&1
xcopy /y C:\Program Files\ArcGIS\ArcIMS\AppServer %dest%\AppServer_backups /e >> %log%_AppServer.txt 2>>&1
echo AppServer_Backup ended at %date% %time%  >> %log%_AppServer.txt 2>>&1
type %log%_AppServer.txt >> %logAll%

But then the following would show up on the .txt file:

AppServer_Backup started at Tue 12/23/2008 19:05:01.02  
Invalid number of parameters
AppServer_Backup ended at Tue 12/23/2008 19:05:01.08

What is missing????

Also, what would I need to do to the script in order to see the files that are currently being copied over I know there is /F but that seems kind of like overkill, I really don't need to see where it's coming from and where it's going to but I would like to see "it" being transferred, because I always have a feeling of it being frozen.

Out and about at the moment so quick answer... but think your error is due to no " characters around the paths in your xcopy line - I.e. "c:program files..." etc.

Will lok back tomorrow probably but try that for starters.

Steve

If you want to see the output of the xcopy command you could do something like this:

xcopy "c:\program files...." "%dest%\whatever.." /x | tee.exe yourlogfile.txt

You can get tee which sends output to screen and file:
http://unxutils.sourceforge.net/

or you could just choose to capture any errors, i.e. do

xcopy .....  2>> yourlogfile.txt

which would leave the normal output going to the console.  You could check after the xcopy with a
if errorlevel 1 echo Error occured in Xcopy - error  %errorlevel%

Give this lot a try and come back to me.

regards

Steve

Well what I was tying to do was have the output show on the command prompt.  Not in the log file.

Then use either tee to send it to both, or output it to a log file and do type logfile.txt straight after the command.  You could do something like this to exclude all the files:

xcopy "source" "dest" /s/y etc. | tee logfile.txt
type logfile.txt | find /v "." >> yourmainlogfile.txt

So it writes output to the logfile.txt and console then filters any lines without a "." in them into the main logfile?

Steve

So there is no way of just showing the files being copied over (as they copy over) in the command prompt?

You are redirecting the output (all output) of the xcopy to the logfile.  Using something like tee will show all output as normal to the console (i.e. the files being copied) while also writing to a log file.  You can then remove some of the data sent to the log file afterwards if you wish.

You might want to have a look at robocopy if you aren't familiar (is MS utility, google it will find you link to it).  You can send output to log file directly using /LOG or /LOG+ (to append) and mirror source and dest directory.  If you want to go down that route let me know.

Ok Steve I will take a look at it but how about the other thing I was talking about

@echo off

REM Set log destination
set logAll=\\6ces00301447g\D$\Backups\nvzramcgeob\filescopied.txt
set log=\\6ces00301447g\D$\Backups\nvzramcgeob\filescopied

REM Set destination for backups
set dest=\\6ces00301447g\D$\Backups\nvzramcgeob

REM Copy 5 folders to backup drive keeping log in above filename
echo Backup running on %computername% by %username% > %logAll% 2>&1

...............................

echo AppServer_Backup started at %date% %time%  >> %log%_AppServer.txt 2>>&1
xcopy /y C:\Program Files\ArcGIS\ArcIMS\AppServer %dest%\AppServer_backups /e >> %log%_AppServer.txt 2>>&1
echo AppServer_Backup ended at %date% %time%  >> %log%_AppServer.txt 2>>&1
type %log%_AppServer.txt >> %logAll%

But then the following would show up on the .txt file:

AppServer_Backup started at Tue 12/23/2008 19:05:01.02  
Invalid number of parameters
AppServer_Backup ended at Tue 12/23/2008 19:05:01.08

What is missing????

I answered that one in first quick reply... probably missed it in the glut of otther entries! you need " " characters around the source filename as it has spaces in the path, i.e.

echo AppServer_Backup started at %date% %time%  >> %log%_AppServer.txt 2>>&1
xcopy /y "C:\Program Files\ArcGIS\ArcIMS\AppServer" "%dest%\AppServer_backups" /e >> %log%_AppServer.txt 2>>&1
echo AppServer_Backup ended at %date% %time%  >> %log%_AppServer.txt 2>>&1
type %log%_AppServer.txt >> %logAll%

Steve

LOL, sorry I saw it I just wasn't sure if that was a quick answer or if that was a for sure answer...my bad, but I will check all this out thanks again.