speechlys
asked on
robocopy failing on large copy
Hi,
I am after some help with Robocopy.
Basically I have a share with 10 million flat files and 369k folders. this is actually a document management system file store. I need to copy the entire store entirely not missing a single file and I need to leave the source files in place as well.
I need to move this file store to a new source directory. I have done this in the past and been successful using the command below.
@ehho off
@ECHO !!!!!!!!!!!!! RoboCopy - Large file copy - Do you want to continue !!!!!!!!!!!!!
PAUSE
@echo Copy \\server1\copy-fixing-issu
C:\Windows\SysWOW64\Roboco
pause
however now I am hitting a brick wall in that the copy is bombing out complaining robocopy is running out of memory, the sever has adequate memory 32GB, with nothing else on it apps wise etc. I tried upping it to 128 GB Because the server is VM and it makes no difference. The memory never seems to be busy either unless its re-realising back to the OS once its failed.
I was originally doing this on a server 2012 server so I tried it on 2008 R2 server but it made no difference I still get the same error in the robocopy log
the servers are all on the same network in the same office etc so its not like there is link issues to overcome, any ideas?
once I have done the first initial copy, I then have to offline the document management system out of hours and do the same copy again but a differential so pick up any differences. this has to be right, i cant miss any files...
--------------------------
ROBOCOPY :: Robust File Copy for Windows
--------------------------
Started : Sun Jul 12 17:18:23 2015
Source : \\server1\WSDOCS\db1\
Dest : \\server2\WSDOCS2\db1\
Files : *.*
Options : *.* /S /E /COPY:DAT /PURGE /MIR /MT:8 /R:1000000 /W:30
--------------------------
ERROR : Robocopy ran out of memory, exiting.
ERROR : Invalid Parameter #%d : "%s"
ERROR : Invalid Job File, Line #%d :"%s"
Started : %hs
Source %c
Dest %c
I am after some help with Robocopy.
Basically I have a share with 10 million flat files and 369k folders. this is actually a document management system file store. I need to copy the entire store entirely not missing a single file and I need to leave the source files in place as well.
I need to move this file store to a new source directory. I have done this in the past and been successful using the command below.
@ehho off
@ECHO !!!!!!!!!!!!! RoboCopy - Large file copy - Do you want to continue !!!!!!!!!!!!!
PAUSE
@echo Copy \\server1\copy-fixing-issu
C:\Windows\SysWOW64\Roboco
pause
however now I am hitting a brick wall in that the copy is bombing out complaining robocopy is running out of memory, the sever has adequate memory 32GB, with nothing else on it apps wise etc. I tried upping it to 128 GB Because the server is VM and it makes no difference. The memory never seems to be busy either unless its re-realising back to the OS once its failed.
I was originally doing this on a server 2012 server so I tried it on 2008 R2 server but it made no difference I still get the same error in the robocopy log
the servers are all on the same network in the same office etc so its not like there is link issues to overcome, any ideas?
once I have done the first initial copy, I then have to offline the document management system out of hours and do the same copy again but a differential so pick up any differences. this has to be right, i cant miss any files...
--------------------------
ROBOCOPY :: Robust File Copy for Windows
--------------------------
Started : Sun Jul 12 17:18:23 2015
Source : \\server1\WSDOCS\db1\
Dest : \\server2\WSDOCS2\db1\
Files : *.*
Options : *.* /S /E /COPY:DAT /PURGE /MIR /MT:8 /R:1000000 /W:30
--------------------------
ERROR : Robocopy ran out of memory, exiting.
ERROR : Invalid Parameter #%d : "%s"
ERROR : Invalid Job File, Line #%d :"%s"
Started : %hs
Source %c
Dest %c
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
you don't know how stupid I feel right now, the old server was 32 bit and I have just assumed my old script should work from before as it worked fine. I didn't even think about changing the variable to 64bit completely overlooked ! thanks for pointing it out.....
I have done it before with robocopy with a similar number of flat files it handled it ok...thanks again will let you know if the 64 bit doesn't work but I think it will.
I have done it before with robocopy with a similar number of flat files it handled it ok...thanks again will let you know if the 64 bit doesn't work but I think it will.
1) add another disk to server1 in mirror with the FS store
2) sync
3) brake the mirror
4) move disk to server2
For Robocopy, run batch as Administrator. I guess that is User Rights related.