Why does RoboCopy run out of memory when copying millions of files from machine "a" to machine "b" over LAN
Posted on 2008-10-18
I have 2 machines.
Both machines are windows XP Pro.
I copy from one machine to another a folder.
Using the switch on robocopy that tells it to only copy over NEWER files.
I have 256 folders - split up into 16 batch files of 16 lines of robocopy each.
I run all 16 at once.
Over a LAN.
No drive mapping is used.
I use: " robocopy c:\55 \\10.0.0.99\c$\55 /E /XO "
it will do fine unless there are a lot of files.
By "a lot" i mean about 37 million.
It will say "unable to connect to server...waiting 30 secs..."
or sometimes it will say "insuffecient resources waiting 30 secs"
it will wait 30 secs - then it looks like it will send about 5 - 10 more files over
or compare 5 - 10 more files
then it will lock up again. and say "waiting 30 secs"
like the cache or page file or memory pool is not clearing temp memory.
Total files come to about 40 gigs, not very big - but millions of small files.
that need to be compared - and only the newer ones copied over.
other apps to do this?
or if anyone knows how to do this in VB.net I would appreciate it.
I am a beginner in Vb.net
compare folder A to Folder B -
copy all the f iles form Folder B to Folder A that are newer.
and do it for all 256 folders