Copying millions of files in one directory
Posted on 2004-09-01
Currently, we have a lot of raw data flat files that are generated with every request at our business. These are dumped all into one folder and there are millions of them. Whenever we try to backup these files, it takes forever!! Most are under 1kb and the hard drive has a 4K block size. On top of that, the entire hard drive is extremely fragmented.. The problem lies in the time it takes the hard drive to do millions of seeks (which take about 5ms on a scsi hd) and then it ends up taking days to copy! Is there anyway that you could speed this up and maybe do a rawsector copy generated from the locations of the files in the MFT and not start and stop on file transfers and seek so often?? Changing the organization of these files is also not really an option-so we'll ave to figure something out. Thanks!