Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)
Course Of The Month
6 days, 10 hours left to enrollBecome a Premium Member and unlock a new, free course in leading technologies each month.
Solved
Raid 1 - 5Mil+ File Transfer
Posted on 2012-12-31
I am trying to move some data from one external hard drive(which I removed and installed the drive in the machine) onto a new raid 1 array. The transfer speeds seem very slow, but I am not sure as I have never moved this amount of files before.
I am trying to move 5,000,000+ 75-150kb files onto the array, the transfer speeds are about 5MB, is this normal? It seems slow as 160GB has taken 5hours...
I am trying to move 5,000,000+ 75-150kb files onto the array, the transfer speeds are about 5MB, is this normal? It seems slow as 160GB has taken 5hours...
[X]
Welcome to Experts Exchange
Add your voice to the tech community where 5M+ people just like you are talking about what matters.
- Help others & share knowledge
- Earn cash & points
- Learn & ask questions
2-
2
- +1
6 Comments
Hi,
Yes that is very very normal as your number of files are a lot. For example;
1GB data as 1 File only= 1 minute
1GB data as 500 files =8 minutes
Yes that is very very normal as your number of files are a lot. For example;
1GB data as 1 File only= 1 minute
1GB data as 500 files =8 minutes
Active today
Get a faster external HDD interface. Looks like you are using USB2. Go USB3 or eSATA and you'll be at least 10X faster.
(Or just leave the HDD inside the machine and copy via 1GigE ethernet to another computer. That also will be at least 10X faster.
(Or just leave the HDD inside the machine and copy via 1GigE ethernet to another computer. That also will be at least 10X faster.
dlethe Posted on 2013-01-01 at 00:10:21ID: 38734130
Get a faster external HDD interface. Looks like you are using USB2. Go USB3 or eSATA and you'll be at least 10X faster.
(Or just leave the HDD inside the machine and copy via 1GigE ethernet to another computer. That also will be at least 10X faster.
It is in the machine, I removed it from the external.
There is not much you can do, that is nature of the data transfer. If the number of files are so much then you will experince surprisingly slow speeds as all files indivudally have their own attiributes to be written. Upon completion of each file those attributes also written one by one thats why the data transfer speed is unbelivably slow but it is normal, just let it finish its work. You can test that fact by using 1 single file from the same source HDD to same destination. You may use a iso file with a size of a couple of gigabytes. You will see 1 single file will be copied much much faster.
Your number of files are crazy, 5.000.000+ of 150KB files waoow :) Everything just normal dont worry.
Your number of files are crazy, 5.000.000+ of 150KB files waoow :) Everything just normal dont worry.
lots of small files also means a lot of directory and FAT updates, this typically means the disk heads are going back and forth with a lot of random read and writes.
The only things you can really do to improve speed..
stop other services on the machine - particularly antivirus - but then give the drive a manual full scan when you are finished.
If the AV is scanning each file as it gets copied that will represent an overhead you can most likely be comfortable in turning off.
The only things you can really do to improve speed..
stop other services on the machine - particularly antivirus - but then give the drive a manual full scan when you are finished.
If the AV is scanning each file as it gets copied that will represent an overhead you can most likely be comfortable in turning off.
Featured Post
Nowadays, monitoring is a mixture of tools, systems, and codes—making it a very complex process. And with this complexity, comes variables for failure. Get DZone’s new Guide to Performance to learn how to proactively find these variables and solve them before a disruption occurs.
Question has a verified solution.
If you are experiencing a similar issue, please ask a related question
Possible fixes for Windows 7 and Windows Server 2008 updating problem. Solutions mentioned are from Microsoft themselves. I started a case with them from our Microsoft Silver Partner option to open a case and get direct support from Microsoft.
If s…
We look at whether swapping a controller board on a failed hard drive is likely to solve the problem.
This tutorial will show how to push an installation of Backup Exec to an additional server in both 2012 and 2014 versions of the software.
Click on the Backup Exec button in the upper left corner. From here, select Installation and Licensing, then I…
This tutorial will walk an individual through the steps necessary to install and configure the Windows Server Backup Utility.
Directly connect an external storage device such as a USB drive, or CD\DVD burner:
If the device is a USB drive, ensure i…
Suggested Courses
Course of the Month6 days, 10 hours left to enroll
Earn Certification
MCSA Configuring Windows 7 Exam 70-680
$49.00$44.10
726 members asked questions and received personalized solutions in the past 7 days.
Join the community of 500,000 technology professionals and ask your questions.