?
Solved

Transfer Rate on a SAN

Posted on 2011-03-23
13
Medium Priority
?
747 Views
Last Modified: 2012-05-11
Two VM's are on two different host on a fibre channel SAN. The SAN is a CX300 which has a 2 GB backbone. Both VM's have NICs that are 1Gbps. If I'm copying 20 GB of data from one VM to the other, how long should that take? 20 seconds? This is if nothing else is happening on the SAN while the data is being copied.

Thanks,

Brandon
0
Comment
Question by:AGenMIS
  • 6
  • 3
  • 2
  • +2
13 Comments
 
LVL 3

Expert Comment

by:driskollt
ID: 35198853
How many disks are you copying to?  Drive type (SATA or FC)?  Drive RPM?  What RAID type?  Your disks have to be able to keep up with the writes as well.  

20GB would take ~156 seconds in a perfect 1gb world.  You're probably looking at ~200 seconds or more, depending on the types of files you're copying (many small files copy much slower than less large files).

If you don't have enough disks to support your writes, then it will take longer than that.  Usually the disks or host OS will end up being the bottleneck - not the network bandwidth.

0
 

Author Comment

by:AGenMIS
ID: 35198932
Coping from FC drives to SATA drives. Raid is 5. The SATA drives is mapped to our Virtual Center Server. So we're copying from a VM to a mapped drive on the Virtual Center Server. The mapped drive points to the SAN and the data is located on SATA drives.
0
 

Author Comment

by:AGenMIS
ID: 35198961
There are 14 SATA disk that make up that mapped drive. Disk are in a raid 5.
0
NEW Veeam Agent for Microsoft Windows

Backup and recover physical and cloud-based servers and workstations, as well as endpoint devices that belong to remote users. Avoid downtime and data loss quickly and easily for Windows-based physical or public cloud-based workloads!

 

Author Comment

by:AGenMIS
ID: 35198985
Not sure if this means anything but the 14 disk are on a MetaLun.
0
 
LVL 47

Expert Comment

by:David
ID: 35199020
The bottleneck is going to be at the destination, not the SAN.  The SATA-based RAID5 will be slower then the SAN.

So the correct answer is "however long it takes for the destination VM to write 20GB of data to the local RAID5".
RAID5 on SATA disks? Write penalty?  Heck it could take 5-10 minutes.

0
 

Author Comment

by:AGenMIS
ID: 35199083
Well the RAID 5 disk are on the SAN, not the local Virtual Center Server. The Vcenter Server maps to the SAN so we can copy data to it.
0
 

Author Comment

by:AGenMIS
ID: 35199363
I'm trying to figure out why it takes about 25 mins to copy data from a VM on a FC drive to the Vcenter Servers I: drive. The I: drive is a collection of 14 disk on the SAN in a Raid 5. The disk are SATA. The NIC on the VM and the Vcenter server is 1 Gbps.
0
 
LVL 63

Expert Comment

by:SysExpert
ID: 35201044
Try to see how long writing the same amount of data takes from a local machine to the SAN RAID 5.
That will give you the minimum time possible, all the rest will be network related.

I hope this helps !
0
 
LVL 47

Accepted Solution

by:
David earned 2000 total points
ID: 35201305
A 14-disk RAID5 is HORRIBLE, especially on writes, you could very well  be getting a whoppin' 10 MB/sec throughput on writes on a shared device.  The problem has to do with queued I/Os, the need to perform parity calculations on ANY RAID5; block size; and NCQ optimization.  It is an inherently poor design to use a large RAID5 SATA array in a virtual environment UNLESS you have a premium controller that is vm aware.  

5-10 MB/sec for writes is quite common.  If you want better performance, make yourself a RAID1, and use it for any write-intensive I/O.  Even just putting in a small local hard drive non-RAIDed for scratch copy would make this transfer complete in under 5 mins.
0
 
LVL 47

Expert Comment

by:David
ID: 35201407
Actually, a really quick fix would to put in a 32GB or larger USB flash memory stick on the host, and dedicate it to that particular VM.   (Just make sure it runs at the higher USB2 / USB3 speeds).
0
 
LVL 3

Expert Comment

by:driskollt
ID: 35201909
Do you have analyzer on your CX300?  If so, can you send me a .nar file that shows your file copy?

You're just copying over the network to a drive?  You're not using storage vmotion then I take it?
You've got lots of potential issues then.  TCP window size, mtu, etc.

I guess the 14-disk RAID-5 LUN is basically a "dumpster" for you?  A place to stick old/archive/backup data that you don't care about too much?

What are you copying?  Large files, lots of small files.  If you're copying lots of small files - then it could be very slow.

Also - Just because something is 1GB/4GB/10GB doesn't mean that you're going to see those kind of transfer rates.  It usually takes a lot of disks to push that kind of bandwidth.

Flash drives don't really perform any better for large sequential writes - They actually perform a little slower than spinning rust.
0
 

Author Comment

by:AGenMIS
ID: 35202216
I agree that it is a poor design and the poor design could possibly be the issue. We are copying lots of small files.
0
 
LVL 30

Expert Comment

by:Duncan Meyers
ID: 35202679
You should get excellent read performance from the FC drives. As everyone's saying, the problem is with writing to the SATA drives. The CX300 had 2GB of cache memory, some of which was used for the array's OS, so the real, usable amount of write cache you've got to play with will be around about 300MB (remembering that CLARiiON write cache is mirrored). The SATA drives will munch all that write cache pretty quickly for random I/O, so what I'd expect to see is initial high performance dropping to what you're seeing after a few seconds as once write cache is filled, you're down to the background performance of the SATA drives. Unless of course, you have write caching turned off for the SATA LUNs. So: is write cache on or off for the SATA LUNs? Right click on the LUN and select Properties. The Write cache status is toward the bottom of the pop-up (depending on FLARE version).

You can also right-click on the array serial number in Navisphere, select Properties and watch the cache status during the copy. If the cache status hits 100%and stays there, then that's the problem. That is; you're writing random I/O to the SATA . If you're writing sequential I/O, you should see 80 to 200MB/sec depending on how the SATA MetaLUN is configured. So: a few things to look at:

- Can you post a screen shot of Navisphere showing the MetaLUN components please?
- Can you provide more information about how the MetaLUN is configured? We frequently see misconfigured Metas where the misconfiguration is causing spindle contention.
- Check to see if you need to defrag the volume from Windows. A heavily fragmented file system will turn nice, neat sequential I/O into a random mess.
0

Featured Post

Keep up with what's happening at Experts Exchange!

Sign up to receive Decoded, a new monthly digest with product updates, feature release info, continuing education opportunities, and more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

It’s time for spooky stories and consuming way too much sugar, including the many treats we’ve whipped for you in the world of tech. Check it out!
Article by: evilrix
Looking for a way to avoid searching through large data sets for data that doesn't exist? A Bloom Filter might be what you need. This data structure is a probabilistic filter that allows you to avoid unnecessary searches when you know the data defin…
Teach the user how to join ESXi hosts to Active Directory domains Open vSphere Client: Join ESXi host to AD domain: Verify ESXi computer account in AD: Configure permissions for domain user in ESXi: Test domain user login to ESXi host:
This Micro Tutorial steps you through the configuration steps to configure your ESXi host Management Network settings and test the management network, ensure the host is recognized by the DNS Server, configure a new password, and the troubleshooting…
Suggested Courses

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question