Solved

Transfer Rate on a SAN

Posted on 2011-03-23
13
696 Views
Last Modified: 2012-05-11
Two VM's are on two different host on a fibre channel SAN. The SAN is a CX300 which has a 2 GB backbone. Both VM's have NICs that are 1Gbps. If I'm copying 20 GB of data from one VM to the other, how long should that take? 20 seconds? This is if nothing else is happening on the SAN while the data is being copied.

Thanks,

Brandon
0
Comment
Question by:AGenMIS
  • 6
  • 3
  • 2
  • +2
13 Comments
 
LVL 3

Expert Comment

by:driskollt
Comment Utility
How many disks are you copying to?  Drive type (SATA or FC)?  Drive RPM?  What RAID type?  Your disks have to be able to keep up with the writes as well.  

20GB would take ~156 seconds in a perfect 1gb world.  You're probably looking at ~200 seconds or more, depending on the types of files you're copying (many small files copy much slower than less large files).

If you don't have enough disks to support your writes, then it will take longer than that.  Usually the disks or host OS will end up being the bottleneck - not the network bandwidth.

0
 

Author Comment

by:AGenMIS
Comment Utility
Coping from FC drives to SATA drives. Raid is 5. The SATA drives is mapped to our Virtual Center Server. So we're copying from a VM to a mapped drive on the Virtual Center Server. The mapped drive points to the SAN and the data is located on SATA drives.
0
 

Author Comment

by:AGenMIS
Comment Utility
There are 14 SATA disk that make up that mapped drive. Disk are in a raid 5.
0
 

Author Comment

by:AGenMIS
Comment Utility
Not sure if this means anything but the 14 disk are on a MetaLun.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
The bottleneck is going to be at the destination, not the SAN.  The SATA-based RAID5 will be slower then the SAN.

So the correct answer is "however long it takes for the destination VM to write 20GB of data to the local RAID5".
RAID5 on SATA disks? Write penalty?  Heck it could take 5-10 minutes.

0
 

Author Comment

by:AGenMIS
Comment Utility
Well the RAID 5 disk are on the SAN, not the local Virtual Center Server. The Vcenter Server maps to the SAN so we can copy data to it.
0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 

Author Comment

by:AGenMIS
Comment Utility
I'm trying to figure out why it takes about 25 mins to copy data from a VM on a FC drive to the Vcenter Servers I: drive. The I: drive is a collection of 14 disk on the SAN in a Raid 5. The disk are SATA. The NIC on the VM and the Vcenter server is 1 Gbps.
0
 
LVL 63

Expert Comment

by:SysExpert
Comment Utility
Try to see how long writing the same amount of data takes from a local machine to the SAN RAID 5.
That will give you the minimum time possible, all the rest will be network related.

I hope this helps !
0
 
LVL 47

Accepted Solution

by:
dlethe earned 500 total points
Comment Utility
A 14-disk RAID5 is HORRIBLE, especially on writes, you could very well  be getting a whoppin' 10 MB/sec throughput on writes on a shared device.  The problem has to do with queued I/Os, the need to perform parity calculations on ANY RAID5; block size; and NCQ optimization.  It is an inherently poor design to use a large RAID5 SATA array in a virtual environment UNLESS you have a premium controller that is vm aware.  

5-10 MB/sec for writes is quite common.  If you want better performance, make yourself a RAID1, and use it for any write-intensive I/O.  Even just putting in a small local hard drive non-RAIDed for scratch copy would make this transfer complete in under 5 mins.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
Actually, a really quick fix would to put in a 32GB or larger USB flash memory stick on the host, and dedicate it to that particular VM.   (Just make sure it runs at the higher USB2 / USB3 speeds).
0
 
LVL 3

Expert Comment

by:driskollt
Comment Utility
Do you have analyzer on your CX300?  If so, can you send me a .nar file that shows your file copy?

You're just copying over the network to a drive?  You're not using storage vmotion then I take it?
You've got lots of potential issues then.  TCP window size, mtu, etc.

I guess the 14-disk RAID-5 LUN is basically a "dumpster" for you?  A place to stick old/archive/backup data that you don't care about too much?

What are you copying?  Large files, lots of small files.  If you're copying lots of small files - then it could be very slow.

Also - Just because something is 1GB/4GB/10GB doesn't mean that you're going to see those kind of transfer rates.  It usually takes a lot of disks to push that kind of bandwidth.

Flash drives don't really perform any better for large sequential writes - They actually perform a little slower than spinning rust.
0
 

Author Comment

by:AGenMIS
Comment Utility
I agree that it is a poor design and the poor design could possibly be the issue. We are copying lots of small files.
0
 
LVL 30

Expert Comment

by:Duncan Meyers
Comment Utility
You should get excellent read performance from the FC drives. As everyone's saying, the problem is with writing to the SATA drives. The CX300 had 2GB of cache memory, some of which was used for the array's OS, so the real, usable amount of write cache you've got to play with will be around about 300MB (remembering that CLARiiON write cache is mirrored). The SATA drives will munch all that write cache pretty quickly for random I/O, so what I'd expect to see is initial high performance dropping to what you're seeing after a few seconds as once write cache is filled, you're down to the background performance of the SATA drives. Unless of course, you have write caching turned off for the SATA LUNs. So: is write cache on or off for the SATA LUNs? Right click on the LUN and select Properties. The Write cache status is toward the bottom of the pop-up (depending on FLARE version).

You can also right-click on the array serial number in Navisphere, select Properties and watch the cache status during the copy. If the cache status hits 100%and stays there, then that's the problem. That is; you're writing random I/O to the SATA . If you're writing sequential I/O, you should see 80 to 200MB/sec depending on how the SATA MetaLUN is configured. So: a few things to look at:

- Can you post a screen shot of Navisphere showing the MetaLUN components please?
- Can you provide more information about how the MetaLUN is configured? We frequently see misconfigured Metas where the misconfiguration is causing spindle contention.
- Check to see if you need to defrag the volume from Windows. A heavily fragmented file system will turn nice, neat sequential I/O into a random mess.
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

Suggested Solutions

HOW TO: Install and Configure VMware vSphere Hypervisor 6.5 (ESXi 6.5), Step by Step Tutorial with screenshots. From Download, Checking Media, to Completed Installation.
Veeam Backup & Replication has added a new integration – Veeam Backup for Microsoft Office 365.  In this blog, we will discuss how you can benefit from Office 365 email backup with the Veeam’s new product and try to shed some light on the needs and …
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:
This Micro Tutorial steps you through the configuration steps to configure your ESXi host Management Network settings and test the management network, ensure the host is recognized by the DNS Server, configure a new password, and the troubleshooting…

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now