Improve company productivity with a Business Account.Sign Up

x
?
Solved

iSCSI Speed - Why so inconsistant?

Posted on 2013-11-27
10
Medium Priority
?
309 Views
Last Modified: 2016-11-23
I have created tow SANs using OPEN-E for an Active -Active High Availablility Failover. I also Have created a Microsoft 2008 Cluster R2. I have two iscsi ports in each 2008 Server Node.

On Node A - I was able to grab a 7GB file and copy and paste it to my desktop in 12 seconds. It showed 500+ MBs a second.

However on Node B - I am only able to get 120 MBs a second and it takes 45 seconds to copy the same file. The two nodes are the exact same spec, processor, memory, etc...

Nic settings are the same. I used dells recomendation to set up iSCSI network accept for Jumbo frames and the Netsh commands.

I double checked the cards i didn't miss anything. The switches that are used in the high availability setup are the same.

I stopped and restarted the cluster and now Node one no longer copies at 500mbs a second. Now it only copies at 120mbs and takes 45 seconds to copy the same file.

What's weird is when it was copying at 500mbs a second I couln't see any iSCSI traffic on the TASK MANAGER NEtwork monitor tab and it said 0% under iSCSI Nics during the lightning fast copy . Now I can see the graph during the slower copy and it shows about 55%.

Can this be explained. I am using MPIO.
0
Comment
Question by:MEATBALLHERO
  • 5
  • 2
  • 2
  • +1
10 Comments
 
LVL 47

Accepted Solution

by:
David earned 2000 total points
ID: 39682756
Lots of things
 * Your benchmark is lying about when the file has actually been copied & flushed to disk. If you shut the computer off you might actually see that only 2-3 GB have actually been copied.
* Your benchmark includes a large number of tests, from uncached & cached reads, network transfer, and writes, yet you consider this nothing more than an iSCSI throughput test.

* The system cache is skewing the results.  That is why it is so fast at the beginning. It hits the ceiling after a few MB.


Set some baselines and do benching of both local reads/writes, then only test an iSCSI benchmark.
0
 
LVL 40

Expert Comment

by:Aaron Tomosky
ID: 39682857
If it's gigabit you can't go over 120MB/s no matter how fast your array may be.
0
 
LVL 47

Expert Comment

by:David
ID: 39683341
To summarize ... you are actually getting reasonable results, and test A is an invalid test because the O/S is lying about the file really being copied.   Nothing is wrong, you are getting what you should be getting.
0
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

 

Author Comment

by:MEATBALLHERO
ID: 39684642
Wait...are you saying that even though it looks like I copied a 7GB file in 12 seconds, thats not true?

Are you saying that even though the window that shows the file copying closes, that the file is still copying in the back ground or something?
0
 
LVL 47

Expert Comment

by:David
ID: 39684703
Yes, that is exactly what I am saying.   Do the math.  Let's ignore inefficiency and overhead.
7GB * 8 bits/byte / 12 seconds = 4.7 gigabits/sec

do you even HAVE a 4.7 gigabit connection, or do you have perhaps a 1Gigabit connection?

For that matter, I doubt you even have a disk drive that can read or write 7Gbytes in 12 seconds, let alone read the 7GB in the first 6 seconds, then write it at the other end in 6 seconds ... and, even then, we ignore the physics of the speed of light and the time it takes to move those electrons from point A to point B.

Bottom line, your benchmarks are giving you bad data. Based on the sustained numbers with the longer test, I see no reason for any concern.  Sure you can always do some tweaking, but don't expect anything more than +/- 20%
0
 
LVL 22

Expert Comment

by:eeRoot
ID: 39685422
Can you verify the write speed of your desktop's hard drive?  And if the drive has built in cache?  You may get odd benchmark results if the drive has high speed memory to cache writes to disk, but then has to save that data to disk at regular disk write speeds.
0
 
LVL 40

Expert Comment

by:Aaron Tomosky
ID: 39685638
And what nics are in both systems
0
 
LVL 47

Expert Comment

by:David
ID: 39685690
The NICs don't matter.  Benchmarking HAS to be wrong.   You are reading 7GB, transferring 7GB, and then writing 7 GB in all of 12 seconds.

That is roughly 2 GigaBYTES per second.  It doesn't matter what NIC you have.  You'd have to have infiniband, and some rather expensive RAID hardware to obtain this.  If you had such hardware you would know it.
0
 

Author Comment

by:MEATBALLHERO
ID: 39694545
I have two San Servers that replicate. Both have a 16 port promise raid with 512MB of Cache. There are 16 SAS drives in each SAN that run at 15K RPM. this is where the 7GB file sits. I copied it to the desktop of my host which is 4 SAS raid 10 drives at 15K RPM. The drives all have cache enabled and so does the raid cards on all servers.

I do have a 10GB ethernet but that is connected SAN to SAN for replication and not SAN to Host.

Each San has 4 ISCSI Ports and My Host Has two but I put in the IP addresses of all 4 SAN Ports usining MPIO.
0
 
LVL 47

Expert Comment

by:David
ID: 39694557
It doesn't matter. No way can your hardware do 2Gbytes/sec.
0

Featured Post

Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

I’m willing to make a bet that your organization stores sensitive data in your Windows File Servers; files and folders that you really don’t want making it into the wrong hands.
In this article, we will discuss how you can secure Active Directory using free tools, and how you can choose a safe and secure Active Directory security auditing tool.
To efficiently enable the rotation of USB drives for backups, storage pools need to be created. This way no matter which USB drive is installed, the backups will successfully write without any administrative intervention. Multiple USB devices need t…
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …

606 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question