• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1234
  • Last Modified:

Determine iSCSI overhead via perfmon?

Is there any way to determine how much overhead there is because we're using Microsoft' iSCSI software initiator with an onboard NIC? We're not sure if we want to invest $500+ in an iSCSI HBA card. I've heard it can save 60% of CPU utilization.

0
MrVault
Asked:
MrVault
  • 12
  • 4
  • 3
  • +2
5 Solutions
 
Matt VCommented:
Trying adding a Process monitor, % of Processor Time with the iSCSI process selected.
0
 
MrVaultAuthor Commented:
What is the iSCSI process called?
0
 
Gerald ConnollyCommented:
what is your cpu running at now?
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
MrVaultAuthor Commented:
some are running at 40-70%. They are SQL servers running intensive operations, but it's hard to tell if the iSCSI piece is a bottleneck worth investing in.
0
 
mastooCommented:
You need to look at something closer to pure i/o.  Maybe run iometer and look at cpu%.  It would be nice to benchmark it against the equivalent system with an iScsi hba but you'll at least get an idea.  If iometer hits the drives and you just show 10% cpu it is easy to say you don't need to spend the money.
0
 
MrVaultAuthor Commented:
when you say just how 10%, do you mean it increases it by 10% or do you mean it categorizes iSCSI overhead as only 10%?

I've never used iometer before.
0
 
mastooCommented:
The simplest way is with the server doing nothing, then you run iometer and look at total cpu usage.  I just suggested that as a start because if you get a very low cpu number in that scenario, you know the iScsi overhead is even smaller.  Otherwise I'm not sure how you identify overhead specific to running an iScsi software adapter.
0
 
MrVaultAuthor Commented:
iSCSI overhead, is that tied to activity? Meaning is the overhead percent the same whether there's data going over iSCSI to the bulk storage somewhere or if there's really no data being transferred at all?
0
 
mastooCommented:
I won't call myself an expert in this area, but I imagine there is some cpu load attributed to the simulation of a hardware iScsi adapter.  So I would think it only manifests during I/O, which is why I thought iometer would be a good way to exercise it.  You get kind of the simplest i/o you can get.  The trouble with looking at cpu while running sql is that a large portion of the cpu is due to sql activity, which a smaller portion due to i/o, and of that even a smaller portion would be due to having a software iScsi adapter.  On our servers, I've observed that total cpu is fairly low during stress testing of the i/o so I've never worried about needing a hardware adapter.  The main drawback to the test I suggest is that if you get a high cpu, you can't reach any conclusion.  But a low cpu would lead to the conclusion that there would be little benefit to going hardware.
0
 
Gerald ConnollyCommented:
If your current CPU utilisation is in the 40-70% range its debateble whether you need a hardware accelerator at present, sure its likely to reduce you CPU utilisation but is that worth $600, i doubt that! You will probably do better to buy some more memory.

Now if you are expecting your workload to increase its probably worth doing due diligence on make, model and pricing so that if you utilisation starts creeping up you can make a quick but informed decision. Obviously it would be better to go through the full system integration route if this is a production system, so think about getting one into one of your UAT systems sooner rather than later.
0
 
MrVaultAuthor Commented:
LOL. A UAT system? If only I could convince them to spend the $$ to have an extra for testing. In all serious though I am trying to go that route.

Connollyg, I'm curious if you'd ever see the benefit of an iSCSI hba because if you're thinking it's not worth the benefit if it' near 70% utilization, then too much higher and the server is hosed. We've seen spikes easily hold 99% for a long time. the 40-70 is more a typical utilization. How far would you have it go before seeing the benefit? Most documents point to a reduction to 30% total utilization after implementing HBAs. of course YMMV.
0
 
Gerald ConnollyCommented:
99%! Well you didnt say that last time!

But seriously, it comes down to a cost benefit! If your server is running short of CPU resource then you have to work though the options of why and what, and then how you could reduce it.
Then you have to balance the resultant actions against cost! Would some other upgrade give you a bigger bang for your buck in overall system performance, or would, what we used to call a ToE (TCP Offload Engine) be cost effective. [ i know a iSCSI HBA isnt quite the same thing as a ToE, but close enough ]

And YES i am in favour of iSCSI HBA's but they have to be cost effective too.
0
 
MrVaultAuthor Commented:
Sorry for not getting back sooner. We're not always running at 99%, but it does happen on some of our servers for extended periods of time.

Does anyone know how to monitor the process as "mattvmotas" suggested?
0
 
DavidPresidentCommented:
The 60% overhead is a valid number, but only on single-core systems.  You will also see big performance penalties in older PCI-based systems where the NIC and the disk controllers are on the same bus.

On a multi-core PCI-e based system, you will see a much smaller impact that may or may not be a factor.   Easiest way to tell, is just take a downtime window when nothing else is going on, and configure iometer to set up a multi-threaded workload that emulates as closely as possible the type of I/O you do real-world.  If you had a modern quad or more processors, and a PCIe-2.0 system, then I would just save myself the trouble and not even worry about the overhead.  

If, however, you don't have such a server, then it is worthwhile doing a test.   I can't tell you the right I/O model, you are just going to have to figure out how many threads you need, and the proper mix of  of read/write/random/sequential I//O and correct block sizes.

The closer you are to the right synthetic load, the more accurate the results.  

If you have a dual processor, or uniprocessor, or don't have a PCIe based motherboard, then by all means, save yourself the time and get a iSCSI with the built-in processor ... you WILL need it.

So only go through the exercise if you are in this grey area


0
 
MrVaultAuthor Commented:
Thanks. These are all 2 year old servers with 2 quad core processors. I know it's not a guarantee but I'm pretty sure they all support PCIe.

They are all use the PERC6i raid controllers for the non-OS disks.
0
 
DavidPresidentCommented:
Then personally, I would not worry about it unless I needed to try to squeeze a little more network performance out of the systems. Keep an eye out for a bargain on a used card on ebay in the interim, but I wouldn't make it a priority.
0
 
MrVaultAuthor Commented:
Do you think the same conclusion should be drawn regarding segregating our network with VLANs and/or physical switches? Right now they have 3 switches on a 4gig backplane all in a single vlan so the wan connections, iscsi connections, san replication, etc are all over the same subnet. best pracitces are to segregate it out both for performance and security reasons. not to dismiss security, but right now their biggest concern is performance. I need to determine if this network config is a bottleneck and if so pitch a solution (like getting separate switches for iscsi traffic or segregating via VLANs).
0
 
DavidPresidentCommented:
No, segregating a network and using TOE for iSCSI are completely different.   Do your switches support RMON?  If so, they have a great deal of monitoring capability built-in, and you can see utilization and bottlenecks via any SNMP package, or perhaps through a built-in web interface.

0
 
MrVaultAuthor Commented:
And which things should I be looking for specifically? Seeing if they are running close to 1Gbps per port? Or looking for collisions or retransmits? They are Foundry managed switches so i'm guessing they support SNMP.
0
 
MrVaultAuthor Commented:
thanks guys. thoughts were helpful but it appears there's no metric to measure iSCSI overhead on a system without a test system or changing the config which involves downtime. I was hoping there was some perfmon counter. the first guy seemed to think there was a process but I'm guessing he was mistaken. oh well.
0
 
Gerald ConnollyCommented:
There has been too much debate and input from several experts to give away all this expertise without assigning points!
0
 
MrVaultAuthor Commented:
totally true. didn't mean to imply there'd be no points assigned. just hoping my latest comment might remind someone of a way :-)

Points awarded now.
0
 
MrVaultAuthor Commented:
no live method was given, but I'm guessing there is none. Only option is to have a like system with an HBA and measure differences.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 12
  • 4
  • 3
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now