NIC options for segregating iSCSI traffic?

Posted on 2011-03-24
Medium Priority
Last Modified: 2012-05-11
We have Dell Poweredge 2950, R510, and R710 servers. We want to use separate NICS for the iSCSI traffic to the servers than the LAN traffic. We need 2 of each (LAN and iSCSI) for HA.

I think the 2950 and R710 have 4 onboard NICs and the R510 has 2.

I've heard of iSCSI HBA's, regular gigabit NICs and then recently something about a TOE NIC which is kind of half way.

Any recommendations for which cards to buy? or if I should just use onboard NICs when possible? We don't anticipate booting from SAN but our CPU % utilization is above 70% quite often.

Question by:MrVault
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +1

Expert Comment

ID: 35213869
It would depend on the kind of load you anticipate on your storage. If you need high-performance (e.g. big SQL databases with lots of I/O, then spend the extra for the HBA or TOE option. If it's for file storage / user data where speed is not so critical, then you can save money by using a standard GB NIC. You could also think about teaming for redundancy.

Author Comment

ID: 35214340
it's a big i/o sql server. what does a toe nic do that an hba doesn't? which one is recommended for dell servers?

Accepted Solution

Chev_PCN earned 1000 total points
ID: 35214359
The options are not brand-specific, alhtough some companies may have their own branded options. HPO is one.
TOE is a NIC with a TCP/IP Offload Engine, which takes all the TCP/IP processing off the CPU and onto the NIC. The iSCSI HBA is an all-in-one NIC with TOE AND a HW initiator. For a big SQL server, go with the HBA - it will give you the best performance. I'm not sure, but I think they're around US$400. Again, think about redundancy if it's a mission-critical server.
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

LVL 42

Assisted Solution

kevinhsieh earned 1000 total points
ID: 35216267
Before you go out and buy stuff, do you know that the CPU utilization is related to iSCSI? You can start by looking at task manager and compare the SQL Server processor utilization compared to the total. If SQL is most of your CPU I don't think that a ToE or HBA is going to help much.

Are you using jumbo frames? If not you should turn them on for your iSCSI NICs, switch ports, and SAN. That can really reduce the CPU associated with I/O.

All of your servers have ToE capable LOMs. Check with Dell tech support about getting them activated at both the hardware and OS level.

Finally, adding RAM can usually reduce the I/O requirements of SQL, which would improve performance and reduce I/O related CPU utilization.

Alacratech was big in the iSCSI HBA market a few years ago when people thought that software iSCSI initiators were too risky. They were pretty pricy, like north of $1000 as I recall. I have no idea what the market is like now.

Author Comment

ID: 35216521
jumbo frames are on across the board. the servers are either maxed out on ram or it will cost $2000 to upgrade the ram. we're using 16GB chips, so we have to buy in pairs which is $1000 per.
LVL 42

Expert Comment

ID: 35217397
I would call Dell Tech support and ask them to help you with enabling ToE on your Broadcom LOMs. It's quite possible that you already have it as it's pretty much a default option when buying them. They also may be upgradeable to iSCSI offload. I would talk to your Dell server rep. It looks like Alacritech got out of the iSCSI HBA business, and that QLogic is still making stand alone cards.

You maxed out an R710 on RAM? That's 144 GB with 8 GB chips, and 288 GB with 16 GB chips. I am jealous.


Author Comment

ID: 35217619
no. we maxed out the 2950's. the r710's are not maxed out. we have 8x8gb. but it's not cheap to buy 2 more 8gb chips. the R510's are 4x16GB and even more expensive to buy 2x16GB chips to grow.

does TOE take the CPU part out of the CPU?
LVL 42

Expert Comment

ID: 35217879
ToE takes part of the CPU out. The CPU still needs to process iSCSI, but it takes off processing related to TCP/IP traffic.


Author Comment

ID: 35217913
thanks. maybe we'll try toe first and see how it preforms before buying hba's across the board.
LVL 70

Expert Comment

ID: 35455662
This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.

Featured Post

Prepare for your VMware VCP6-DCV exam.

Josh Coen and Jason Langer have prepared the latest edition of VCP study guide. Both authors have been working in the IT field for more than a decade, and both hold VMware certifications. This 163-page guide covers all 10 of the exam blueprint sections.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The recent Microsoft changes on update philosophy for Windows pre-10 and their impact on existing WSUS implementations.
Resolving an irritating Remote Desktop connection that stops your saved credentials from being used.
This tutorial will walk an individual through the steps necessary to configure their installation of BackupExec 2012 to use network shared disk space. Verify that the path to the shared storage is valid and that data can be written to that location:…
In this video, Percona Director of Solution Engineering Jon Tobin discusses the function and features of Percona Server for MongoDB. How Percona can help Percona can help you determine if Percona Server for MongoDB is the right solution for …
Suggested Courses

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question