NIC options for segregating iSCSI traffic?

Posted on 2011-03-24
Last Modified: 2012-05-11
We have Dell Poweredge 2950, R510, and R710 servers. We want to use separate NICS for the iSCSI traffic to the servers than the LAN traffic. We need 2 of each (LAN and iSCSI) for HA.

I think the 2950 and R710 have 4 onboard NICs and the R510 has 2.

I've heard of iSCSI HBA's, regular gigabit NICs and then recently something about a TOE NIC which is kind of half way.

Any recommendations for which cards to buy? or if I should just use onboard NICs when possible? We don't anticipate booting from SAN but our CPU % utilization is above 70% quite often.

Question by:MrVault
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
  • +1

Expert Comment

ID: 35213869
It would depend on the kind of load you anticipate on your storage. If you need high-performance (e.g. big SQL databases with lots of I/O, then spend the extra for the HBA or TOE option. If it's for file storage / user data where speed is not so critical, then you can save money by using a standard GB NIC. You could also think about teaming for redundancy.

Author Comment

ID: 35214340
it's a big i/o sql server. what does a toe nic do that an hba doesn't? which one is recommended for dell servers?

Accepted Solution

Chev_PCN earned 250 total points
ID: 35214359
The options are not brand-specific, alhtough some companies may have their own branded options. HPO is one.
TOE is a NIC with a TCP/IP Offload Engine, which takes all the TCP/IP processing off the CPU and onto the NIC. The iSCSI HBA is an all-in-one NIC with TOE AND a HW initiator. For a big SQL server, go with the HBA - it will give you the best performance. I'm not sure, but I think they're around US$400. Again, think about redundancy if it's a mission-critical server.
Get 15 Days FREE Full-Featured Trial

Benefit from a mission critical IT monitoring with Monitis Premium or get it FREE for your entry level monitoring needs.
-Over 200,000 users
-More than 300,000 websites monitored
-Used in 197 countries
-Recommended by 98% of users

LVL 42

Assisted Solution

kevinhsieh earned 250 total points
ID: 35216267
Before you go out and buy stuff, do you know that the CPU utilization is related to iSCSI? You can start by looking at task manager and compare the SQL Server processor utilization compared to the total. If SQL is most of your CPU I don't think that a ToE or HBA is going to help much.

Are you using jumbo frames? If not you should turn them on for your iSCSI NICs, switch ports, and SAN. That can really reduce the CPU associated with I/O.

All of your servers have ToE capable LOMs. Check with Dell tech support about getting them activated at both the hardware and OS level.

Finally, adding RAM can usually reduce the I/O requirements of SQL, which would improve performance and reduce I/O related CPU utilization.

Alacratech was big in the iSCSI HBA market a few years ago when people thought that software iSCSI initiators were too risky. They were pretty pricy, like north of $1000 as I recall. I have no idea what the market is like now.

Author Comment

ID: 35216521
jumbo frames are on across the board. the servers are either maxed out on ram or it will cost $2000 to upgrade the ram. we're using 16GB chips, so we have to buy in pairs which is $1000 per.
LVL 42

Expert Comment

ID: 35217397
I would call Dell Tech support and ask them to help you with enabling ToE on your Broadcom LOMs. It's quite possible that you already have it as it's pretty much a default option when buying them. They also may be upgradeable to iSCSI offload. I would talk to your Dell server rep. It looks like Alacritech got out of the iSCSI HBA business, and that QLogic is still making stand alone cards.

You maxed out an R710 on RAM? That's 144 GB with 8 GB chips, and 288 GB with 16 GB chips. I am jealous.


Author Comment

ID: 35217619
no. we maxed out the 2950's. the r710's are not maxed out. we have 8x8gb. but it's not cheap to buy 2 more 8gb chips. the R510's are 4x16GB and even more expensive to buy 2x16GB chips to grow.

does TOE take the CPU part out of the CPU?
LVL 42

Expert Comment

ID: 35217879
ToE takes part of the CPU out. The CPU still needs to process iSCSI, but it takes off processing related to TCP/IP traffic.

Author Comment

ID: 35217913
thanks. maybe we'll try toe first and see how it preforms before buying hba's across the board.
LVL 70

Expert Comment

ID: 35455662
This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

For anyone that has accidentally used newSID with Server 2008 R2 (like I did) and hasn't been able to get the server running again because you were unlucky (as I was) and had no backups - I was able to get things working by doing a Registry Hive rec…
WARNING:   If you follow the instructions here, you will wipe out your VTP and VLAN configurations.  Make sure you have backed up your switch!!! I recently had some issues with a few low-end Cisco routers (RV325) and I opened a case with Cisco TA…
This tutorial will give a short introduction and overview of Backup Exec 2012 and how to navigate and perform basic functions. Click on the Backup Exec button in the upper left corner. From here, are global settings for the application such as conne…
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …

717 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question