[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1873
  • Last Modified:

AX4 Dual Ch iSCSI with ESXi server

We are getting set to migrate some servers to VMware ESXi and NAS storage using an EMC AX4 with dual processors.  After performing several speed test we determined that using raw RDMs was slightly faster and used less CPU than having each guest OS use their own iSCSI initiator.  However, I'd like advice on configuration of the server side topology - for presentation of the iSCSI to the ESX host.  

We have a completely independent iSCSI fabric using two redundant Gigabit Switches and two processors on the EMC Clariion AX4.  And each server has two dedicated nics for iSCSI traffic.  Our concept was to use two subnets, one for each processor.  This would make the four iSCSI targets:

A0=172.31.1.150
A1=172.31.1.151
B0=172.31.2.150
B1=172.31.2.151

In further tests, we set up two independent ESXi servers.  One configured with a second virtual switch using two physical adapters bound to it.  And a second using a second and third virtual switch, each with a single nic bound to them.  Both of these configurations works without error.  However, we have conflicting test data between iometer, jetstress, sqlio, and the VMware performance indicators.  So, I like to get some real world feedback from users with similar setups or experience.

What are the pros and cons of these two configurations:

ESXi #1
vSwitch 0 = vmnic0 & vmnic1 (for LAN traffic)
vSwitch 1 = vmnic2 & vmnic3 (for iSCSI traffic to both subnets)

ESXi #2
vSwitch 0 = vmnic0 & vmnic1 (for LAN traffic)
vSwitch 1 = vmnic2 (for iSCSI traffic to 1st subnet)
vSwtich 2=  vmnic3 (for iSCSI traffic to 2nd subnet)

Any additional comments or advice welcome.  Thanks in advance.
0
pwm001
Asked:
pwm001
  • 4
1 Solution
 
kumarnirmalCommented:
How are you planning to use your SAN ? In Active Active Mode or Active Passive Mode ?

When choosing RDM feature, please bear in mind that choosing Virtual Compatibility Mode would allow you to make use of Snapshot feature which is not available in the Physical Compatibility Mode.

Another suggestion and as per VMware, the best practices indicate to segregate your Service Console, VM and iSCSI Traffic using separate NICs and also using VLANs.
0
 
pwm001Author Commented:
I'm unable to find a setting for either active/active or active/passive on the EMC AX-4.  This system comes with Navisphere Express.  It may take the full blown Navisphere to control that.  I'll attempt to find out what it's default is - as obviouslly this is moot for an active/passive scenario.  

The traffic is going to be segregated.  The iSCSI is on two completely independant switches.
0
 
pwm001Author Commented:
The Clariion is Active/Active.
0
Get your Disaster Recovery as a Service basics

Disaster Recovery as a Service is one go-to solution that revolutionizes DR planning. Implementing DRaaS could be an efficient process, easily accessible to non-DR experts. Learn about monitoring, testing, executing failovers and failbacks to ensure a "healthy" DR environment.

 
aldanchCommented:
What are the pros and cons of these two configurations:

ESXi #1
vSwitch 0 = vmnic0 & vmnic1 (for LAN traffic)
vSwitch 1 = vmnic2 & vmnic3 (for iSCSI traffic to both subnets)

ESXi #2
vSwitch 0 = vmnic0 & vmnic1 (for LAN traffic)
vSwitch 1 = vmnic2 (for iSCSI traffic to 1st subnet)
vSwtich 2=  vmnic3 (for iSCSI traffic to 2nd subnet)

Are your physical NICs on the motherboard (Quad-port) or two Dual-port (2 on Mobo, 2 on another NIC)?

If latter, then use 1 NIC per card for your VMkernel (Management Network) and your LAN. This gives you additional redundancy in case one of the dualport cards fail, you are able to failover to another card.
For example if vmnic0 & 1 are on Card1 and vmnic2 & 3 on Card2 then

ESX1
vSwitch0 (VMkernel + LAN)
Port group 0 (VMkernel) = vmnic0 (active adapter), vmnic1 (standby adapter)
Port group 1 (LAN) = vmnic2 (active adapter), vmnic0 (standby adapter)
vSwitch1 (iSCSI)
Port group 0 = vmnic1 (172.16.31.1.x iSCSI subnet), vmnic3 = unused
Port group 1 = vmnic3 (172.16.31.2.x iSCSI subnet), vmnic1 = unused

You can also separate vSwitch1 to two vSwitches if you like (like your case with ESX2). Alternatively, you can place all your NICs on one vSwitch (0) and create port groups and assign them accordingly.

For example
ESX1
Port group 0 (VMkernel) = vmnic0 (active adapter), vmnic1 (standby adapter)
Port group 1 (LAN) = vmnic2 (active adapter), vmnic0 (standby adapter)
Port group 2 (iSCSI SubnetA) = vmnic1 (172.16.31.1.x iSCSI subnet), vmnic0, 2, 3 = unused
Port group 3 (iSCSI SubnetB) = vmnic3 (172.16.31.2.x iSCSI subnet), vmnic0, 1, 2 = unused

My recommendation is to add an additional NIC (quad-port is best, but dual-port if you can't afford) or replace your dualport NIC with a Quad-port to increase ports available. Then you can set up your NICs to have redundant paths.

You have a choice of separating your network by using either port groups on the same vswitch or using vswitches to do it. Whichever works best for you. If you can add more NICs, then you can use port groups and dedicate NICs for LAN and VMkernel for additional redundancy. If you don't care for redundancy, then either setup you have will work.
0
 
pwm001Author Commented:
aldanch,

Great idea seperating motherboard/cards.  On one of our servers are 2 & 2.  The other 4 servers have 4 on-board nics.

But really my question is regarding throughput not redundacy, as we have two complete fully redundant paths to the LUNS in either scenario.  Reading through the muli-vendor post that lightbulb recommended is pretty comprehensive, but intense.  From this, I believe they are saying that ESX (and ESXi) 3.5 can only establish a single connection to each SCSI target.  And a session must take place on a single connection - meaning not concurrently accross two connections.  Therefore, the maximum throughput will be that of a single TCP connection - in my case 1GB.  Link aggregation will not increase the throughput and load balancing is best handled through manual route targeting.

So in the case of Exchagne & SQL (or large intense database type) traffic, using the ESX initiator is probably NOT the best method to use.  Although our test indicated this to have higher throughput per connection, the ESX server doesn't handle MPIO in the same method as PowerPath would in the guest OS.  AT THIS TIME...of course!

All this seems to make theoretical sense, however...conflicting test data remains.  

What other setups are users having success with and there pros and cons.  For instance, the two setups of aldanch.
0
 
pwm001Author Commented:
Didn't answer my complete question, but very good information.
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now