Solved

Best practices for switching portion of iSCSI setup

Posted on 2013-01-14
10
897 Views
Last Modified: 2013-05-05
I do not normally touch network side and am struggling getting my head around exact requirements/preferences for iSCSI traffic. So, I thought I'd break down to see if I get a consensus. I have a single HP 2810 switch for the iSCSI traffic and have 3 Vmware hosts and a small SAN with 6 ports plugged into the switch. My issue is the Vmware side seems to disconnect from SAN and I have been going around with Vmware, SAN vendor and switch vendor. Hosts have multiple iSCSI NICs and vNIC configured round robin.

I have updated to most current firmware on both and Vmware 5.1. SAN vendor says that iSCSI session is dropping after 5 min or so. Basically Vmware performs slowly all day.

Spanning tree- I have heard both ways whether it should be enabled or not. Last version I read said enabled but set for edge on HP switch. Can't find exact guide on how to do this.

Flow control- I have not had this set and want to try and several references suggest.

Jumbo frames- I do not have configured across as I did not want to introduce new issues until I resolved original.

Any other suggestions or settings on the iSCSI switch side, let me know? I am attaching my current config on switch 1 and I am about to add sw2 so I don't have the multiple VLANs (so I will be transplanting the storage VLAN configuration).
sw1-config-2013-01-11.txt
0
Comment
Question by:smeek
  • 5
  • 4
10 Comments
 
LVL 57

Expert Comment

by:giltjr
ID: 38776627
Is the iSCSI connection from host or the guest operating systems?

Since there is a single VLAN on the iSCSI ports spanning-tree should be disabled on those ports.

Is that file the whole configuration for the switch?  I have not used the HP 2810, but that config looks fairly sparse to me.

Do you have NIC's that are dedicated for iSCSI on the VMWare host?

I'm not sure using round robin for iSCSI is a good choice.
0
 
LVL 8

Author Comment

by:smeek
ID: 38776647
iSCSi from host.
HP config much shorter than comparable Cisco config.
Using Hardware iSCSI (4 NICs per host and Vmware verified all is correct on the Vmware side of things for iSCSI config).
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38776847
Found one  post:

http://communities.vmware.com/thread/297185?start=15&tstart=0

That seem to indicate the 2810 does not have enough buffer memory to keep up under heavy load.  Are you stressing the system?
0
 
LVL 8

Author Comment

by:smeek
ID: 38776867
My understanding was that buffer was only important during contention or in blocking switches.

Utilization is not high. HP support and I pulled logs and they said issue is not the switch, but of course, I am looking for second opinion.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38777979
What iSCSI SAN is it?

If it is disconnecting that often you might be able to do a packet capture and see what is going on.  I am assuming that the HP Switch supports port mirroring.

Is the traffic low enough that you could disable/disconnect some of the NIC's?  That would make it easier to do the packet capture from the switch.
0
Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

 
LVL 8

Author Comment

by:smeek
ID: 38778614
I appreciate your help trying to troubleshoot but my post was aimed at finding what the best practices were (enable or disable functionality). Do you have hands on knowledge of what functions should be enabled and why on my switch?
0
 
LVL 57

Accepted Solution

by:
giltjr earned 250 total points
ID: 38778700
Best practices from a switch point of view:

1) Enable Jumbo Frames and use the biggest common frame size.
2) Enable QOS.  Especially important when traffic is flowing over any port using tagged VLAN's.
3) iSCSI traffic should be on isolated ports when possible.  That is do not use tagged interfaces and mix iSCSI and non-iSCSI traffic on the same port if at all possible.  If you do, see #2.
4) Spanning-tree should be disabled on any port that you know for sure is connected to a host.  

IIRC VMWare's vswitches will prevent loops and so any connection to a VMWare host should be treated as being connected to a host, not to another switch.  So you should disable spanning-tree on the ports connected to the VMWare hosts.
0
 
LVL 10

Assisted Solution

by:ddiazp
ddiazp earned 250 total points
ID: 38780672
The things i'd check that nobody has mentioned:

1. Cables - are the cables in good working condition? Are these the proper cables? Do interfaces on the switch flap? Can you check interface statistics and look for packet drops, packet errors?

2. Speed and duplex - what's configured on vmware hosts, switch and san? i'd hardcode them all.

3. If you're enabling jumbo frames, you must do it everywhere (switch, san and vmware hosts). Check MTU on all those 3, they should all be the same. Switch may need to be higher to allow for extra frame headers

4. Perhaps there's a duplicate IP address in use by some host somewhere? If a host is using one of the IPs of the SAN, you'd have similar issues

5. What's the path status of all 3 hosts? Go to Configuration>Storage, select each datastore, and look at the Paths for each one. They should be either Active, Active (I/O) or Standby. If you see 'Dead' then verify your vmkernel interfaces, IPs in use, subnets, etc.
0
 
LVL 8

Author Closing Comment

by:smeek
ID: 38794470
I will try verify all and follow general suggestions. Thanks for your feedback.

Someone else told me I need Cat 6 cables. The cables are newer Cat 5 cables but I think they should be adequate as we just changed them all out.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38794505
To do GigE you need Cat 5e at a minimum.  In environments where you will really push GigE to its limits it is suggested you use Cat6.

Cat 6 is even certified for 10 G for short runs (20 meters).
0

Featured Post

Netscaler Common Configuration How To guides

If you use NetScaler you will want to see these guides. The NetScaler How To Guides show administrators how to get NetScaler up and configured by providing instructions for common scenarios and some not so common ones.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Join Greg Farro and Ethan Banks from Packet Pushers (http://packetpushers.net/podcast/podcasts/pq-show-93-smart-network-monitoring-paessler-sponsored/) and Greg Ross from Paessler (https://www.paessler.com/prtg) for a discussion about smart network …
Finding original email is quite difficult due to their duplicates. From this article, you will come to know why multiple duplicates of same emails appear and how to delete duplicate emails from Outlook securely and instantly while vital emails remai…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

920 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now