• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1032
  • Last Modified:

Best practices for switching portion of iSCSI setup

I do not normally touch network side and am struggling getting my head around exact requirements/preferences for iSCSI traffic. So, I thought I'd break down to see if I get a consensus. I have a single HP 2810 switch for the iSCSI traffic and have 3 Vmware hosts and a small SAN with 6 ports plugged into the switch. My issue is the Vmware side seems to disconnect from SAN and I have been going around with Vmware, SAN vendor and switch vendor. Hosts have multiple iSCSI NICs and vNIC configured round robin.

I have updated to most current firmware on both and Vmware 5.1. SAN vendor says that iSCSI session is dropping after 5 min or so. Basically Vmware performs slowly all day.

Spanning tree- I have heard both ways whether it should be enabled or not. Last version I read said enabled but set for edge on HP switch. Can't find exact guide on how to do this.

Flow control- I have not had this set and want to try and several references suggest.

Jumbo frames- I do not have configured across as I did not want to introduce new issues until I resolved original.

Any other suggestions or settings on the iSCSI switch side, let me know? I am attaching my current config on switch 1 and I am about to add sw2 so I don't have the multiple VLANs (so I will be transplanting the storage VLAN configuration).
sw1-config-2013-01-11.txt
0
smeek
Asked:
smeek
  • 5
  • 4
2 Solutions
 
giltjrCommented:
Is the iSCSI connection from host or the guest operating systems?

Since there is a single VLAN on the iSCSI ports spanning-tree should be disabled on those ports.

Is that file the whole configuration for the switch?  I have not used the HP 2810, but that config looks fairly sparse to me.

Do you have NIC's that are dedicated for iSCSI on the VMWare host?

I'm not sure using round robin for iSCSI is a good choice.
0
 
smeekAuthor Commented:
iSCSi from host.
HP config much shorter than comparable Cisco config.
Using Hardware iSCSI (4 NICs per host and Vmware verified all is correct on the Vmware side of things for iSCSI config).
0
 
giltjrCommented:
Found one  post:

http://communities.vmware.com/thread/297185?start=15&tstart=0

That seem to indicate the 2810 does not have enough buffer memory to keep up under heavy load.  Are you stressing the system?
0
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

 
smeekAuthor Commented:
My understanding was that buffer was only important during contention or in blocking switches.

Utilization is not high. HP support and I pulled logs and they said issue is not the switch, but of course, I am looking for second opinion.
0
 
giltjrCommented:
What iSCSI SAN is it?

If it is disconnecting that often you might be able to do a packet capture and see what is going on.  I am assuming that the HP Switch supports port mirroring.

Is the traffic low enough that you could disable/disconnect some of the NIC's?  That would make it easier to do the packet capture from the switch.
0
 
smeekAuthor Commented:
I appreciate your help trying to troubleshoot but my post was aimed at finding what the best practices were (enable or disable functionality). Do you have hands on knowledge of what functions should be enabled and why on my switch?
0
 
giltjrCommented:
Best practices from a switch point of view:

1) Enable Jumbo Frames and use the biggest common frame size.
2) Enable QOS.  Especially important when traffic is flowing over any port using tagged VLAN's.
3) iSCSI traffic should be on isolated ports when possible.  That is do not use tagged interfaces and mix iSCSI and non-iSCSI traffic on the same port if at all possible.  If you do, see #2.
4) Spanning-tree should be disabled on any port that you know for sure is connected to a host.  

IIRC VMWare's vswitches will prevent loops and so any connection to a VMWare host should be treated as being connected to a host, not to another switch.  So you should disable spanning-tree on the ports connected to the VMWare hosts.
0
 
ddiazpCommented:
The things i'd check that nobody has mentioned:

1. Cables - are the cables in good working condition? Are these the proper cables? Do interfaces on the switch flap? Can you check interface statistics and look for packet drops, packet errors?

2. Speed and duplex - what's configured on vmware hosts, switch and san? i'd hardcode them all.

3. If you're enabling jumbo frames, you must do it everywhere (switch, san and vmware hosts). Check MTU on all those 3, they should all be the same. Switch may need to be higher to allow for extra frame headers

4. Perhaps there's a duplicate IP address in use by some host somewhere? If a host is using one of the IPs of the SAN, you'd have similar issues

5. What's the path status of all 3 hosts? Go to Configuration>Storage, select each datastore, and look at the Paths for each one. They should be either Active, Active (I/O) or Standby. If you see 'Dead' then verify your vmkernel interfaces, IPs in use, subnets, etc.
0
 
smeekAuthor Commented:
I will try verify all and follow general suggestions. Thanks for your feedback.

Someone else told me I need Cat 6 cables. The cables are newer Cat 5 cables but I think they should be adequate as we just changed them all out.
0
 
giltjrCommented:
To do GigE you need Cat 5e at a minimum.  In environments where you will really push GigE to its limits it is suggested you use Cat6.

Cat 6 is even certified for 10 G for short runs (20 meters).
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Cloud Class® Course: Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

  • 5
  • 4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now