Solved

Best practices for switching portion of iSCSI setup

Posted on 2013-01-14
10
901 Views
Last Modified: 2013-05-05
I do not normally touch network side and am struggling getting my head around exact requirements/preferences for iSCSI traffic. So, I thought I'd break down to see if I get a consensus. I have a single HP 2810 switch for the iSCSI traffic and have 3 Vmware hosts and a small SAN with 6 ports plugged into the switch. My issue is the Vmware side seems to disconnect from SAN and I have been going around with Vmware, SAN vendor and switch vendor. Hosts have multiple iSCSI NICs and vNIC configured round robin.

I have updated to most current firmware on both and Vmware 5.1. SAN vendor says that iSCSI session is dropping after 5 min or so. Basically Vmware performs slowly all day.

Spanning tree- I have heard both ways whether it should be enabled or not. Last version I read said enabled but set for edge on HP switch. Can't find exact guide on how to do this.

Flow control- I have not had this set and want to try and several references suggest.

Jumbo frames- I do not have configured across as I did not want to introduce new issues until I resolved original.

Any other suggestions or settings on the iSCSI switch side, let me know? I am attaching my current config on switch 1 and I am about to add sw2 so I don't have the multiple VLANs (so I will be transplanting the storage VLAN configuration).
sw1-config-2013-01-11.txt
0
Comment
Question by:smeek
  • 5
  • 4
10 Comments
 
LVL 57

Expert Comment

by:giltjr
ID: 38776627
Is the iSCSI connection from host or the guest operating systems?

Since there is a single VLAN on the iSCSI ports spanning-tree should be disabled on those ports.

Is that file the whole configuration for the switch?  I have not used the HP 2810, but that config looks fairly sparse to me.

Do you have NIC's that are dedicated for iSCSI on the VMWare host?

I'm not sure using round robin for iSCSI is a good choice.
0
 
LVL 8

Author Comment

by:smeek
ID: 38776647
iSCSi from host.
HP config much shorter than comparable Cisco config.
Using Hardware iSCSI (4 NICs per host and Vmware verified all is correct on the Vmware side of things for iSCSI config).
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38776847
Found one  post:

http://communities.vmware.com/thread/297185?start=15&tstart=0

That seem to indicate the 2810 does not have enough buffer memory to keep up under heavy load.  Are you stressing the system?
0
What is SQL Server and how does it work?

The purpose of this paper is to provide you background on SQL Server. It’s your self-study guide for learning fundamentals. It includes both the history of SQL and its technical basics. Concepts and definitions will form the solid foundation of your future DBA expertise.

 
LVL 8

Author Comment

by:smeek
ID: 38776867
My understanding was that buffer was only important during contention or in blocking switches.

Utilization is not high. HP support and I pulled logs and they said issue is not the switch, but of course, I am looking for second opinion.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38777979
What iSCSI SAN is it?

If it is disconnecting that often you might be able to do a packet capture and see what is going on.  I am assuming that the HP Switch supports port mirroring.

Is the traffic low enough that you could disable/disconnect some of the NIC's?  That would make it easier to do the packet capture from the switch.
0
 
LVL 8

Author Comment

by:smeek
ID: 38778614
I appreciate your help trying to troubleshoot but my post was aimed at finding what the best practices were (enable or disable functionality). Do you have hands on knowledge of what functions should be enabled and why on my switch?
0
 
LVL 57

Accepted Solution

by:
giltjr earned 250 total points
ID: 38778700
Best practices from a switch point of view:

1) Enable Jumbo Frames and use the biggest common frame size.
2) Enable QOS.  Especially important when traffic is flowing over any port using tagged VLAN's.
3) iSCSI traffic should be on isolated ports when possible.  That is do not use tagged interfaces and mix iSCSI and non-iSCSI traffic on the same port if at all possible.  If you do, see #2.
4) Spanning-tree should be disabled on any port that you know for sure is connected to a host.  

IIRC VMWare's vswitches will prevent loops and so any connection to a VMWare host should be treated as being connected to a host, not to another switch.  So you should disable spanning-tree on the ports connected to the VMWare hosts.
0
 
LVL 10

Assisted Solution

by:ddiazp
ddiazp earned 250 total points
ID: 38780672
The things i'd check that nobody has mentioned:

1. Cables - are the cables in good working condition? Are these the proper cables? Do interfaces on the switch flap? Can you check interface statistics and look for packet drops, packet errors?

2. Speed and duplex - what's configured on vmware hosts, switch and san? i'd hardcode them all.

3. If you're enabling jumbo frames, you must do it everywhere (switch, san and vmware hosts). Check MTU on all those 3, they should all be the same. Switch may need to be higher to allow for extra frame headers

4. Perhaps there's a duplicate IP address in use by some host somewhere? If a host is using one of the IPs of the SAN, you'd have similar issues

5. What's the path status of all 3 hosts? Go to Configuration>Storage, select each datastore, and look at the Paths for each one. They should be either Active, Active (I/O) or Standby. If you see 'Dead' then verify your vmkernel interfaces, IPs in use, subnets, etc.
0
 
LVL 8

Author Closing Comment

by:smeek
ID: 38794470
I will try verify all and follow general suggestions. Thanks for your feedback.

Someone else told me I need Cat 6 cables. The cables are newer Cat 5 cables but I think they should be adequate as we just changed them all out.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38794505
To do GigE you need Cat 5e at a minimum.  In environments where you will really push GigE to its limits it is suggested you use Cat6.

Cat 6 is even certified for 10 G for short runs (20 meters).
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Unable to reach an IP located on a Dell switch from an HP switch 18 57
Raid 0 2 63
should I enable deduplication in this case? 4 43
Drivers  Repository for Win 7 13 59
#Citrix #Citrix Netscaler #HTTP Compression #Load Balance
This article will inform Clients about common and important expectations from the freelancers (Experts) who are looking at your Gig.
Viewers will learn how to connect to a wireless network using the network security key. They will also learn how to access the IP address and DNS server for connections that must be done manually. After setting up a router, find the network security…
After creating this article (http://www.experts-exchange.com/articles/23699/Setup-Mikrotik-routers-with-OSPF.html), I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

825 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question