Solved

Hyper v, HP ISCSI SAN

Posted on 2014-11-17
3
247 Views
Last Modified: 2014-11-29
Morning

We currently have an issue with iSCSI connectivity on a setup we have inherited from another IT provider.

To summerise, this is a Hyper-V 2K8 R2 cluster, which is using CSV's through ISCSI stored on a HP MSA 2324i iSCSI SAN.  Each host has 2 NIC's for ISCSI, and they are on there own subnet.  They are split between two switches for redundancy - nothing has changed prior to the fault.

Friday evening, all of the VM's went offline and the storage was marked as failed in the cluster.  On investigation I pinged the iSCSI interfaces on the SAN from the hyper-V hosts.  The ping times were either 1300ms+ or they were dropped, on all controller interfaces, from all hosts.

After several hours the storage came back online, the cluster storage could be brought back online and the ISCSI initiators on the hosts went from "reconnecting" to "connected".  Nothing had changed to bring it back online.

Several hours later the same issue occurs.  After 20+ hours it comes back up again, for one hour then dropped.

I have tried the following:

- Removed the physical cables and tried one at a time into each controller
- restarted the storage and management controllers
- restarted the hosts
- changed the IP's on the host interfaces on the SAN
- enabled loop protection on the switches

Given that all of the hosts are disconnected I am certain the SAN is the issue.  In the logs of the SAN you can see the port connections going up and down, however everything is reporting as healthy.  

Colleague is on the phone to HP but thought I would throw it out to EE in case anyone has had a similar issue they managed to fix.....

Thanks
0
Comment
Question by:DLeaver
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
3 Comments
 
LVL 24

Expert Comment

by:VB ITS
ID: 40447187
I've seen a similar issue with a three node cluster, except my pings would just time out the majority of the time. Are all of your DCs on the cluster?

When you restarted the storage and management controllers, did you restart the hosts at the same time?
0
 
LVL 12

Accepted Solution

by:
DLeaver earned 0 total points
ID: 40462150
Apologies for the delay

The issue ended up being due to another colleague plugging in two network hubs in a seperate part of the building.

The cabling at this site appears to be a little odd, and it wasn't behaving like a normal loop.  The degradation increased more by the day.  As soon as it was stated there had been two hubs added they were removed and the network performance came back - thats what happens when you are relying on all of the information being relayed and it isn't!

Marking as solved myself
0
 
LVL 12

Author Closing Comment

by:DLeaver
ID: 40471414
Solution found from further troubleshooting onsite
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

In this article we will learn how to backup a VMware farm using Nakivo Backup & Replication. In this tutorial we will install the software on a Windows 2012 R2 Server.
The business world is becoming increasingly integrated with tech. It’s not just for a select few anymore — but what about if you have a small business? It may be easier than you think to integrate technology into your small business, and it’s likely…
How to install and configure Citrix XenApp 6.5 - Part 1. In this video tutorial we have explained step by step installation of Citrix XenApp 6.5 Server on Windows Server 2008 R2 is explained in this video. We have explained the difference between…
In this video tutorial I show you the main steps to install and configure  a VMware ESXi6.0 server. The video has my comments as text on the screen and you can pause anytime when needed. Hope this will be helpful. Verify that your hardware and BIO…

756 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question