asked on

vmware vsphere lost connectivity all guests powered off

I have 2 hosts with HA and FT, each has a fiber link to iSCSI san. Each also has separate a vswitch for guests (vswitch1) and console/vmkernel (vswitch0).

I lost physical connectivity to the switch (it was power cycled). I got an alert about vswith0 losing connectivity, no alert about vswitch1.

Here's the weird bit - when the switch came back online, ALL of the guests on both hosts were powered off.

As far as I can tell the fiber switch did not lose power - so the connection to the SAN *should* have been good the entire time.

My theory is that both hosts tried to vMotion their guests to the other host, which was also down, and the net result is everyting powers off.

Question 1: would loss of L1 connection for console/kernel cause all guests to end up powered off?

Question 2: if I lost connection from hosts to SAN, shouldn't I see an alert related to the storage adapters as well?

ASKER CERTIFIED SOLUTION

coolsport00

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

SOLUTION

Paul Solovyovsky

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

SOLUTION

rvivek_2002

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

snowdog_2112

ASKER

You are correct, the isolation was set to "power off", and the failuredetectiontime was the default - 15s. The switch was offline for about 45 minutes.

This is good, however...the VM guys (keep in mind, these are the knuckleheads who set this up - I'm just cleaning up the mess) said "the switch rebooted, so start a case with Cisco to look at the switch logs".

Whoo...that's funny stuff...

Um...yeah, I know the switch freaked out, but why did all the VM's POWER OFF! Wow.

Thanks for the links! Very useful stuff!

snowdog_2112

ASKER

I split points because coolsport was first, paulsolov led me to the info, and rvivek provided some good background/foundation info. THANKS A TON!!!

Paul Solovyovsky

Just a quick question. You said that each has a fiber link to an iSCSI SAN. Is the link to the SAN Fiber or Copper? Usually iSCSI is hardware or software initiator but most of the time the hardware initiator (HBA) is still cat45.