How do I validate a 2008 R2 cluster that has an iSCSI network?

I am trying to build a 3 node 2008 R2 cluster.  Each node has (8) network ports.  (2) on the LAN, (2) dedicated to Hyper-V guests and hidden from the host OS, and (4) on an iSCSI SAN network.  To allow for iSCSI offload, the node's SAN ports are not firewalled at the host OS level.  For added security, the network switch on the SAN network has port isolation turned on so all server ports can talk to the SAN storage but the server ports cannot talk to each other.  This prevents a possibly compromised machine from attacking others across the SAN network.  The cluster nodes are not the only servers connected to the SAN network.  All SAN ports on the host nodes are using IPs on the same private subnet.  All LAN ports on the host nodes are using IPs on the same public subnet.

When I try to validate the cluster, it fails saying that the SAN ports are on the same subnet but cannot ping each other.  That is by our design.  We only want iSCSI traffic using those ports and we do not want servers communicating with each other on the SAN network.  The nodes can still reach each other on the LAN ports and that should be enough for validation.

Is there a way to exclude the node's SAN ports from the validation test?  Is there another way to pass validation?
CMES-ITAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

abelenkiyCommented:
change your binding order in advanced network settings.
also make sure you configure which network does what private/public access in the failover cluster manager
0
CMES-ITAuthor Commented:
I changed the order in the advanced settings but it did not change the validation results.  It may also be worth noting that I had already removed most of the bindings from the SAN iSCSI nics already.  SMB networking and such had already been removed.  The only bound protocols are those necessary to allow the iSCSI and MPIO to function.

Also, how would I change the settings in the cluster manager if it won't let me create the cluster until it validates?  Or will it?
0
abelenkiyCommented:
What about adding additional nic that will go to a switch that only has the private networks of each node connected to it for heartbeat?
0
Making Bulk Changes to Active Directory

Watch this video to see how easy it is to make mass changes to Active Directory from an external text file without using complicated scripts.

CMES-ITAuthor Commented:
I am setting up an isolated, private network between the nodes to see if that helps.  I will report back.
0
CMES-ITAuthor Commented:
So, I setup an isolated network and, after struggling to get around the fact that 2008 will not let you choose the firewall profile for neworks that do not have a gateway (different story), I was able to set each node up on that network too.

It did not help though.  The new, isolated network passes validation but the SAN network still fails because of the ping.  That, in turn, causes the whole validation to be flagged as failed, even though everything else passes.
0
msmamjiCommented:
Can you temporarily allow communication between the iSCSI NIC ports and see if it passes?
0
msmamjiCommented:
Have you tried unchecking the "File and Printer Sharing for Microsoft Networks" and "Client for Microsoft Networks" in the iSCSI NIC properties and check if validation passes?
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
CMES-ITAuthor Commented:
"Can you temporarily allow communication between the iSCSI NIC ports and see if it passes?"

I can and it does.  However, if I was to need support from Microsoft, I'm sure one of the first things they would want to do is run the validator.

"Have you tried unchecking the "File and Printer Sharing for Microsoft Networks" and "Client for Microsoft Networks" in the iSCSI NIC properties and check if validation passes?"

All protocols that are not needed to support iSCSI have been disabled on the iSCSI NICs, including the SMB networking.
0
msmamjiCommented:
Can you just allow ICMP traffic on the iSCSI NIC b/w the servers and check?
I think you will have to make a trade off here as its seems to be a limitation you will have to abide by in order to have it the cluster validated.

Regards,
Shahid
0
CMES-ITAuthor Commented:
After opening a case with the storage vendor and reviewing as much MS documentation as possible, I'm presented with the only solution being to remove the port isolation from the SAN switches and allow the pings to pass.  My situation is identical in theory to this MS KB:  http://support.microsoft.com/kb/951434.

I would consider this a shortcoming in that validation process since, in the majority of scenarios, cluster traffic will not be passing across the SAN network and the only validation should be whether the nodes can connect to the storage.  Not whether the iSCSI NICs can ping each other.  The reference in the KB that paths to the storage should be on separate subnets if they cannot ping each other is an unrealistic solution in most scenarios since many storage vendors need the same subnet for MPIO.

So, the answer is to make sure all protocols other than TCP/IP are disabled on the SAN NICS, remove the port isolation from the switches, and hope we don't get caught by another "vulnerability in the TCP/IP stack could allow remote code execution."
0
CMES-ITAuthor Commented:
This answer combined with removing the port isolation at the switches allowed validation.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Storage

From novice to tech pro — start learning today.