We help IT Professionals succeed at work.

QLogic Fibrechannel fabric setup

ein_mann_betrieb
on
Medium Priority
718 Views
Last Modified: 2013-11-14
Hello all,

I am trying to setup a redundant fabric for our SAN and I am getting strange behavior.
Our servers all have redundant HBAs with multipathing and we are moving up to a second switch.
Both are QLogic Sanbox2-16 with the same firmware v1.3-64-0 (latest we can run on our primary switch, so thats what we are running on both)

I thought I understood this, but apparently I am remiss...
I split the redundant connections from each of the servers and the storage array across both switches. Then I cross-connected the switches.

The servers see the storage array but any real I/O throws all kinds of errors.
When I disconnect the link between the switches, things seem Ok... but now all traffic is on one switch only and I can't load balance paths to the disk array.

Is there a way to implement these two switches as a single fabric?  Or is it not at all worth it?

Thanks!
-Cheers, Peter.

<Edit>
Forgot to mention...  I reset all my zoning so no zone is set and I did increment the ID of the second switch to 2.
Thanks again!
</Edit>
Comment
Watch Question

Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
Single Fabric - usually you link the FC switches.

Author

Commented:
Hi hanccocka,
  Yea, thats what I tried to do.  I setup the link ports as E ports, but the I/O errors are confusing me.
Is it possible that I have a bad switch port?  My SanSurfer application shows both switches when they are linked as green and no blocking is reported on the port linking them...

Thanks!
-Cheers, Peter.
Andrew Hancock (VMware vExpert PRO / EE Fellow)VMware and Virtualization Consultant
CERTIFIED EXPERT
Fellow
Expert of the Year 2017

Commented:
Yes, bad switch ports do exist!
Top Expert 2015

Commented:
qlogic devices need to be at same firmware level to couple them.

fabric is a term to loosely describe FC cabling and switchis as opposed to target and initiator which actually can be connected directly.

Author

Commented:
Hi hanccocka,

I tested the port on the primary switch by using it with a host.  It works fine for normal communications.  I will test the new switch's port tomorrow am.

Hi gheist,
  Both switches are running firmware v1.3-64-0...  Are there other software components in the switch that may be a cause?

Thanks to you both!
-Cheers, Peter.
Top Expert 2015

Commented:
interconnect with two cables?

Author

Commented:
Hi gheist,
  I had been only using one.  Ah... I take it that two are required?

I will try that tomorrow.   :-)

Thanks!  -Cheers, Peter.
Top Expert 2015
Commented:
Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION
saggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019
Commented:
Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION

Author

Commented:
Hi andyalder,
  Ok.  I can understand the zoning issue...  that makes sense.

We have a five-server esxi vmware cluster, all feeding from one switch.
We have two target servers in an Active/Standby configuration.
The end intent is to make the cluster as resilient to single component failures, while maximizing resources.
So we purchased an additional switch of the same model (just newer hardware revision) but one that could run the same software.

My initial thinking with the single fabric would be that we can provide extra bandwidth to each box rather than have the the second hbas just sit idle.

But maybe my approach is wrong.  I am still learning about FC...  I have much more experience with iSCSI than FC.


FYI gheist... I am going to still try the second uplink cable in just under an hour here.

Thanks!  -Cheers, Peter.
andyaldersaggar maker's framemaker
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
With iSCSI it's the same, have 2 separate unjoined fabrics if you can.

You haven't said which SAN storage you've got but even if it's active/active on individual LUNs you're very unlikely to have fast enough disks to swamp a single fibre channel port.
Top Expert 2015

Commented:
Dear asker, you are absolutely right that iSCSI is more cost effective, and it is of value gathering experience there.

Author

Commented:
Hi all,

Ok.  So I found my problem...  Looks like I had some mislabeled fiber that was causing the grief.  We ordered 62.5 micron and in the bag marked 62.5, we had two strands of 50 micron...  so that was my core problem.   Replaced the cable and now I have all four paths appearing in VMware.

So that solves my initial question.  I am looking to reconfigure as @andyalder suggests and setup two fabrics.  I see how dangerous it can be if the zoning goes bad on both switches if they are linked.

I do need to figure out how the standby server will be seen from VMware's side, but I think that deserves a separate question and separate points, so I will post again shortly.

Thanks to everyone for all your help.
-Cheers, Peter.
Top Expert 2015

Commented:
I would back andy's suggestion.
What is standby server seen from vmware? Please explain a bit more.
Unlock the solution to this question.
Thanks for using Experts Exchange.

Please provide your email to receive a sample view!

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.