Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

QLogic Fibrechannel fabric setup

Posted on 2011-04-19
14
Medium Priority
?
706 Views
Last Modified: 2013-11-14
Hello all,

I am trying to setup a redundant fabric for our SAN and I am getting strange behavior.
Our servers all have redundant HBAs with multipathing and we are moving up to a second switch.
Both are QLogic Sanbox2-16 with the same firmware v1.3-64-0 (latest we can run on our primary switch, so thats what we are running on both)

I thought I understood this, but apparently I am remiss...
I split the redundant connections from each of the servers and the storage array across both switches. Then I cross-connected the switches.

The servers see the storage array but any real I/O throws all kinds of errors.
When I disconnect the link between the switches, things seem Ok... but now all traffic is on one switch only and I can't load balance paths to the disk array.

Is there a way to implement these two switches as a single fabric?  Or is it not at all worth it?

Thanks!
-Cheers, Peter.

<Edit>
Forgot to mention...  I reset all my zoning so no zone is set and I did increment the ID of the second switch to 2.
Thanks again!
</Edit>
0
Comment
Question by:ein_mann_betrieb
  • 5
  • 5
  • 2
  • +1
14 Comments
 
LVL 124
ID: 35433720
Single Fabric - usually you link the FC switches.
0
 
LVL 5

Author Comment

by:ein_mann_betrieb
ID: 35433799
Hi hanccocka,
  Yea, thats what I tried to do.  I setup the link ports as E ports, but the I/O errors are confusing me.
Is it possible that I have a bad switch port?  My SanSurfer application shows both switches when they are linked as green and no blocking is reported on the port linking them...

Thanks!
-Cheers, Peter.
0
 
LVL 124
ID: 35433875
Yes, bad switch ports do exist!
0
Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

 
LVL 62

Expert Comment

by:gheist
ID: 35436497
qlogic devices need to be at same firmware level to couple them.

fabric is a term to loosely describe FC cabling and switchis as opposed to target and initiator which actually can be connected directly.
0
 
LVL 5

Author Comment

by:ein_mann_betrieb
ID: 35437015
Hi hanccocka,

I tested the port on the primary switch by using it with a host.  It works fine for normal communications.  I will test the new switch's port tomorrow am.

Hi gheist,
  Both switches are running firmware v1.3-64-0...  Are there other software components in the switch that may be a cause?

Thanks to you both!
-Cheers, Peter.
0
 
LVL 62

Expert Comment

by:gheist
ID: 35437032
interconnect with two cables?
0
 
LVL 5

Author Comment

by:ein_mann_betrieb
ID: 35437809
Hi gheist,
  I had been only using one.  Ah... I take it that two are required?

I will try that tomorrow.   :-)

Thanks!  -Cheers, Peter.
0
 
LVL 62

Assisted Solution

by:gheist
gheist earned 400 total points
ID: 35439244
Dual path gives warm feeling that you can exchange one cable without disrupting service (i have 5 switches and double cables between, topology resembles token-ring)

0
 
LVL 56

Accepted Solution

by:
andyalder earned 1600 total points
ID: 35439892
>Is there a way to implement these two switches as a single fabric?  Or is it not at all worth it?

It's counter-productive, you want two separate fabrics for redundancy. Say the zone table on the principle switch gets corrupted, it'll corrupt the other switch as well. Also it's a waste of ports as no data is going to go over those ISLs.

Without knowing your SAN storage I can't tell why traffic is only going through 1 HBA but it's possible that there's only one LUN owned by one controller.
0
 
LVL 5

Author Comment

by:ein_mann_betrieb
ID: 35442000
Hi andyalder,
  Ok.  I can understand the zoning issue...  that makes sense.

We have a five-server esxi vmware cluster, all feeding from one switch.
We have two target servers in an Active/Standby configuration.
The end intent is to make the cluster as resilient to single component failures, while maximizing resources.
So we purchased an additional switch of the same model (just newer hardware revision) but one that could run the same software.

My initial thinking with the single fabric would be that we can provide extra bandwidth to each box rather than have the the second hbas just sit idle.

But maybe my approach is wrong.  I am still learning about FC...  I have much more experience with iSCSI than FC.


FYI gheist... I am going to still try the second uplink cable in just under an hour here.

Thanks!  -Cheers, Peter.
0
 
LVL 56

Expert Comment

by:andyalder
ID: 35442701
With iSCSI it's the same, have 2 separate unjoined fabrics if you can.

You haven't said which SAN storage you've got but even if it's active/active on individual LUNs you're very unlikely to have fast enough disks to swamp a single fibre channel port.
0
 
LVL 62

Expert Comment

by:gheist
ID: 35444004
Dear asker, you are absolutely right that iSCSI is more cost effective, and it is of value gathering experience there.
0
 
LVL 5

Author Comment

by:ein_mann_betrieb
ID: 35486935
Hi all,

Ok.  So I found my problem...  Looks like I had some mislabeled fiber that was causing the grief.  We ordered 62.5 micron and in the bag marked 62.5, we had two strands of 50 micron...  so that was my core problem.   Replaced the cable and now I have all four paths appearing in VMware.

So that solves my initial question.  I am looking to reconfigure as @andyalder suggests and setup two fabrics.  I see how dangerous it can be if the zoning goes bad on both switches if they are linked.

I do need to figure out how the standby server will be seen from VMware's side, but I think that deserves a separate question and separate points, so I will post again shortly.

Thanks to everyone for all your help.
-Cheers, Peter.
0
 
LVL 62

Expert Comment

by:gheist
ID: 35489479
I would back andy's suggestion.
What is standby server seen from vmware? Please explain a bit more.
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Finding original email is quite difficult due to their duplicates. From this article, you will come to know why multiple duplicates of same emails appear and how to delete duplicate emails from Outlook securely and instantly while vital emails remai…
Is your phone running out of space to hold pictures?  This article will show you quick tips on how to solve this problem.
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
Despite its rising prevalence in the business world, "the cloud" is still misunderstood. Some companies still believe common misconceptions about lack of security in cloud solutions and many misuses of cloud storage options still occur every day. …
Suggested Courses
Course of the Month21 days, 2 hours left to enroll

810 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question