I will be astonished if anyone has the time to read this book and help, but I do desperately need assistance. Believe it or not, I have actually tried to abbreviate this question. If you have the time, please read it and give me your thoughts. It would be most appreciated.
** Environment **
Switching:
(2) MDS 9222i fibre channel fabric switches
Storage:
(1) StorageTek 6140, 1 fully populated 4TB tray
Storage to be divided equally between NTFS and VMFS vdisks/volumes
(1) StorageTek SL48 tape library
Hosts:
(1) Win2003R2 file server (gets the NTFS volume)
(4) ESX4.0 bare metal hosts cluster (get the VMFS volume)
All hosts have 2 NICs and 2 FC HBAs. Each HBA pair is jumpered to a different switch. Windows host has FC multipathing driver installed.
Applying LAN concepts to the SAN, concerns such as purpose, security, and availability drive segmentation. In the above case, the array will likely be in only one VSAN, but that's only because its disparate functions are served from a single physical tray. Otherwise, I'd put a designated VMware data source (which we don't have) in the same VSAN with VMware hosts. As it happens, all I can do is segment host and device types as best I can.
My initial thinking about installing the above equipment was to create VSAN200 for the array; VSAN300 for the file server; VSAN400 for all VMware hosts; and VSAN500 for the tape library. Both initiators for each host would reside in the same VSAN, though on separate switches, whether ISL'd or not. IVR zoning would then be employed to allow the FC tape library to talk to all VSAN hosts and all hosts on different VSANs to talk to the array. In my thinking, ISLing the switches would just keep me from duplicating the config. However, I have two stumbling blocks to what I thought was a straightforward setup.
First, it was strongly suggested that I completely forgo use of ISLs and instead maintain two physical switches, isolating their respective virtual fabrics but also creating for each host HBA pair non-intersecting VSANs, one on each isolated switch. To me, this seemed to be expectant of an ultimate ISL and, thus, contradictory. Contradictions aside, though, it was an idea that at least got me thinking my next point represented some kind of standard.
An instructor at a recent training venue suggested that in addition to the recommended "single-initiator-per-zone
(1 VSAN/HBA) (7 nodes) (2 HBAs/node) = 14 VSANs
...add to that 14 zonesets, etc., etc.
Questions
1. Should I ISL these switches or keep them isolated? They currently represent my "core".
2. What would you say my VSANs and zoning should look like?
Thanks!
This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.
Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.
If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.
Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.
Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.
Access the answers to your technology questions today.
30-day free trial. Register in 60 seconds.
Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.
Try it out and discover for yourself.
30-day free trial. Register in 60 seconds.
Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.
Ditto on not over-complicating it, we didn't have VSANs before Cisco introduced them and we really don't need them in any but the biggest SANs and even then we got away without them in the past. Some clustering requires ports of all nodes in a cluster to be in the same zone but most don't. ISLs just waste of ports since there's not going to be any data going through them.
Thanks folks. I agree that what I was talking about looks too complicated, which is why I posted these questions.
With respect to placing partner initiators into non-intersecting VSANs, here is the response I got from a former Verizon senior engineer:
>"You will get more reliability because you get individual fabric services per VSAN and a bad HBA and many LIP resets won't bring down the other servers. Especially for ESX, that's an important point. The older Linux kernel that ESX uses doesn't handle that too elegantly (especially with the Emulex driver) and a single HBA can severely impact a whole VSAN by sending too many LIPs. Nothing you want because that basically kills the idea of DRS, VMotion and such because all your servers will suffer from the same."
Except that you are in a switched fabric and LIPs (Loop Initialisation Primitive) aren't used. FC fabrics use other mechanisms. In any case, I haven't seen an FC port (loop or otherwise) jabber for years - the most common issue is flaky cables and SFPs which generate muliple FLOGI (Fabric LOG In) events. I have to say, I don't agree with the statement, and I do think you're over-complicating your environment.
>"...avoid interswitch links unless there is absolutely no way around it. Cisco has a long document on nothing other than troubleshooting ISLs and even the Cisco engineers we had on site for a while recommended that we get rid of ISLs unless absolutely needed."
Hmmmmmmm. That would apply to Cisco FC switches which are typically at over subscribed (that is; multiple FC ports share the bandwidth of a single switching ASIC). Some port cards are 8:1 over-subscribed - and if you use over-subscribed ports for ISLs, you *will* end in a world of pain. ISLs and fully meshed core-edge switching fabrics have been part of Fibre Channel since its inception and they work, and work well, but as with any network, you have to design it properly.
Thanks ...and just so you know, I am not disagreeing with your assessment, only trying to demonstrate the kind of information--or misinformation, as it were--that I, a SAN noob, am hearing.
Can't believe I didn't catch that about LIPs. That's a pretty glaring mistake, but in his defense maybe he just responded too quickly and was instead thinking of any kind of chatter that can bring down a VSAN ...much like broadcast storms bring down VLANs, sometimes from faulty hardware, sometimes accidental misconfiguration, and sometimes malice. Who knows.
I am glad to hear a different opinion on ISLs, and yes there is no way I'd configure one without a dedicated rate. I am not even sure NX-OS allows shared rate ISLs.
How about the tape issue? Any tips for zoning the tape library?
meyersd, I reread your initial comment. Sorry I got caught up in the VSAN explanation and forgot you already dealt with tape best practices. Just so I am clear, though, I need to zone the tape library to essentially present it to each host as if it were direct attached storage, right? Then I need to prevent its direct access to the array itself?
>NX-OS allows shared rate ISLs.
Worse - it blocks out a bunch of ports if they share the same switching ASIC!
>Just so I am clear, though, I need to zone the tape library to essentially present it to each host as if it were direct attached storage, right? Then I need to prevent its direct access to the array itself?
Yes - that's correct. If you are using NDMP, you may want the array and the tape library in the same zone, so the answer to that is "it depends"..
Business Accounts
Answer for Membership
by: meyersdPosted on 2009-08-29 at 02:33:28ID: 25213485
You can't overlay Ethernet concepts on Fibre Channel - they ain't the same. Your environment is not complicated - why then, make it complicated?
Create a single VSAN for each switch - VSAN 10, say for switch 1, VSAN 11 for switch 2. Put all the FC ports in the VSAN then create aliase, zones and zonesets. Single-initiator zones are best practice - so one host HBA port per zone with as many storage ports as you like. Best practice also separates tape and disk, so create a zone that maps host HBA to storage and a seperate zone for host HBA and the tape library. You might also consider splitting disk and tape I/O completely if you are having performance issues. You can ISL the switches if you want but I'm not sure I'd bother given your environment.
The reason for single initiator zoning is to reduce the impact of RSCNs - an RSCN is flooded to all "interested" ports - which can be disruptive to host I/O. More here: http://en.wikipedia.org/wi
Why MDS9222is by the way? Typically, you'd only use one if you needed FCIP or iSCSI functionality...