Solved

Redundancy with a Point to Point (pt to pt) SAN

Posted on 2009-07-07
10
730 Views
Last Modified: 2013-11-14
Hello Experts,
I have an old SAN that I bought to familiarize myself with the technology and configuration options outside of a production environment.  (I am a network Engineer and we are in the process of moving to Blades and soon a SAN).  This SAN has proved to not only be an invaluable learning tool but also much more usefulness that I had originally anticipated.  I am now trying to determine the best redundancy configuration that I can put it in.

The SAN is a Infortrend IFT-6330-12.  2GB Fibre channel model, 128mb Cache Raid Controller, 12 IDE ATA Drives, Point to Point (pt to pt) topology. The SAN is populated with (12) x 200GBs drives, out of which I configured (2) x RAID 5 arrays; 6 Disks in each, giving me (2) x 1TB Volumes.  I currently have the SAN connected to a Emulex LP902-DC FC 2GB HBA, connected to a Dell PowerEdge Tower Server.

It took some learning and hard trial & error but I have the unit running quite well. I used 1 array for my nightly backups and was using the other array as random storage.  With the upcoming RTM release of Server 2008 R2, I have been working a lot with the R2 Beta and Hyper-V (moving VMs from Virtual Server 2005 to Hyper-V, etc).  On a whim I threw a VHD on the catch all SAN array and was shocked at the performance.  (ATA with a decent cached controller runs better than SATA through onboard mobo port.  Who'd have thought it?)

Since then, I have been researching SAN Point to Point topology and trying to determine my options in regard to redundancy/availability.  It is my understanding that from an availability standpoint, Point to Point SAN is pretty close to Direct Attached storage.  Access to storage area network is dependant on the Host (Host's OS).  From a configuration standpoint, I defined LDs and/or LVs, setup  the Raid configuration, Mask/Map the LUNS, then configure the FC access. HBA connects to the SAN to the host.  The SAN's arrays are formatted by the Host's OS.  From that point on the access (reads, writes, etc) are done by or through a connect to the Host.

My issue is that I would like to use the storage on the SAN to house VHDs/VMs that will be running on machines other than the host.  (I know, technically NAS, Network Storage, etc is a no-no with VMs but it can be done.  Have even though of using an opensource iSCSI solution to manage the SAN arrays but that is getting off topic).  

My problem boils down to if the SAN is housing data being used by multiple machines, and its host needs to reboot, gets turned off, HBA has an fault etc; it would present an issue.  Ultimately I'd like to configure some redundancy to mitigate this.  

Although it was my understanding that this is one of the draw backs with Pt to Pt SAN topology, I have done my homework and read about a number of scenarios that accomplish some of this.  (1 Pt to Pt SAN, using multiple PIDs for each LUN, multiple HBAs, and multiple fibre switches for mirrored fibre fabric, etc).  I  have also read about various SAN/HBA multipath/multi-pathing solutions.  Additionally I recently took the certification exams for SharePoint and read a decent amount about NLB and Clustering and wondered it that could also lend to this.

Although I believe in theory there may be a combination of these technologies that accomplish my desired out come, I haven't come across anything that covered a Point to Point configuration, with Multiple HBA's, and hosts being configured in exactly this manner.

I know my goal is reaching and before I purchase any additional components I wanted to get some feedback.

Thanks,
jbhaire2004
0
Comment
Question by:JBHaire2004
  • 5
  • 4
10 Comments
 
LVL 55

Expert Comment

by:andyalder
ID: 24792768
You cannot have proper reundancy with that SAN box because it only has one controller. You can certainly connect it to several hosts and create LUNS for them using fibre channel switches. You won't get new 2Gb switches and there are very few 4Gb ones left in the channel, they are 8GB now but will negotiate down to 2Gb.

point to point in the SAN world implies switched fabric, not direct attached. Since your infortrend box ofers LUN masking and supports up to 1024 hosts it must support fabric.
0
 

Author Comment

by:JBHaire2004
ID: 24799376
I agree that there is a single point of failure with the SAN controller but my focus is not redundancy to avoid an internal failure of the SAN itself; but rather of a failure of the SAN's Host machine.  

The most likely issues I see is that the Server acting as the SAN's Host will reboot, get turn off, or be disrupted in some way.  This interruption in the SANs Host availability would sever access to the storage on the SAN.  

AND MAYBE THERE LIES MY ISSUE...  

I am working under the assumption, per the IFT-6330-12's manual, that the point to point san can have only 1 physical Host.  It is also documented in the manual that the manner to allow access to the SANs Storage is by creating a FC connect between the SANs FC controller and the SAN Hosts FC HBA, after which the HBA and SAN controller are configured to communicated, the SAN Host then sees the connected volume.  The IFT-6630-12 Manual then states the method to allow access to the SAN storage from other servers is to create a network share for the drive with the SAN Host managing the volume.  

The root of my thinking is based on the assumption that there can be only 1 SAN Host.  If that is not a false assumption, I would like to determine if there is a way to cluster the SANs Host so that there are 2 separate boxes managing the SAN.

Barring something I may have missed or miss understood, I want to wind up with 2 Hosts for a given LUN, (Cluster, Active/Passive, however), so that if SAN Host 1 needs a maintenance reboot, SAN Host 2 can keep the access up uninterrupted.  

If I am missing something here or should be looking at this differently please let me know.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24801434
It seems rather unlikely that it would only be able to support one host. Do you have an URL to the manual and reference to where it says that. It may only support one host directly connected but if you use a fabric switch it ought to support more and they ought to be able to share LUNs.
0
 
LVL 2

Expert Comment

by:martino87r
ID: 24803259
Well for production environments I'll go always for an Infiniband fabric network. Usually they provide good reliability and fast speed. For redundancy you have to go for a redundant file system (like GFS).

Obviously is a very hard work to build up a project like that, I'll suggest you to check these documents to improve your skills:

http://www.redhat.com/gfs/
https://www.redhat.com/docs/manuals/csgfs/pdf/rh-gfs-en-6_0.pdf

and inifiniband fabric switched networks:
http://en.wikipedia.org/wiki/Infiniband
0
 

Author Comment

by:JBHaire2004
ID: 24805547
The IFT-6330-12 Manual can be found at:  http://www.a1computer.de/download/ift_6330_12.pdf

As I re-read this again I think that I was not understanding some elements properly.

"2.1.7 RAID Sharing (Network)
RAID sharing with IFT-6330-12 subsystems involves allowing more
than just the host computer to store and retrieve data from the unit.

Once a RAID array has been created and assigned a drive
designation, 2-it can be shared to a LAN or WAN using the same
OS-dependent method as other host storage devices (e.g., 2-"local"
hard drives) are shared. Consult your OS documentation for
information as to how to share storage devices.

In order to connect the array to different host computers, 2-host
channel IDs or LUN numbers should be mapped with different
logical configurations (logical drives). If there is only one logical
configuration, 2-different ID/LUNs can be mapped with the same
array capacity. However, 2-file-locking mechanism must be
implemented with management software running to handle the
sharing."

"1.3.5 Notice for Fibre Channel Connection
The system Fibre port is not equipped with by-pass circuits. Bypass
services should be provided outside the system. Connection
to the host should be made using a 3rd party hub or switch, unless
the system is connected point-to-point to a single host computer."

"4.1.1 Host Requirements
In order for the IFT-6330-12F2D (Fibre models) to be connected to
the host computer, the host must have a fibre host bus adapter
installed and a SFP module. Most commonly, a host bus adapter
(HBA) is an add-on card that includes external fibre ports used to
connect with fibre devices, including the IFT-6330-12 array. It is
necessary to use an SFP module to connect the HBA to the IFT-
6330-12. If your HBA does not come with an SFP module, you will
not be able to connect it to the IFT-6330-12. Please contact our
technical support to obtain IFTs Quality Vendor List (QVL) to
determine which SFP modules and HBAs are compatible with the
IFT-6330-12.

Use Fiber optic cables to connect to host. If connection to host is
made to a GBICs converter, an LC-SC cable may be used. You may
use cables recommended by your HBA manufacturer. Do not bend
or allow the cable to be damaged following the instructions by the
manufacturer of the cable you select.

Please note that there is no by-pass circuits along the host port. Bypass
circuitry must be provided with a Fibre hub or switch.

Once the array has been connected, you must reboot the host and
then assign the RAID array a drive letter designation. Each OS has a
different procedure for assigning drive letters to storage devices.
Consult your particular OSs documentation and the documentation
for your HBA for more information."
0
Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

 

Author Comment

by:JBHaire2004
ID: 24805616
Additionally I found another Infortrend manual with more specific information that may help point me in the right directions.

http://ftp4.de.freesbie.org/pub/misc/infortrend/ftp.infortrend.com.tw/EonRAID_SentinelRAID/Manual/GMv162.pdf
0
 
LVL 55

Accepted Solution

by:
andyalder earned 500 total points
ID: 24805792
I'm getting a server not found for the aicomputer.de download site at the moment but basically that says you can share a LUN between two servers so long as you buy a switch. The normal thing to do is to list the WWPNs or WWNNs of both hosts against the LUN masking configuration so the particular (array in their terms) is presented to both.

The rest is a generic warning about two servers without special software (clustering etc) will corrupt the data if they both access the same LUN.

So now you need a switch, if you're buying new I'd get a Brocade 8/8 (from HP, IBM, Dell or whoever your preferred server vendor is), if you're getting second hand there are some Brocade Silkworm 3200s on eBay for about $250.
0
 

Author Comment

by:JBHaire2004
ID: 24806194
That is odd about the server being unavailable.  I have attached the pdf to this post.

I will look at picking up a Silkworm 3200.  In the interim, if I were to pick up a less expensive switch like a Vixel Emulex Inspeed 330 would I be able to accomplish the majority of the same functions?

Also when I connect 2 hosts to a switch and then (as you suggested), "list the WWPNs or WWNNs of both hosts against the LUN masking configuration so the particular (array in their terms) is presented to both", will there be any additional steps required so that the hosts can both work with the array when required?  I am green in this area so you have to excuse me.  I do remember specifying WWPNs and WWNNs in the SAN controller software and the HBA management software when I set up the point to point configuration.  It was to my understanding that if 2 hosts connected to a given array, Host 1 would be unaware if Host 2 was making changes to the array/volume, and such if Host 1 and Host 2 both tried to write to the volume/array at the same time there is the possibility that they could write over data that was written by the other causing corruption.  Is that correct?  Additionally, the Infortrend manual made a reference to, "If there is only one logical configuration, 2-different ID/LUNs can be mapped with the same array capacity. However, 2-file-locking mechanism must be implemented with management software running to handle the sharing."  Would that be handled by the HBAs, Switch or Hosts' OS's; or would I need to take additional steps?  I do not intend to have the 2 Hosts working with the array at the same time, rather I would like for the array to remain available by Host 2 if Host 1 was shut down.

GMv162.pdf
0
 

Author Comment

by:JBHaire2004
ID: 24835238
I bought a Silkworm 3200.  Should be here in a few days.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 24835377
>It was to my understanding that if 2 hosts connected to a given array, Host 1 would be unaware if Host 2 was making changes to the array/volume, and such if Host 1 and Host 2 both tried to write to the volume/array at the same time there is the possibility that they could write over data that was written by the other causing corruption.  Is that correct?

It's inevitable, hosts store some of the file information in RAM, so data will become corrupted if two hosts have access to the same LUN unless you use clustering or a shared file system. However you can create several LUNs and present them to different servers so they are not shared.

Various options exist to share data between more than one server, active/passive clustering like MSCS, VMware HA etc. and there are shared filesystems that use file locking or special clients like Quantum's StorNext.

0

Featured Post

Google Storage: Standard vs. Nearline vs. Coldline

Google Cloud Storage has a number of classes to choose from. Although there are a lot in common, they vary in price and usage terms. This post explains Google Cloud Storage classes and helps to understand which  one to choose.

Join & Write a Comment

How to update Firmware and Bios in Dell Equalogic PS6000 Arrays and Hard Disks firmware update.
The article will include the best Data Recovery Tools along with their Features, Capabilities, and their Download Links. Hope you’ll enjoy it and will choose the one as required by you.
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
How to install and configure Citrix XenApp 6.5 - Part 1. In this video tutorial we have explained step by step installation of Citrix XenApp 6.5 Server on Windows Server 2008 R2 is explained in this video. We have explained the difference between…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now