Attaching multiple external SCSI storage arrays to a two node cluster

Posted on 2004-09-23
Medium Priority
Last Modified: 2013-11-15
Ok, so Im purchasing a two node win 2k3 cluster that will run SQL 2000 in an active/passive failover HA config.

I want to have an external SCSI storage array and have sourced the IBM EXP400 that will support 14 drives.

As I understand it - a single SCSI adapter can have 14 drives connected to it.

So in my external storage - one of the disks will be a quorum disk leaving me with 13 active drives for data.

What i want to know is whether i can have two SCSI adapters in each of my servers, and connect the two node cluster to two external SCSI storage enclosures.  That way I'd have 13 active drives in the first array and 14 active drives in the second array.  Is there any restriction on doing this?  Can I have two SCSI adapters running in my servers?  Would there be any restrictions on this imposed by Win 2k3 or SQL Server 2000 (i.e. will they only recognize a limited total number of SCSI disks?)

Basically - I don't want to be limited by the 2TB of storage I can get out of one external storage array.  I want to get about 4 TB but i don't to jump up to SAN.  I also want to stick with the major hardware vendors - as a very high level of SLA is required - if there's hardware failure I want the part replaced by the vendor within 10 hours or so.  Thats why I'm looking at IBM, Dell, HP.
Question by:kenshaw
LVL 30

Expert Comment

by:Duncan Meyers
ID: 12140672
Yes, you can. I've been involved with a 2 node cluster with each node connected to 2 Dell Powervault 220S SCSI arrays via LSI/AMI RAID controllers (aka Dell PERC 3/DC).

The only restriction is a slightly ludicrous one - you run out of drive letters (you'll have external 27 discs connected)! I'd suggest using RAID controllers so that you also have some redundancy in your configuration. Run the external discs as RAID 1/0 sets or RAID 5 stes depending on what you intend.

Hope this helps.
LVL 30

Expert Comment

by:Duncan Meyers
ID: 12140684
BTW - if you go with a RAID controller, you'll need one taht supports clustering - that is, you need to be able to change the controller SCSI address. Not all RAID controllers will let you do this.

Author Comment

ID: 12140721
ok... thanks for that.  When you say if I go for a RAID controller I'm a bit confused.  Won't my SCSI controller support RAID?  i.e. in my servers I'll have two adapters, and each adapter will be a SCSI/RAID controller won't they?  

Also - why do I need to be able to change the controller SCSI address for clustering?  To make sure that each node's SCSI cards are set up properly?  I don't get this bit.
Free Tool: Port Scanner

Check which ports are open to the outside world. Helps make sure that your firewall rules are working as intended.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

LVL 30

Expert Comment

by:Duncan Meyers
ID: 12140780
>why do I need to be able to change the controller SCSI address for clustering?  To make sure that each node's SCSI cards are set up properly?  I don't get this bit.

Each node has to have different SCSI addresses on the controller, typically 7 for one node and six for the other. If you don't change the controller SCSI addresses you'll have an instant conflict and the cluster simply won't work. So you'd set the controller SCSI addresses on one node to 7 (same on each SCSI card) and on the other node 6 (again, same on each SCSI card).

>When you say if I go for a RAID controller I'm a bit confused.  Won't my SCSI controller support RAID?  i.e. in my servers I'll have two adapters, and each adapter will be a SCSI/RAID controller won't they?  

Not necessarily. Depends what you've got. For example, an Adaptec 39160 is a SCSI controller only - no RAID. If you want to set up RAID, you'd need to use software RADI within your OS (Dynamic discs in Disk Manager for Windows, Software RAID in Disk Druid in Red Hat Linux and so on). An Dell PERC 3/DC is both a SCSI and a hardware RAID controller. With this, you can group a number of discs into a RAID array - RAID 5, 1/0, 1 and so on.

Given what you've posted about you're storage requirements, I'd strongly reccommend you look at an entry level SAN such as EMC's CX300. It  gives you a greater degree of performance, flexibilityand expandability than direct attach storage. And it isn't *that* much more expensive. Apparently they start at around AUS$20K - which is not that much more tahn what you'll spend with the direct attach anyway (I'd estimate AUS$15K-16K for what you're describing). The extra money buys you an extra *heap* of reliability and robustness.

FWIW. Sorry to complicate it for you, but it's far better to make the right decision now, rather than try and make a crappy system work well.
LVL 56

Expert Comment

ID: 12142721
HP's DL380 packaged clusters can be used with redundant RAID controllers if you use the high availability kit for the MSA500 and you don't lose 2 SCSI addresses as it's not a shared SCSI bus so you still get 14 or 28 drives and because it has an internal RAID controller you don't have to dedicate a disk for the quorum but simply define a 1GB slice of an array for it.

Have you thought about using SQL replication rather than clustering or using something like doubletake? For clustering you need Windows enterprise and SQL enterprise and that can cost more than the hardware.
LVL 97

Accepted Solution

Lee W, MVP earned 1000 total points
ID: 12145263
I used to manage two Windows clustered file servers using both a SAN and direct attach SCSI.  This is from my experience:

1.  STAY AWAY FROM DELL.  If you want a server, notebook, desktops, Dell is fine, I even recommend them.  I've had a 50% failure rate on Dell backup devices in the first year and roughly a 50% failure rate on Dell SCSI enclosures (200S, 210S, 220S).  Doesn't matter the OS, this was on Linux and Windows.  Both at a client, and at my day job.  Both clustered and non-clustered.  I had probably 8-10 of these devices and the number of times multiple disks failed AND the number of backplanes replaced on the SCSI enclosures just scared me and created nightmares.  That combined with the fact that Dell doesn't use NEW parts for mission critical parts replacement, they use USED/REFURBISHED parts (at least in every instance where I saw/asked), using Dell Storage devices is a recipe for disaster.  The one exception to this was the Dell PowerVault 650F and expansion 630F's.  These devices haven't been sold by Dell in 3-5 years, but ours (1x 650F+4x 630F Expansion units) were both the most reliable storage and hard drives I've ever had the pleasure of using.  These were Fibre SAN devices, basically rebranded clariion units - which is now EMC.

2.  I can't speak for other manufacturers.  Before I left that job (a little over two months ago), we were switching to IBM and Aberdeen equipment (Aberdeeninc.com - a small manufacturer that had some NICE configurations - 6TB of storage on one 5U box using 3Ware RAID and SATA drives - For $10-15K-US$.)

3.  You might be able to create a SCSI cluster from regular SCSI cards... BUT, especially with clusters - you want to be able to get MS support.  Which means sticking to the Hardware Compatibility List (HCL).  I actually did setup a cluster using a couple of SCSI drives and a couple of Adaptec 2910 SCSI controllers.  That worked.  I tried again more recently with a couple of 29160 controllers, that worked briefly then went offline and I haven't had it running since (6 months).  These were my own test clusters since I'm not affording your typically EMC device or RAID Controllers.

4.  Stay away from Dell PERC cards.  They MIGHT not be bad, but in addition to replacing the Dell storage unit components, they replaced these cards a number of times.  I simply don't trust them.  And half the time, they didn't perform correctly - taking a system down when a single drive failed (it should running with a single drive failure).

5.  I have to agree with Andy Adler - unless you have an open budget, I would strongly consider SQL replication technologies rather than clustering.  Obviously, this depends somewhat on what you are doing with the database, but the licenses for SQL server and Windows Enterprise server are anything but cheap.

6. Quorum - makes little sense to put it on a disk by itself - I understand that's what Microsoft recommends, but Quorum takes up FAR LESS than 1 GB of storage.  If you're sticking 14x146 GB drives in a system, you are loosing 145 GB+.  Further you aren't making your quorum redundant - if you did, then your loosing 290GB+ for a mirrored quorum drive.  You're also loosing another 146 GB for a HOT SPARE - which you don't NEED, but REALLY SHOULD HAVE.  Plus as point 8 below illustrates, you can generally only access 13 drives in a SCSI Cluster, so you loose yet another 146GB.  That's over half a terabyte of wasted space.  Quorum can reside on the same RAID container as your data.  Just use Windows to cut a 1GB partition for it and leave the rest to your data.

7.  SCSI RAID Controllers/SCSI Controllers.  I've had servers with 4 SCSI controllers and 2 RAID controllers.  There's generally no problems with this.  The only problems are in running out of IRQ/IO addresses and with server technology and IRQ sharing, this isn't really a problem anymore.

8.  SCSI cards have a SCSI ID for themselves.  As do RAID cards.  You can have 16 devices (SCSI card Included) on a SCSI or SCSI RAID card.  Each SCSI RAID Card needs 1 SCSI ID.  The SCSI backplane/Enclosure typically (at least with Dell units) requires one SCSI ID.  That means 3 IDs are used.  That leaves 13 IDs left.  So a 14 Drive enclosure is only going to make 13 drives usable at all.  That 14th drive can be a cold spare, but NOT a HOT spare - to be a hot spare, it would need it's own SCSI ID - which there aren't enough of.

In short, SPEND THE MONEY on an EMC Fibre SAN (NOT NAS) storage unit and you'll be setting things up with growth potential and that will LIKELY (but nothing is ever guarenteed) keep your reliability headaches to a minimum.
LVL 56

Expert Comment

ID: 12145928
IBM's unit is a shared SCSI bus, you have to use special Y cables to provide termination when one of the servers is powered off. That's why I like HP's offering, whether fibre or SCSI attached each SCSI is buffered onto a PCI bus in the RAID box, then through the internal RAID controller onto another SCSI bus for the drives. With dual attached servers there's SecurePath software so that the OS doesn't see both sets of drives and think you've got twice as much storage than you really have.

Our firm sold a packaged cluster in all good faith as a stand alone solution so we had to impliment a domainlet since there were no other servers to hold the Global Catalog etc. Got called out because it went wrong and when I arrived on site the software developer said he didn't want a cluster! Made me take the shared drives out and put them in the servers instead, remove AD (which was what broke) and said he was going to use sql replication instead. Now it's going to a co-lo hosting site.

Anyone want a 3 month old MSA 500, rack, UPS, KVM switch and rackmount keyboard? (joke, I know EE isn't eBay).

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

The following article sheds light on easy-to-use steps to recover non-responding hard drives without data loss. Count on these approaches to fix undetectable, not responding, or non-working hard drives.
Organisation is organized in a pattern to flow the day to day business, every application and system is interdepended on each other and when very important “Exchange Server downtime” happened.
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…
Despite its rising prevalence in the business world, "the cloud" is still misunderstood. Some companies still believe common misconceptions about lack of security in cloud solutions and many misuses of cloud storage options still occur every day. …

586 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question