Attaching multiple external SCSI storage arrays to a two node cluster

Ok, so Im purchasing a two node win 2k3 cluster that will run SQL 2000 in an active/passive failover HA config.

I want to have an external SCSI storage array and have sourced the IBM EXP400 that will support 14 drives.

As I understand it - a single SCSI adapter can have 14 drives connected to it.

So in my external storage - one of the disks will be a quorum disk leaving me with 13 active drives for data.

What i want to know is whether i can have two SCSI adapters in each of my servers, and connect the two node cluster to two external SCSI storage enclosures.  That way I'd have 13 active drives in the first array and 14 active drives in the second array.  Is there any restriction on doing this?  Can I have two SCSI adapters running in my servers?  Would there be any restrictions on this imposed by Win 2k3 or SQL Server 2000 (i.e. will they only recognize a limited total number of SCSI disks?)

Basically - I don't want to be limited by the 2TB of storage I can get out of one external storage array.  I want to get about 4 TB but i don't to jump up to SAN.  I also want to stick with the major hardware vendors - as a very high level of SLA is required - if there's hardware failure I want the part replaced by the vendor within 10 hours or so.  Thats why I'm looking at IBM, Dell, HP.
Who is Participating?
Lee W, MVPConnect With a Mentor Technology and Business Process AdvisorCommented:
I used to manage two Windows clustered file servers using both a SAN and direct attach SCSI.  This is from my experience:

1.  STAY AWAY FROM DELL.  If you want a server, notebook, desktops, Dell is fine, I even recommend them.  I've had a 50% failure rate on Dell backup devices in the first year and roughly a 50% failure rate on Dell SCSI enclosures (200S, 210S, 220S).  Doesn't matter the OS, this was on Linux and Windows.  Both at a client, and at my day job.  Both clustered and non-clustered.  I had probably 8-10 of these devices and the number of times multiple disks failed AND the number of backplanes replaced on the SCSI enclosures just scared me and created nightmares.  That combined with the fact that Dell doesn't use NEW parts for mission critical parts replacement, they use USED/REFURBISHED parts (at least in every instance where I saw/asked), using Dell Storage devices is a recipe for disaster.  The one exception to this was the Dell PowerVault 650F and expansion 630F's.  These devices haven't been sold by Dell in 3-5 years, but ours (1x 650F+4x 630F Expansion units) were both the most reliable storage and hard drives I've ever had the pleasure of using.  These were Fibre SAN devices, basically rebranded clariion units - which is now EMC.

2.  I can't speak for other manufacturers.  Before I left that job (a little over two months ago), we were switching to IBM and Aberdeen equipment ( - a small manufacturer that had some NICE configurations - 6TB of storage on one 5U box using 3Ware RAID and SATA drives - For $10-15K-US$.)

3.  You might be able to create a SCSI cluster from regular SCSI cards... BUT, especially with clusters - you want to be able to get MS support.  Which means sticking to the Hardware Compatibility List (HCL).  I actually did setup a cluster using a couple of SCSI drives and a couple of Adaptec 2910 SCSI controllers.  That worked.  I tried again more recently with a couple of 29160 controllers, that worked briefly then went offline and I haven't had it running since (6 months).  These were my own test clusters since I'm not affording your typically EMC device or RAID Controllers.

4.  Stay away from Dell PERC cards.  They MIGHT not be bad, but in addition to replacing the Dell storage unit components, they replaced these cards a number of times.  I simply don't trust them.  And half the time, they didn't perform correctly - taking a system down when a single drive failed (it should running with a single drive failure).

5.  I have to agree with Andy Adler - unless you have an open budget, I would strongly consider SQL replication technologies rather than clustering.  Obviously, this depends somewhat on what you are doing with the database, but the licenses for SQL server and Windows Enterprise server are anything but cheap.

6. Quorum - makes little sense to put it on a disk by itself - I understand that's what Microsoft recommends, but Quorum takes up FAR LESS than 1 GB of storage.  If you're sticking 14x146 GB drives in a system, you are loosing 145 GB+.  Further you aren't making your quorum redundant - if you did, then your loosing 290GB+ for a mirrored quorum drive.  You're also loosing another 146 GB for a HOT SPARE - which you don't NEED, but REALLY SHOULD HAVE.  Plus as point 8 below illustrates, you can generally only access 13 drives in a SCSI Cluster, so you loose yet another 146GB.  That's over half a terabyte of wasted space.  Quorum can reside on the same RAID container as your data.  Just use Windows to cut a 1GB partition for it and leave the rest to your data.

7.  SCSI RAID Controllers/SCSI Controllers.  I've had servers with 4 SCSI controllers and 2 RAID controllers.  There's generally no problems with this.  The only problems are in running out of IRQ/IO addresses and with server technology and IRQ sharing, this isn't really a problem anymore.

8.  SCSI cards have a SCSI ID for themselves.  As do RAID cards.  You can have 16 devices (SCSI card Included) on a SCSI or SCSI RAID card.  Each SCSI RAID Card needs 1 SCSI ID.  The SCSI backplane/Enclosure typically (at least with Dell units) requires one SCSI ID.  That means 3 IDs are used.  That leaves 13 IDs left.  So a 14 Drive enclosure is only going to make 13 drives usable at all.  That 14th drive can be a cold spare, but NOT a HOT spare - to be a hot spare, it would need it's own SCSI ID - which there aren't enough of.

In short, SPEND THE MONEY on an EMC Fibre SAN (NOT NAS) storage unit and you'll be setting things up with growth potential and that will LIKELY (but nothing is ever guarenteed) keep your reliability headaches to a minimum.
Duncan MeyersCommented:
Yes, you can. I've been involved with a 2 node cluster with each node connected to 2 Dell Powervault 220S SCSI arrays via LSI/AMI RAID controllers (aka Dell PERC 3/DC).

The only restriction is a slightly ludicrous one - you run out of drive letters (you'll have external 27 discs connected)! I'd suggest using RAID controllers so that you also have some redundancy in your configuration. Run the external discs as RAID 1/0 sets or RAID 5 stes depending on what you intend.

Hope this helps.
Duncan MeyersCommented:
BTW - if you go with a RAID controller, you'll need one taht supports clustering - that is, you need to be able to change the controller SCSI address. Not all RAID controllers will let you do this.
Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

kenshawAuthor Commented:
ok... thanks for that.  When you say if I go for a RAID controller I'm a bit confused.  Won't my SCSI controller support RAID?  i.e. in my servers I'll have two adapters, and each adapter will be a SCSI/RAID controller won't they?  

Also - why do I need to be able to change the controller SCSI address for clustering?  To make sure that each node's SCSI cards are set up properly?  I don't get this bit.
Duncan MeyersCommented:
>why do I need to be able to change the controller SCSI address for clustering?  To make sure that each node's SCSI cards are set up properly?  I don't get this bit.

Each node has to have different SCSI addresses on the controller, typically 7 for one node and six for the other. If you don't change the controller SCSI addresses you'll have an instant conflict and the cluster simply won't work. So you'd set the controller SCSI addresses on one node to 7 (same on each SCSI card) and on the other node 6 (again, same on each SCSI card).

>When you say if I go for a RAID controller I'm a bit confused.  Won't my SCSI controller support RAID?  i.e. in my servers I'll have two adapters, and each adapter will be a SCSI/RAID controller won't they?  

Not necessarily. Depends what you've got. For example, an Adaptec 39160 is a SCSI controller only - no RAID. If you want to set up RAID, you'd need to use software RADI within your OS (Dynamic discs in Disk Manager for Windows, Software RAID in Disk Druid in Red Hat Linux and so on). An Dell PERC 3/DC is both a SCSI and a hardware RAID controller. With this, you can group a number of discs into a RAID array - RAID 5, 1/0, 1 and so on.

Given what you've posted about you're storage requirements, I'd strongly reccommend you look at an entry level SAN such as EMC's CX300. It  gives you a greater degree of performance, flexibilityand expandability than direct attach storage. And it isn't *that* much more expensive. Apparently they start at around AUS$20K - which is not that much more tahn what you'll spend with the direct attach anyway (I'd estimate AUS$15K-16K for what you're describing). The extra money buys you an extra *heap* of reliability and robustness.

FWIW. Sorry to complicate it for you, but it's far better to make the right decision now, rather than try and make a crappy system work well.
HP's DL380 packaged clusters can be used with redundant RAID controllers if you use the high availability kit for the MSA500 and you don't lose 2 SCSI addresses as it's not a shared SCSI bus so you still get 14 or 28 drives and because it has an internal RAID controller you don't have to dedicate a disk for the quorum but simply define a 1GB slice of an array for it.

Have you thought about using SQL replication rather than clustering or using something like doubletake? For clustering you need Windows enterprise and SQL enterprise and that can cost more than the hardware.
IBM's unit is a shared SCSI bus, you have to use special Y cables to provide termination when one of the servers is powered off. That's why I like HP's offering, whether fibre or SCSI attached each SCSI is buffered onto a PCI bus in the RAID box, then through the internal RAID controller onto another SCSI bus for the drives. With dual attached servers there's SecurePath software so that the OS doesn't see both sets of drives and think you've got twice as much storage than you really have.

Our firm sold a packaged cluster in all good faith as a stand alone solution so we had to impliment a domainlet since there were no other servers to hold the Global Catalog etc. Got called out because it went wrong and when I arrived on site the software developer said he didn't want a cluster! Made me take the shared drives out and put them in the servers instead, remove AD (which was what broke) and said he was going to use sql replication instead. Now it's going to a co-lo hosting site.

Anyone want a 3 month old MSA 500, rack, UPS, KVM switch and rackmount keyboard? (joke, I know EE isn't eBay).
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.