Attaching multiple external SCSI storage arrays to a two node cluster

Posted on 2004-09-23
Last Modified: 2013-11-15
Ok, so Im purchasing a two node win 2k3 cluster that will run SQL 2000 in an active/passive failover HA config.

I want to have an external SCSI storage array and have sourced the IBM EXP400 that will support 14 drives.

As I understand it - a single SCSI adapter can have 14 drives connected to it.

So in my external storage - one of the disks will be a quorum disk leaving me with 13 active drives for data.

What i want to know is whether i can have two SCSI adapters in each of my servers, and connect the two node cluster to two external SCSI storage enclosures.  That way I'd have 13 active drives in the first array and 14 active drives in the second array.  Is there any restriction on doing this?  Can I have two SCSI adapters running in my servers?  Would there be any restrictions on this imposed by Win 2k3 or SQL Server 2000 (i.e. will they only recognize a limited total number of SCSI disks?)

Basically - I don't want to be limited by the 2TB of storage I can get out of one external storage array.  I want to get about 4 TB but i don't to jump up to SAN.  I also want to stick with the major hardware vendors - as a very high level of SLA is required - if there's hardware failure I want the part replaced by the vendor within 10 hours or so.  Thats why I'm looking at IBM, Dell, HP.
Question by:kenshaw
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
LVL 30

Expert Comment

by:Duncan Meyers
ID: 12140672
Yes, you can. I've been involved with a 2 node cluster with each node connected to 2 Dell Powervault 220S SCSI arrays via LSI/AMI RAID controllers (aka Dell PERC 3/DC).

The only restriction is a slightly ludicrous one - you run out of drive letters (you'll have external 27 discs connected)! I'd suggest using RAID controllers so that you also have some redundancy in your configuration. Run the external discs as RAID 1/0 sets or RAID 5 stes depending on what you intend.

Hope this helps.
LVL 30

Expert Comment

by:Duncan Meyers
ID: 12140684
BTW - if you go with a RAID controller, you'll need one taht supports clustering - that is, you need to be able to change the controller SCSI address. Not all RAID controllers will let you do this.

Author Comment

ID: 12140721
ok... thanks for that.  When you say if I go for a RAID controller I'm a bit confused.  Won't my SCSI controller support RAID?  i.e. in my servers I'll have two adapters, and each adapter will be a SCSI/RAID controller won't they?  

Also - why do I need to be able to change the controller SCSI address for clustering?  To make sure that each node's SCSI cards are set up properly?  I don't get this bit.
Three Reasons Why Backup is Strategic

Backup is strategic to your business because your data is strategic to your business. Without backup, your business will fail. This white paper explains why it is vital for you to design and immediately execute a backup strategy to protect 100 percent of your data.

LVL 30

Expert Comment

by:Duncan Meyers
ID: 12140780
>why do I need to be able to change the controller SCSI address for clustering?  To make sure that each node's SCSI cards are set up properly?  I don't get this bit.

Each node has to have different SCSI addresses on the controller, typically 7 for one node and six for the other. If you don't change the controller SCSI addresses you'll have an instant conflict and the cluster simply won't work. So you'd set the controller SCSI addresses on one node to 7 (same on each SCSI card) and on the other node 6 (again, same on each SCSI card).

>When you say if I go for a RAID controller I'm a bit confused.  Won't my SCSI controller support RAID?  i.e. in my servers I'll have two adapters, and each adapter will be a SCSI/RAID controller won't they?  

Not necessarily. Depends what you've got. For example, an Adaptec 39160 is a SCSI controller only - no RAID. If you want to set up RAID, you'd need to use software RADI within your OS (Dynamic discs in Disk Manager for Windows, Software RAID in Disk Druid in Red Hat Linux and so on). An Dell PERC 3/DC is both a SCSI and a hardware RAID controller. With this, you can group a number of discs into a RAID array - RAID 5, 1/0, 1 and so on.

Given what you've posted about you're storage requirements, I'd strongly reccommend you look at an entry level SAN such as EMC's CX300. It  gives you a greater degree of performance, flexibilityand expandability than direct attach storage. And it isn't *that* much more expensive. Apparently they start at around AUS$20K - which is not that much more tahn what you'll spend with the direct attach anyway (I'd estimate AUS$15K-16K for what you're describing). The extra money buys you an extra *heap* of reliability and robustness.

FWIW. Sorry to complicate it for you, but it's far better to make the right decision now, rather than try and make a crappy system work well.
LVL 55

Expert Comment

ID: 12142721
HP's DL380 packaged clusters can be used with redundant RAID controllers if you use the high availability kit for the MSA500 and you don't lose 2 SCSI addresses as it's not a shared SCSI bus so you still get 14 or 28 drives and because it has an internal RAID controller you don't have to dedicate a disk for the quorum but simply define a 1GB slice of an array for it.

Have you thought about using SQL replication rather than clustering or using something like doubletake? For clustering you need Windows enterprise and SQL enterprise and that can cost more than the hardware.
LVL 95

Accepted Solution

Lee W, MVP earned 250 total points
ID: 12145263
I used to manage two Windows clustered file servers using both a SAN and direct attach SCSI.  This is from my experience:

1.  STAY AWAY FROM DELL.  If you want a server, notebook, desktops, Dell is fine, I even recommend them.  I've had a 50% failure rate on Dell backup devices in the first year and roughly a 50% failure rate on Dell SCSI enclosures (200S, 210S, 220S).  Doesn't matter the OS, this was on Linux and Windows.  Both at a client, and at my day job.  Both clustered and non-clustered.  I had probably 8-10 of these devices and the number of times multiple disks failed AND the number of backplanes replaced on the SCSI enclosures just scared me and created nightmares.  That combined with the fact that Dell doesn't use NEW parts for mission critical parts replacement, they use USED/REFURBISHED parts (at least in every instance where I saw/asked), using Dell Storage devices is a recipe for disaster.  The one exception to this was the Dell PowerVault 650F and expansion 630F's.  These devices haven't been sold by Dell in 3-5 years, but ours (1x 650F+4x 630F Expansion units) were both the most reliable storage and hard drives I've ever had the pleasure of using.  These were Fibre SAN devices, basically rebranded clariion units - which is now EMC.

2.  I can't speak for other manufacturers.  Before I left that job (a little over two months ago), we were switching to IBM and Aberdeen equipment ( - a small manufacturer that had some NICE configurations - 6TB of storage on one 5U box using 3Ware RAID and SATA drives - For $10-15K-US$.)

3.  You might be able to create a SCSI cluster from regular SCSI cards... BUT, especially with clusters - you want to be able to get MS support.  Which means sticking to the Hardware Compatibility List (HCL).  I actually did setup a cluster using a couple of SCSI drives and a couple of Adaptec 2910 SCSI controllers.  That worked.  I tried again more recently with a couple of 29160 controllers, that worked briefly then went offline and I haven't had it running since (6 months).  These were my own test clusters since I'm not affording your typically EMC device or RAID Controllers.

4.  Stay away from Dell PERC cards.  They MIGHT not be bad, but in addition to replacing the Dell storage unit components, they replaced these cards a number of times.  I simply don't trust them.  And half the time, they didn't perform correctly - taking a system down when a single drive failed (it should running with a single drive failure).

5.  I have to agree with Andy Adler - unless you have an open budget, I would strongly consider SQL replication technologies rather than clustering.  Obviously, this depends somewhat on what you are doing with the database, but the licenses for SQL server and Windows Enterprise server are anything but cheap.

6. Quorum - makes little sense to put it on a disk by itself - I understand that's what Microsoft recommends, but Quorum takes up FAR LESS than 1 GB of storage.  If you're sticking 14x146 GB drives in a system, you are loosing 145 GB+.  Further you aren't making your quorum redundant - if you did, then your loosing 290GB+ for a mirrored quorum drive.  You're also loosing another 146 GB for a HOT SPARE - which you don't NEED, but REALLY SHOULD HAVE.  Plus as point 8 below illustrates, you can generally only access 13 drives in a SCSI Cluster, so you loose yet another 146GB.  That's over half a terabyte of wasted space.  Quorum can reside on the same RAID container as your data.  Just use Windows to cut a 1GB partition for it and leave the rest to your data.

7.  SCSI RAID Controllers/SCSI Controllers.  I've had servers with 4 SCSI controllers and 2 RAID controllers.  There's generally no problems with this.  The only problems are in running out of IRQ/IO addresses and with server technology and IRQ sharing, this isn't really a problem anymore.

8.  SCSI cards have a SCSI ID for themselves.  As do RAID cards.  You can have 16 devices (SCSI card Included) on a SCSI or SCSI RAID card.  Each SCSI RAID Card needs 1 SCSI ID.  The SCSI backplane/Enclosure typically (at least with Dell units) requires one SCSI ID.  That means 3 IDs are used.  That leaves 13 IDs left.  So a 14 Drive enclosure is only going to make 13 drives usable at all.  That 14th drive can be a cold spare, but NOT a HOT spare - to be a hot spare, it would need it's own SCSI ID - which there aren't enough of.

In short, SPEND THE MONEY on an EMC Fibre SAN (NOT NAS) storage unit and you'll be setting things up with growth potential and that will LIKELY (but nothing is ever guarenteed) keep your reliability headaches to a minimum.
LVL 55

Expert Comment

ID: 12145928
IBM's unit is a shared SCSI bus, you have to use special Y cables to provide termination when one of the servers is powered off. That's why I like HP's offering, whether fibre or SCSI attached each SCSI is buffered onto a PCI bus in the RAID box, then through the internal RAID controller onto another SCSI bus for the drives. With dual attached servers there's SecurePath software so that the OS doesn't see both sets of drives and think you've got twice as much storage than you really have.

Our firm sold a packaged cluster in all good faith as a stand alone solution so we had to impliment a domainlet since there were no other servers to hold the Global Catalog etc. Got called out because it went wrong and when I arrived on site the software developer said he didn't want a cluster! Made me take the shared drives out and put them in the servers instead, remove AD (which was what broke) and said he was going to use sql replication instead. Now it's going to a co-lo hosting site.

Anyone want a 3 month old MSA 500, rack, UPS, KVM switch and rackmount keyboard? (joke, I know EE isn't eBay).

Featured Post

Space-Age Communications Transitions to DevOps

ViaSat, a global provider of satellite and wireless communications, securely connects businesses, governments, and organizations to the Internet. Learn how ViaSat’s Network Solutions Engineer, drove the transition from a traditional network support to a DevOps-centric model.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Selecting Right Partition 6 82
Economy of Scale - IT Business 4 96
Dell PowerEdge R620 with Perc H310 Raid Controller 6 62
Runtime Error -2147467259 (80004005) 7 50
VM backup deduplication is a method of reducing the amount of storage space needed to save VM backups. In most organizations, VMs contain many duplicate copies of data, such as VMs deployed from the same template, VMs with the same OS, or VMs that h…
The Delta outage: 650 cancelled flights, more than 1200 delayed flights, thousands of frustrated customers, tens of millions of dollars in damages – plus untold reputational damage to one of the world’s most trusted airlines. All due to a catastroph…
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This tutorial will walk an individual through setting the global and backup job media overwrite and protection periods in Backup Exec 2012. Log onto the Backup Exec Central Administration Server. Examine the services. If all or most of them are stop…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question