Hello experts,
My company installed CX500 + 2xPE6850 + 2xBrocade + 4CPU Powerpath in 2006.
That time only for a consolidadtion of 5 MS SQL Servers, total of 3 trays with a couple of LUNs, all of them in RAID-5.
Last and during present years, a number of other things are stored there.
Follows a sample list:
- 380 GB SCSI R-5 with Veritas File System under Solaris 8
- 400 GB SCSI R-5 for Oracle RAC under Solaris 11
- 400 GB SCSI R-5 with NTFS for Notes under Windows 2003
- 400 GB SCSI R-5 with NTFS for SQL, used by Veritas Replicator, under Windows 2000
- 200 GB ATA no RAID for Red Hat Linux
- 500 GB ATA R-5 for VMWare
- etc, a very heterogeneous environment
There are servers, accesing such SAN, from single Xeon to 4x Dual Core Xeon to 4x RISC Sun and 2x PA-RISC.
People around is criticizing such macarronade as the worst possible combination to cause bad performance and erroneous behavoir. Of course I'd like to have all my systems running the same O.S. and the same DBMS, one system per tray... but world isn't as we want...
The problem, and I hope the experts can have a hint or a guess, or even an experience by his/her own, is that erroneous functioning seems to ponit in favor of the critic people.
In the last 30 or 40 days, every week yellow (orange?) lights turn on, one disk die, the spare assumes, Dell people change the malfunctioning disk and the cycle repeats again and again.
What are probable causes and how to repair the system?
Jose
This Question has been solved and asker verified All Experts Exchange premium technology solutions are available to subscription members.
Experts Exchange has been collecting answers to technology questions since 1996…3 million and counting! If you have a question, chances are we already have your answer.
If you can't find the exact answer you're looking for, ask our exclusive community of 50,000 experts. You’ll get a personalized answer from a trusted professional.
Thousands of free tech tips, tricks, how-to’s and tutorials are available in our peer reviewed articles section. See for yourself how smart our experts are, no login required.
Save solutions to your questions, answers you’ve discovered through searching plus helpful articles in your personal knowledgebase for easy future access.
Access the answers to your technology questions today.
30-day free trial. Register in 60 seconds.
Members of the expert community talk about why the experience at Experts Exchange is different than what you will find anywhere else.
Try it out and discover for yourself.
30-day free trial. Register in 60 seconds.
Join the community of experts here and help other tech pros by answering question in your area of expertise. You can earn FREE access to all Experts Exchange's premium features and resources.
2 Years old isn't brand new, in IT that is almost ancient. Disks can always go bad, it is one the most sensitive pieces of hardware in a Computer, as it has moving parts. I'm not saying that it is normal, but it happens. Very often failures are caused by bad cabling, termination, or bad power supplies, though.
The failing disks are SCSI 146GB FC 15.000 rpm.
In the first incident, Dell support simply exchange the failing disk at Bus0 Enclosure 3, in hot swap.
In the second time, they did the same in Bus0 Enclosure 2 and open a ticket at their EMC support, because they don't know the root cause.
In the 3rd occurrence, they scaled to EMC, which will do the same (tomorrow), that time asking for a total shutdown.
I don't know about any software fixes, but they spent a time in checking configurations. For my surprise, we discovered that the "firmware" runs on a modified Microsoft Windows...
I have noticed that the two enclosures which have presented problems have a raid with disks from both enclosures, as in the figure below: RG18, from Enclosure 2, has a disk at Enclosure 3. Is the problem to occur right in these enclosures a mere coincidence?
Jose
Partial configuration
rindi,
Dell people didn't reported anything wrong with the components, except the failing disks.
In the past we had 4 Sun StorEdge T3. All had failures and malfunctioning disks. I think Sun customers around were very happy, because I was responsible for the bad part of the average MTBF.
Despite the moving parts, Sun claims better MTBF to disks than to HBA, as per the below table:
Component Sample MTBF (hours)
HBA 1 800,000
Link A 400,000
Concentrator 1 580,000
Disk A 1,000,000
as in the paper at http://www.sun.com/bluepri
Probably due to the connectors and cabling...
I don't know the same data from Dell.
Jose
Jose,
But the MTBF of the whole system is much lower than the MTBF of the individual parts, eg for disks IIRC you divide the number of disks by the MTBF of one disk to get the actual number for the whole set of disks, which means for those people with a large number of disks - 500 or so they can expect at least one disk to fail every week (even more if they are SATA that are run above their rated duty-cycles (normally about 40%))
G
connollyq,
Thanks for your comment.
You're right. My understanding from the Sun paper, was those MTBFs to be for components. Another article from HP shows the same 1,000,000 hs MTBF for SCSI disks.
What I was suspecting is that Dell CX500 disks weren't the cause of the failures. My guess in Thrusday was that Dell (and then EMC) tech people made wrong diagnostics. Yesterday they confirmed my guess... after to substitute the "defective" disks, the problem appeared again, that way in other disks, and they decided to substitute one of the trays. They are working on right now.
I would appreciate if you can help me in clarifying the central question: is such "salade" (mixing of RAIDs in the same enclosure, RAIDs with disks in two different enclosures, extreme heterogeneous environment: Linux, Windows Server 2000, 2003, Sun Solaris 10, VMWare, all accessing the storage) a enough reason to cause instability or malfunctioning of our CX500?
Once again, thank you all for the attention.
Jose
Jose,
There is nothing really wrong with the "salade" that you have, its just that it may not be the optimum layout for performance and/or availability!
In general ts normal practice to spread a RAIDset across the enclosures, for performance (depending on the number of back-end channels on the RAID controller) and also for availability (if you have three enclosures and a three disk RAID-5 set with one disk in each enclosure, you could survive a enclosure failure [not a very common occurence though])
But if it is working now and you are happy with the performance i would advise caution in doing any wholesale changes.
G
Business Accounts
Answer for Membership
by: connollygPosted on 2008-05-20 at 02:32:22ID: 21604541
Jose,
What do the disks fail with?
Are the disks that fail the SATA ones?
What do the Dell engineers say about the failures?