HP Proliant ML115 - Raid5 Drive Failure - Raid Failure?
Got called out to a customer site. They have a HP Proliant ML115 Server. They are using the onboard Nvidia Controller and 3 x 250GB SATA hard drives in a RAID5 configuration.
One of the drives has reported SMART failure (Drive 0.1)
Although only one drive has reported failure as shown in the screen shot below:
When booting up I get the following message while detecting the arrays:
When I press F8, The screen now shows two arrays of one drive only in a error state as shown (it used to show only the one array with all three drives healthy):
I contacted HP and they sent a replacement drive. THey could not offer any intelligent advice as to what I can do just to "replace the drive and hope for the best"
Unfortunatley the customer also reports that they have not had a sucessful backup in over two weeks. Better than nothing I suppose but if possible I would like to retain the data. THe other two drives I can assume are ok and have the intact parity, I have never dealt with an onboard array controller like this though and am seeking some advice based on experience.
I have hte replacement drive, the question I suppose is, do I replace the drive that has the smart array warning, or the drive that doesnt show up in the array configuration utility at all.
Drive 1.0 doesnt show up when I press F8, but shows fine in the post drive listing.
Advice is appreciated.
One of the drives has reported SMART failure (Drive 0.1)
Although only one drive has reported failure as shown in the screen shot below:
When booting up I get the following message while detecting the arrays:
When I press F8, The screen now shows two arrays of one drive only in a error state as shown (it used to show only the one array with all three drives healthy):
I contacted HP and they sent a replacement drive. THey could not offer any intelligent advice as to what I can do just to "replace the drive and hope for the best"
Unfortunatley the customer also reports that they have not had a sucessful backup in over two weeks. Better than nothing I suppose but if possible I would like to retain the data. THe other two drives I can assume are ok and have the intact parity, I have never dealt with an onboard array controller like this though and am seeking some advice based on experience.
I have hte replacement drive, the question I suppose is, do I replace the drive that has the smart array warning, or the drive that doesnt show up in the array configuration utility at all.
Drive 1.0 doesnt show up when I press F8, but shows fine in the post drive listing.
Advice is appreciated.
rabindrajha
In a nutshell, my suggestion will be replace both drive. the smart error you are getting, that drive is going to be failure soon. take the data backup ASAP. if data is not important, you may take the R&D.
ASKER
it was a 3 drive raid 5 array. I cannot boot the server to do a data backup. Do you believe the third drive which is not showing in the array configuration is also faulty?
what is the status of HDD led? you may take a chance for recovering your data, by booting your server through any live CD, like ubuntu and so and so..which is supporting your raid controller driver. i beleive that ubuntu consist that driver and see if you are able to recover that data. remember RAID can't replace the backup.
ASKER
These are not hotswap drives.
All I can tell you is in post, it the server sees all three drives.
Once the array detection comes up though it only sees two drives. But, it has separated the two drives into their own little array somehow.
Booting to ubuntu didnt work. Since the array wont load, I can only assume that is why it cant see anything.
Fully aware that raid cant replace backup...they have backup. Just looking to avoid hours and hours of a complete rebuild.
All I can tell you is in post, it the server sees all three drives.
Once the array detection comes up though it only sees two drives. But, it has separated the two drives into their own little array somehow.
Booting to ubuntu didnt work. Since the array wont load, I can only assume that is why it cant see anything.
Fully aware that raid cant replace backup...they have backup. Just looking to avoid hours and hours of a complete rebuild.
now this leads many questions.
You may consider upgrading the firmware. as per scenario, not enough sure through it seems drive is failed which is not showing in the array.
You may consider upgrading the firmware. as per scenario, not enough sure through it seems drive is failed which is not showing in the array.
again i gone through your screenshot. however the drive is having SMART failure my be the drive which is not showing in raid array, you may do onething, disconnect the smart failure drive and see, if still there are two drives in array. not to worry about the raid configuration as you know this configuration is on two places, one is in controller and another on disk. you may give a try.
sorry for above comment. you may give a try by reconnecting the drive, whether it still shows two or one only. if not it may the problem of both drive.
ASKER
if i disconnect the smart failure drive, it does not show in the array config utility anymore
I supect that the third master may be my issue. it doesnt show in the array config utility anywhere.
The second master is the one showing smart failure, but at least it shows in the array config utility. If I remove the second master it disappears from the config utility, but then I only have one drive in the array.
Primary master is index 1
Secondary master is index 2
currenlty the two drives are somehow separated into two separate arrays.
Strange thing is the third master is no where to be seen in the config utility
I supect that the third master may be my issue. it doesnt show in the array config utility anywhere.
The second master is the one showing smart failure, but at least it shows in the array config utility. If I remove the second master it disappears from the config utility, but then I only have one drive in the array.
Primary master is index 1
Secondary master is index 2
currenlty the two drives are somehow separated into two separate arrays.
Strange thing is the third master is no where to be seen in the config utility
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thx Drives are not making any noises, thank you for trying anyways
I have to admit that I haven't seen what you are describing on any of my ML115 servers.
From what you have described, I think you are saying that originally there was on RAID 5 array of 3 drives, and the Nvidia raid controller i now only seeing two drive and two arrays.
If this is the case you have probably lost all of the data on the array.
I would try swapping out the drive that the array controller doesn't see and see if it will rebuild. I've also found this http://thegreenbutton.com/forums/t/54005.aspx I'm not quite sure how usefull it will be, but it could explain why you are seeing two arrays.
From what you have described, I think you are saying that originally there was on RAID 5 array of 3 drives, and the Nvidia raid controller i now only seeing two drive and two arrays.
If this is the case you have probably lost all of the data on the array.
I would try swapping out the drive that the array controller doesn't see and see if it will rebuild. I've also found this http://thegreenbutton.com/forums/t/54005.aspx I'm not quite sure how usefull it will be, but it could explain why you are seeing two arrays.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks dlethe. Great tip.
I ended up doing a complete rebuild of the server, and as it turns out the customer did indeed have complete backups so all is well. I have the old drive still so I plan to try the raid reconstructor.
- rabindrajha, thank you for your help and attempts. The effort was much appreciated.
I ended up doing a complete rebuild of the server, and as it turns out the customer did indeed have complete backups so all is well. I have the old drive still so I plan to try the raid reconstructor.
- rabindrajha, thank you for your help and attempts. The effort was much appreciated.