Solved

RAID configuration after drive failure

Posted on 2011-09-12
15
1,798 Views
Last Modified: 2016-12-08
I recently ran the HP ADU as part of regular maintenance on our main server. The report came back that I had 2 drives failing on a RAID 5. I came in  the next morning and one of the drives had indeed failed overnight. The online spare had kicked in and had done it's job. I purchased 3 new drives and hot swapped one a day later . I ran the ADU again and was informed that the drive I had swapped was good and was being rewritten and that the online spare was active. The second "bad" drive was still functioning but was failing. I swapped that drive out as well. I am getting a failure message and the red led on the second drive I replaced. When I run HP Insight Diagnostics I am getting a report that this (new) drive is still bad. it is as if it is reporting on the drive I had just replaced and hasn't recognized the new drive. I am relatively inexperienced with RAID configurations and need an expert opinion. ADUReport.txt
0
Comment
Question by:JPHopewell
  • 8
  • 3
  • 2
  • +2
15 Comments
 
LVL 47

Expert Comment

by:dlethe
ID: 36522292
Are you using the proper HP-branded disk drives? If not, please elaborate.
0
 

Author Comment

by:JPHopewell
ID: 36522298
Yes, I am using the exact recommended HP replacement drives.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 36522320
No, let me be specific, ARE these the HP-branded disk drives, or are these the same drives, but w/o the HP firmware?
0
 

Author Comment

by:JPHopewell
ID: 36522321
This is a HP ML350 G5 (SMART array E200i  with DG146BB976 HP 146-GB 10K 2.5" DP SAS HDD)
0
 

Author Comment

by:JPHopewell
ID: 36522335
These are HP branded drives. I am unsure as to the firmware version on the new drives. The first drive I replaced seem to have worked fine and was incorporated in to the RAID so I will *cringe* assume that the firmware is correct.
0
 
LVL 17

Expert Comment

by:OriNetworks
ID: 36522344
Also just because it says HP doesnt mean its legit HP. Hopefully you purchased the replacement directly from HP or an authorized/reputable reseller. Also it might be good to verify that the firmware version is the same across all drives.
0
 

Author Comment

by:JPHopewell
ID: 36522363
I purchased them from http://www.harddrivesdirect.com/contact_us.php?PHPSESSID=ejh6lu1hbjsdv6d2imvo9qht83 A cursory check did not return any known problems with the reseller.
May I ask how to check firmware versions on the drives?
0
PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

 
LVL 55

Accepted Solution

by:
andyalder earned 500 total points
ID: 36522390
Last Failure Reason                  Hot Removed (0x14)

Looks like you took the wrong drive out (disk 1)

Firmware is OK although not all are up to date.

It also says that the array (sas array A) has failed, is that the case? Is the system down?
0
 

Author Comment

by:JPHopewell
ID: 36522418
No the system is still up and functioning. It appears that I may have indeed pulled the wrong drive although
I replaced the drive that was showing the red LED.
0
 

Author Comment

by:JPHopewell
ID: 36522469
Currently It appears visually as if I have 4 good drives (slots 2-5) and a bad one (red LED) in slot 1 The diagnostic isn't finding the drive in the first slot and telling me that the drive in the second slot is going bad.
The drive in the 5th slot was my online spare.
0
 
LVL 1

Expert Comment

by:cmlbaete
ID: 36522498
Just to concur with Andyalder my thoughts are the wrong drive was replaced.
0
 
LVL 55

Assisted Solution

by:andyalder
andyalder earned 500 total points
ID: 36522572
Hmm, bit misleading in the ADU report then although it may be old.

If you search for "Physical Drive Error Log Entries" you'll see lists against each disk with entries such as this:
 
0x02       0x5a                0x00        0x22       0x00      0x00      0x00       0x00        0x3f000000 0x00000001     0x0000

A couple of them have rather long error lists but none of them have predictive failures or read / write errors so I'd suggest that except for missing drive 4 all is good.
0
 

Author Comment

by:JPHopewell
ID: 36522707
So what should be my course of action? I have a drive in the first slot that is showing amber and is not being found, The drive next to it is going bad apparently with a 640004 error. The HDD in the first slot (showing amber) is one of the new drives I just purchased. Should I replace the new HDD in the first slot with the one I removed initially? Should I then replace the one going bad (slot2) with the new drive I pulled from the first slot?

See attached ADU report run just a few minutes ago. ADUReport.txt
0
 

Author Comment

by:JPHopewell
ID: 36522729
Here is the Diagnosis log diagnosislog.html
0
 
LVL 55

Expert Comment

by:andyalder
ID: 36523344
I'd replace disk 1 since it's not found.
Disk 2 doesn't look that bad, only thing I can see is a few read errors that were corrected with retry.
0

Featured Post

Best Practices: Disaster Recovery Testing

Besides backup, any IT division should have a disaster recovery plan. You will find a few tips below relating to the development of such a plan and to what issues one should pay special attention in the course of backup planning.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

I previously wrote an article addressing the use of UBCD4WIN and SARDU. All are great, but I have always been an advocate of SARDU. Recently it was suggested that I go back and take a look at Easy2Boot in comparison.
Create your own, high-performance VM backup appliance by installing NAKIVO Backup & Replication directly onto a Synology NAS!
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

863 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now