Raid 5 array, 2 drive failure imminent!

Posted on 2011-10-23
Last Modified: 2013-03-18
I have a 6 disk Raid 5 array in an HP Proliant server.  A single drive failed last week.  I hot swapped my last new replacement for it and the rebuild began.  Errors were found on another physical drive and the rebuild stopped, the array is now in a state of 'pending recovery'.  So now I have one drive unwritten to and another about to fail...  the other drive ( with errors) is now in a 'pending failure' state.  My new replacement drives will arrive tomorrow.

Is there any way to take the array offline and force a rebuild to the now failing 'error' disk?
My data is safe but the downtime for system rebuild will be inconvenient, to say the least.
Question by:kmorrison65
    LVL 46

    Expert Comment

    Not without high degree of risk for making things worse.  Your system is in stress. Best thing you can do is leave it alone.   Rebuilding a disk drive is as stressful as it gets (other than a power cycle), and your priority is to get the RAID optimal again.   The best thing you can do is eliminate any unnecessary I/O and wait it out.  You have a known bad against a drive that may even be a false positive.

    If it was me, however, and you had some vital files that need to be backed up that are stale, then I would risk creating a backup for the files that would cost me the most to recover.

    Think of it this way, if the drive does fail, then it might cost you $5000 to get everything recovered via a professional recovery.  If the data that has not been backed up recently is worth more than $5K to you, then kick off a backup now.   Otherwise cross  your fingers and disable any automated tasks that might generate significant I/O until tomorrow (like a defrag).


    Author Comment

    The data was backed up 2 days ago.  At this point, I'm wondering what my options are to rescue the array, if any.  Out of 6 disks, it's running on 5 with 1 known bad.  What would happen if I brought the system down, swapped the failing drive  with a new one and tried to reboot?
    LVL 46

    Expert Comment

    You would lose all of your data.
    LVL 55

    Expert Comment

    You do not "bring the system down" to replace disks on a Smart Array controller, they are designed to have bad disks replaced live.

    With read errors on one drive and another one failed it won't be able to rebuild. Maybe they aren't unrecoverable read errors but that's what normally causes rebuild failures.
    LVL 32

    Accepted Solution

    > if I brought the system down, swapped the failing drive  with a new one and tried to reboot

    Then you would have a RAID 5 array with two disks lost.  That = a failed array.  It's a trip to the data recovery folks at that point.

    With the re-build in process, you may have to leave it alone.  Even running at 100%, it could take 24-36 hours if you are using large (>1TB) drives in RAID 5.

    Check your re-build priority.  I'd put it to the maximum, so the rebuild will not get delayed due to IO from users or services.  (As long as that doesn't force a re-boot.)  If possible, remove all access, so the controller is doing nothing but servicing the drives, not user requests.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    6 Surprising Benefits of Threat Intelligence

    All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

    VM backups can be lost due to a number of reasons: accidental backup deletion, backup file corruption, disk failure, lost or stolen hardware, malicious attack, or due to some other undesired and unpredicted event. Thus, having more than one copy of …
    Microservice architecture adoption brings many advantages, but can add intricacy. Selecting the right orchestration tool is most important for business specific needs.
    This tutorial will walk an individual through the steps necessary to install and configure the Windows Server Backup Utility. Directly connect an external storage device such as a USB drive, or CD\DVD burner: If the device is a USB drive, ensure i…
    The viewer will learn how to download, install and use CrashPlan from on Windows 7. Open your browser and go to Click get started, it’s free then Click the Download CrashPlan button: Click save in Internet Explorer, the…

    761 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    7 Experts available now in Live!

    Get 1:1 Help Now