Solved

Raid 5 crash

Posted on 2011-03-23
9
407 Views
Last Modified: 2016-12-08
I was upgrading my servers hard drives over the past few days. The setup is raid 5 with 5 drives. I switched one drive at a time and let the configuration rebuild. The last I saw, the 5th drive was 92% complete rebuilding. When I came back the server was bsod and a restart will not boot. I put all of my old drives in and still won't boot. I thought it might be the controller so I tried both sets of drives in an identical server, but neither boot.
 
Out last backup was from over 12 hours ago, which means a lot of lost data. Any ideas on why the servers won't recognize the raid configurations? Any way to get that data back without hiring specialists?

Thanks in advance.
0
Comment
Question by:dwils15
  • 2
  • 2
  • 2
  • +3
9 Comments
 
LVL 18

Expert Comment

by:Netflo
ID: 35203020
Do you have a warranty for your server from Dell or HP if so they may be able to help you save the array.
0
 
LVL 1

Author Comment

by:dwils15
ID: 35203086
No these servers are at least 6 years old. poweredge 2650's
0
 
LVL 18

Accepted Solution

by:
Netflo earned 100 total points
ID: 35203483
Firstly, data loss is a possibility based on what you've described. Sounds like the array did not fully rebuild and / or in the unlikely event have suffered from a hard drive failure from one of the disks in the array.

As its a Dell server and I'm assuming you have a PERC controller, can you check in the RAID BIOS if you can import a foreign array? I've seen this happen before where the array can just blink off and the causes are due to an old firmware which should have been updated prior to doing this sort of work.

Can you post details of your RAID card and version, if my point doesn't answer your question?

Thanks
0
 
LVL 47

Assisted Solution

by:dlethe
dlethe earned 100 total points
ID: 35205458
What you had a drive failure while the system is in degraded mode.  Just popping back the old disks won't work because, well, the firmware just doesn't let you do that, nor should it let you do that, because the data is stale.   It can't possibly work. It WOULD have worked if the system wasn't booted.

Without hexdumps of the first few MB of all of the disk drives, and the ability to run some diagnostics on the hardware, then I couldn't begin to tell you what the next step is for an assisted recovery.  At this point you are going to have to decide if it is worth a few thousand dollars to you to contract a pro.

I know it is too late for this, but if you look at some of my old postings, then you'll see at least a dozen of them saying NEVER to do what you attempted because it is high risk.  Picking up a bad block when in degraded mode means 100% data loss for many controllers.  Doing this hot to a booted system that is also reading / writing data adds even more risk because you can't manually extrapolate what you need from the older disk drives.   Also the very action of causing the rebuild puts a great deal of stress in disks, and makes it even a higher risk of failure.

Your ONLY hope is professional data recovery.  Had the system never have been booted then recovery would have been much easier.  Sorry.


0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 2

Assisted Solution

by:chihs
chihs earned 100 total points
ID: 35205528
by the previous experience on storage,
1. software raid need ask the original manufacture and try to recovery by software engineer
2. hardware raid will difficult to recovery because it's handle by the hardware and the firmware, this  need to check the original manufacture for help.
0
 
LVL 55

Assisted Solution

by:andyalder
andyalder earned 100 total points
ID: 35205825
I would recommend using Raid reconstructor (www.runtime.org/raid.htm) with the first 4 drives, the fifth one isn't any use as only half rebuilt. That should enable you to get most of the data back although the BSOD may have caused something to get over-written.
0
 
LVL 32

Assisted Solution

by:PowerEdgeTech
PowerEdgeTech earned 100 total points
ID: 35208808
As it is likely too late now ... just remember this for future reference:

Support is ALWAYS free on Dell servers (in the States and Canada) - regardless of warranty status.  Your first move in a situation like this is to do NOTHING before calling Dell.  Call as soon as it's down, unless you know what you need to do to fix it.  
0
 
LVL 1

Author Comment

by:dwils15
ID: 35212280
Thanks for all the replies. Since we have thousands of people connecting to this server we needed the system running ASAP. We just Installed SQL on another server and restored from our backups. We lost a day of work, but it was our best option.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 35213509
Lost a days work :(

Best change SQL to full recovery model and make sure logs are on a different volume for next time you have a problem so you can do a no loss restore.
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

We all have limited time to study long and complicated information about RAID theories, but you may be interested as to how RAID 5 works. We made it simple for you by providing the shortest and easiest explanation ever.   First we need to remind …
Lets start to have a small explanation what is VAAI(vStorage API for Array Integration ) and what are the benefits using it. VAAI is an API framework in VMware that enable some Storage tasks. It first presented in ESXi 4.1, but only after 5.x sup…
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now