Perc 5/E crashed during raid 5 expansion
We have a Dell server with a Perc 5/e controller attached to a MD1000 storage unit. There was one raid 5 virtual disk array with five 300GB drives in a raid 5 configuration that is not split up. We had a hot spare that was exactly the same as the others so I decided to un-assign it as the hot spare. Then I followed Dell's instructions to add this drive to the raid 5 array to expand it. I used the Dell server administration software to do all this.
The reconstruction phase seemed to be going smoothly and got to about 66% and the server locked up and crashed. With the MD1000 unit attached with the SAS, the server would not boot (the OS boot drives are on the internal raid controller of the server and not on the MD1000). We disconnected the SAS cable from the MD1000 just to make sure the server OS would still boot and it does fine.
We connected the SAS back to the MD1000 and using the built-in raid bios management it shows that this new drive is in a foreign state and it fails when we try to import it. So I decided to pull the new drive we added to the VD just to see if the array and server would come up and of course in our haste we pulled the wrong drive. We didn't swap any drives around as I know that is a no-no with raid 5. So we put the accidentally pulled drive back in and the raid 5 started rebuilding and still in the process now.
Since it never finished the expansion and the new drive is stuck in a foreign state what happens to the array? Is it still expecting to see the new drive or will we be able to clear the configuration of the new drive and bring up the raid 5 array?
The reconstruction phase seemed to be going smoothly and got to about 66% and the server locked up and crashed. With the MD1000 unit attached with the SAS, the server would not boot (the OS boot drives are on the internal raid controller of the server and not on the MD1000). We disconnected the SAS cable from the MD1000 just to make sure the server OS would still boot and it does fine.
We connected the SAS back to the MD1000 and using the built-in raid bios management it shows that this new drive is in a foreign state and it fails when we try to import it. So I decided to pull the new drive we added to the VD just to see if the array and server would come up and of course in our haste we pulled the wrong drive. We didn't swap any drives around as I know that is a no-no with raid 5. So we put the accidentally pulled drive back in and the raid 5 started rebuilding and still in the process now.
Since it never finished the expansion and the new drive is stuck in a foreign state what happens to the array? Is it still expecting to see the new drive or will we be able to clear the configuration of the new drive and bring up the raid 5 array?
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Data recovery guys couldn't rebuild the raid as it was in a state of flux. It had some data on spread onto the new drive but the file tables were half corrupted.
Before I expanded to the spare drive, I did remove it from the hot drive status and the system saw the drive as being unallocated.
There's no chance of rebuilding the old array so I'm closing this.
Before I expanded to the spare drive, I did remove it from the hot drive status and the system saw the drive as being unallocated.
There's no chance of rebuilding the old array so I'm closing this.
ASKER
I did the same thing expanding the array with two drives two years ago without any issues so that's why I assumed it wouldn't be an issue this time. Just never took into consideration that the server would crash during the process.
The system basically sees the raid 5 array as being perfect (without the new drive) and is even mounted with the right drive letter but the file tables seems destroyed.