Solved

Oh, that was stupid.  RAID 5 and 4 HD's that weren't watched when removed from system!!

Posted on 2006-10-29
16
300 Views
Last Modified: 2010-04-03
Okay as the title says we screwed up....yeah we said it.  THANKFULLY, this isn't a mission critical system.  It does have about 700 Gb's of data that we'd rather not lose though.

Okay the back story.

System BSOD'd and when reset decided that it just wasn’t going to work anymore.  We just couldn’t get the system to POST.  We have a bit of on-board diagnostic LED’s and we could see that it was doing most everything correctly.  We suspected the Video card and had an advanced replacement sent out to us and put the card in; and got the SAME issue.  Uh oh, we then removed all the add-in cards, leaving only power, motherboard, CPU, RAM, and the new Video card; SAME issue.  Hrrrmm…Okay, we then try all sorts of combinations with the following and nothing we did got this quite unhappy system back on its feet:

2 Motherboards
2 Video cards
2 CPU’s
2 Power supplies

It wasn’t until we cut the RAM from 4 sticks to just 2 that we understood that our RAM had failed.  We now have the system back up and running with just half the RAM while it’s being replace, but now we have a problem.

Our 4 storage drives that we’ve hooked up in RAID 5 we’re left in the case, and the board was removed while doing all this.  We went to re-hookup all the drives and froze do to the fact that we hadn’t marked which drive went to what RAID port.  Our question is:  does it even matter or do they have to be hooked back up exactly?  The last thing we want to do is hook the drives up, have the RAID get confused, and scrub the drives “clean”.

Any insight would be so very helpful.

Thanks,
Derek

As always; if you'd like more information just ask.
0
Comment
Question by:Evil_RSA
16 Comments
 
LVL 34

Assisted Solution

by:jamietoner
jamietoner earned 50 total points
Comment Utility
That would depend on if the raid controller supports drive roaming, most scsi raid controllers do. What raid controller are you using (make and model)?
0
 

Author Comment

by:Evil_RSA
Comment Utility
On Silcon Image’s site they have the User Guide, and on page 7 they say the following under SATARAID5 Features

Data Security
      ... Drives can be moved between controllers without losing data.

Is this their way of phrasing Drive Roaming by any chance?

Derek
0
 

Author Comment

by:Evil_RSA
Comment Utility
On Silcon Image’s site they have the User Guide, and on page 7 they say the following line under SATARAID5 Features

Data Security
      Drives can be moved between controllers without losing data.

Could this be Silicon Image's way of phrasing Drive Roaming?

Derek
0
 

Author Comment

by:Evil_RSA
Comment Utility
Oops sorry for the double post, I didn't think it took the first post as I didn't see it...but then again I was looking for it to appear below my last post with the links to Silicon Image and DFI, not above it.

Derek
0
 

Author Comment

by:Evil_RSA
Comment Utility
Oh duh.  Daylight savings hit between those two posts....so yeah it would think I posted the other one earlier.

Anyway, I'm off to get some sleep.  Been a long night.

Derek.
0
 

Author Comment

by:Evil_RSA
Comment Utility
on board SATA

Controlling chip is a Silicon Image Chip. SiI3114
http://www.siliconimage.com/products/product.aspx?id=28

and the Board is by DFI
http://us.dfi.com.tw/Product/xx_product_spec_details_r_us.jsp?PRODUCT_ID=3872&CATEGORY_TYPE=LP&SITE=NA

Thanks for the fast response,
Derek
0
 
LVL 30

Assisted Solution

by:pgm554
pgm554 earned 50 total points
Comment Utility
Not knowing which drives are plugged into what ports is a bad thing.

This is an issue with ANY RAID 5 hardware.

Gennerally speaking when the RAID BIOS looks at the drives and doesn't see them in the correct order.it will give an error stating so.

You can try moving the cables and rebooting until you hit the right combo or there is a product called Raid Reconstructor from www.runtime.org that might help.
0
Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

 
LVL 70

Accepted Solution

by:
garycase earned 400 total points
Comment Utility
"... Drives can be moved between controllers without losing data. ... Could this be Silicon Image's way of phrasing Drive Roaming?" ==> No.  That's not the same thing.

I'd do two things:

(1)  Hook up the drives and see if the array is seen.  Do NOT do ANYTHING regarding initialization of the array.   If the controller supports roaming, it will simply "see" the array with no problem;  if not, you'll get a status message about a bad array, failed array, etc.  (not sure just what your specific controller will do).

(2)  If #1 didn't result in a good array, then buy a copy of Raid Reconstructor [http://www.runtime.org/raid.htm] and it will recover your data from the array (there's no "might" about it -- it WILL recover your data).   Note that you will need enough available storage to save all of the recovered data to (Raid Reconstructor -- like any good data recovery program -- does NOT write to the drives it's doing the recovery from).   Raid Reconstructor does not need to "know" which disk belongs in what order ... it will determine that as part of the recovery process.

0
 

Author Comment

by:Evil_RSA
Comment Utility
Okay I have to thank pgm554 & garycase for suggesting Raid Reconstructor.  It found the RAID just fine, and being that there wasn't a damaged drive, we were even able to check the XOR test and see that that it looks like every sector is fine and undamaged.  So, we should have a 100% saved RAID.  The next problem we ran into is, this was are biggest RAID set; 1.36 TB, and we don't have that kind of space just, you know, laying around, all in a single "location" for RR to backup to.

We are currently "viewing" the RAID with a "test" copy of it which is version 3.02 that went something like this:
Hey, {insert friends name here}, have you ever heard of or used RAID Reconstructor?
You have?  You have it?!?  Can I borrow it, just to try and see if it's gonna detect a major screwup we did?
Oh man, your awesome.  Thanks so much.

Okay, all that said my questions are this.  Like I said I don't really have that type of space available.  Does the new version allow the recovered image file to be broken up across multiple drives?  I can make the space out of about 5 or 6 drives I have kicking around.  Or, does it allow viewing of the RAID live and this way I could recover the data, and not all the free space that is also on here.

Next question is also, do you think that I would be able to use the information that RR found about the the RAID set to configure my RAID controller ahead of time and then shutdown, hook them up, and see if it works?

Thanks
Derek
0
 
LVL 70

Expert Comment

by:garycase
Comment Utility
I have (fortunately) never needed to actually use Raid Reconstructor to recover data; but I'm fairly sure you can selectively recover data from the array => so you could recover a subset that would fit on drive;  then do it again with another subset; etc. ... and thus use several drives as you've suggested.   But as I noted, I've not actually done that.   In any event ... the good news is you won't lose anything as long as you do not use the drives until you've completed the recovery process :-)
0
 

Author Comment

by:Evil_RSA
Comment Utility
Well, we started do the opposite instead.  We had all these drives kicking around, and thought, man, if only we could just JBOD them all together......then we started counting available SATA and IDE ports and drives we had.  We then started adding drive after drive into the system that this RAID is going back into, making sure to leave the RAID ports free.  We then started to convert them to Dynamic Disk's and adding them to the container....we're moving some of the data around still, but I think we'll be able to get the 1.36TB we need just to house the RAID image.  Verify the RAID image.  Blow away the RAID drives.  Hook them back up the RAID controller and recreate the RAID 5.  Restore the RAID image from the "Spanned" / JBOD'ed drive.  Verify the data restore.  Blow alway all the Drives used in the JBOD, and put them back on the shelf.  I'll let you all know how that goes tomorrow as I need to add another 4-Port SATA card into the system to get the last bit of space, and the card isn't here with us.

Thanks in all the help,
Derek
0
 
LVL 70

Expert Comment

by:garycase
Comment Utility
Sounds like a plan ==>  A spare 1.36TB always comes in handy :-)
0
 

Author Comment

by:Evil_RSA
Comment Utility
Wow, has it been 21 days already?

Well just an update then.....the RAID is currently at 38%.  This is not to say that there haven't been problems.

First off was finding a place to house the 1.36TB RAID image file that it wants to create....after pooling a bunch of drives together; this poor system has 11 drives hooked up to it now....but we were able to get Windows to do a dynamic span of the 6 drives we jammed together and got something just over 1.36TB, which is, all we cared about.

First time around we had the 6 drives on a second system, don't ask us why, we don't know either.....but we span them and had the OS and the 4 RAID drives on another, and were just gonna back up over the network.....hey I said we didn't know why did this either......well not only was it only at 48% after something like FIVE days...but then the switch between the two systems.......yeah well it locked up.....man, that sucked!

Second time we asked our self....if we through in the IDE RAID card and the SATA RAID card.....we can get all these drives working together on one system....lets try that....  Well that would have been great...it was averaging about 25% every 24 hours so for days and it should be done......cool.......nope...it got to somewhere in the eighties and one of our new drives that had a total run time life of less then 5 days.....DIED!  Needless to say....we got an Advanced RMA started, but with the Thanksgiving weekend that really screwed up shipping.

We just got the replacement drive on Monday, got the RAID started and it was really hauling on the recovery this time around....in the first 3 hours it was up to 25%.....then we checked on it the next morning......Windows Login Box!!  Man....okay so we made sure that Auto updates were turned off, and now also unplugged the system from the network all together...and in 25 hours its at 38%....that was 7am this morning.

so, WHEN I get this RAID de-stripped to an image file....I'll be able to run it through "GetDataBack".  The only thing I didn't like about RAID Reconstructor was that it can't recover your actual files for you.....you have to use both pieces of software together to do that.

Eh....its been a LONG process, and luckily, like I said above...I'm glad this wasn't a mission critical system.....though...if it was.....they'd probably let me order 4 new 500GB drives, build the RAID back up on system, and restore the broken RAID right back to it...but as its not critical they didn't' want me to buy any more drives then absolutely necessary.  Sucks to have your hands tied.

~Derek
0
 

Author Comment

by:Evil_RSA
Comment Utility
Woot!!  Got the RAID back!!!!!!!!!!!!!!!!!!

Had major problems with RAID Recon. just kinda dying on me towards the end of each run.  Thought maybe it was Anti-virus having an issue scanning the couple hundred gig RAID image file at one point, so I clean installed XP again and tried again....thought maybe the drives were over heating on me at one point...took care of that....just could never seem to get it to go past 80%.....not that was the fail area all the time..it was quite random..some times as low as 3%.  However I was able to have it do a scan of the RAID like it does every time...and figure out my RAID order.  I was then able to reconstruct my RAID configuration on the RAID controller and then hook the drives back up in the correct order.  I just booted back into windows with the other drive and everything is happy!!!!

Awesome.  Spent a lot, and I mean a lot of time on this project, but it's worth it to not have to really decontruct the array anymore.  Now I'm printing up labels with our little Brother label maker and labeling the drives as t o what container they goto, and what drive they are...... Alpha Beta Gamma Delta.....oh this is just awesome.

I love this community!!!!
0
 
LVL 70

Expert Comment

by:garycase
Comment Utility
Great !!   ... and labeling the disks is an excellent idea :-)
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Lets start to have a small explanation what is VAAI(vStorage API for Array Integration ) and what are the benefits using it. VAAI is an API framework in VMware that enable some Storage tasks. It first presented in ESXi 4.1, but only after 5.x sup…
The article will include the best Data Recovery Tools along with their Features, Capabilities, and their Download Links. Hope you’ll enjoy it and will choose the one as required by you.
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This tutorial will walk an individual through the process of installing the necessary services and then configuring a Windows Server 2012 system as an iSCSI target. To install the necessary roles, go to Server Manager, and select Add Roles and Featu…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now