DataICU
asked on
How do you identify the Mac RAID 5 block size and order?
I'm working with a Mac issue Pro Server with a 4-drive RAID 5 which failed in service. I imaged with success all drives to image files and have been trying to reassemble the data, relying mostly on R-Studio since it will recognize the HFS. Using the default settings, I went through all 24 permutations and have settled on 6 of them to do a full scan based on a brief scan which showed a partition of proper size. Each scan takes about seven hours to complete, so while I wait I am trying to learn if the standard "blue" Mac card (which has no configuration apart from the Mac standard -- whatever that is) uses a 64KB block size and Left Synchronous block order. The web is void of this information. Anyone have any knowledge or experience on this?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Correct, it is a waste of your time and money. R-Studio is ONLY good for healthy RAID. It does not have the intelligence to deal with what I mentioned. R-Studio does have it's place, i've used it in the past to recover busted filesystems ... but only AFTER I restored the RAID.
Just turn off the rstudio, it is making things worse. You can't recover the file system that is on a volume,w/o recovering the logical volumes first. To do that, you need to figure out mapping of physical volumes -> logical volumes, and see what happened to it, and recover accordingly.
As you won't be able to purchase the software you need, then you may be able to figure things out with a binary editor and making manual XOR calculations every multiple of 4,8,16 ... 64 KB, look for ASCII strings that wrap over the last few bytes of each chunk. Then you have to do some XOR calculations
Anyway even if you had the configuration and knew that a rebuild started and then stopped and the disks and the ordering ... then you would still need some sophisticated software to do some statistical sampling and look at the logical volume data as each stripe is manually recovered or repaired with faux parity.
Take it to a recovery lab with a few thousand dollars. If the money isn't there, then just reformat and move on. Best possible case scenario, you can manually recover files no bigger than about 120KB, and then only if they are contiguous, and even then in a perfect world, where all the files were no bigger than that, don't expect more than 33% of them.
Just turn off the rstudio, it is making things worse. You can't recover the file system that is on a volume,w/o recovering the logical volumes first. To do that, you need to figure out mapping of physical volumes -> logical volumes, and see what happened to it, and recover accordingly.
As you won't be able to purchase the software you need, then you may be able to figure things out with a binary editor and making manual XOR calculations every multiple of 4,8,16 ... 64 KB, look for ASCII strings that wrap over the last few bytes of each chunk. Then you have to do some XOR calculations
Anyway even if you had the configuration and knew that a rebuild started and then stopped and the disks and the ordering ... then you would still need some sophisticated software to do some statistical sampling and look at the logical volume data as each stripe is manually recovered or repaired with faux parity.
Take it to a recovery lab with a few thousand dollars. If the money isn't there, then just reformat and move on. Best possible case scenario, you can manually recover files no bigger than about 120KB, and then only if they are contiguous, and even then in a perfect world, where all the files were no bigger than that, don't expect more than 33% of them.
ASKER
dlethe:
I understand everything you said in the last two posts. I do data recovery for a living and have for 20 years. And I've recovered RAIDS hundreds of times -- Lacies, Buffalows, Windows -- but haven't been presented with a Mac RAID 5 until now.
I can see directory structure, but not good files with my attempts so far.
You said, "R-Studio is ONLY good for healthy RAID" -- and I've used it many times, as well as the Runtime product for non-Mac RAIDS with grand success.
I even created a new working Mac RAID on the same system with four new drives and cannot take those drives into R-Studio and get a read.
The beginning of the RAID is 28 GB from the beginning on the original failed raid (of which I have all drives perfectly imaged to files -- and nothing was rebuilt. The client put in a new drive in the place of the one they thought had failed, but I did a hex read on it and it is all clean, so no rebuild was done.
The real secret seems to be what the standard Mac blue card parameters are. R-Studio defaults to a 64KB block size and Left Synchronous block order. With those settings, I can get six of the 24 possible drive orders to show me the proper size partition with a good directory structure, yet not with good file population.
And, R-Studio is not making matters worse. I'm working with images, and they are not being changed by those operations.
If you know what the Mac blue card defaults are, I'd appreciate getting that info. That seems to be a mystery, Internet-wide.
DataICU
I understand everything you said in the last two posts. I do data recovery for a living and have for 20 years. And I've recovered RAIDS hundreds of times -- Lacies, Buffalows, Windows -- but haven't been presented with a Mac RAID 5 until now.
I can see directory structure, but not good files with my attempts so far.
You said, "R-Studio is ONLY good for healthy RAID" -- and I've used it many times, as well as the Runtime product for non-Mac RAIDS with grand success.
I even created a new working Mac RAID on the same system with four new drives and cannot take those drives into R-Studio and get a read.
The beginning of the RAID is 28 GB from the beginning on the original failed raid (of which I have all drives perfectly imaged to files -- and nothing was rebuilt. The client put in a new drive in the place of the one they thought had failed, but I did a hex read on it and it is all clean, so no rebuild was done.
The real secret seems to be what the standard Mac blue card parameters are. R-Studio defaults to a 64KB block size and Left Synchronous block order. With those settings, I can get six of the 24 possible drive orders to show me the proper size partition with a good directory structure, yet not with good file population.
And, R-Studio is not making matters worse. I'm working with images, and they are not being changed by those operations.
If you know what the Mac blue card defaults are, I'd appreciate getting that info. That seems to be a mystery, Internet-wide.
DataICU
Let me clarify .. it is not capable of dealing with unhealthy, inconsistent RAID. Rebuilding parity is or brute force figuring out topology or file / volume info is the extent of the capability. If it was capable of multiple failure scenarios then you would be online by now and/or the software would cost 100X more than what you paid for it.
Have you done XOR parity verification at the stripe level? Do any two disks have the same content? Do you have a non recoverable read error at any particular block, and does the parity the opposite of what it should be?
Since you are no novice (I sometimes get a little short with people, I apologize if I came off a little harsh ...) then here are some things you could try
#1 Forget the file system.
It isn't important right now. This is where you are going wrong. You MUST get the hardware figured out first then worry about striping & block size. You don't need to worry about that until you first see if they ran degraded, or partial rebuild started, or whatever ..
Run an xor test
Have you done XOR parity verification at the stripe level? Do any two disks have the same content? Do you have a non recoverable read error at any particular block, and does the parity the opposite of what it should be?
Since you are no novice (I sometimes get a little short with people, I apologize if I came off a little harsh ...) then here are some things you could try
#1 Forget the file system.
It isn't important right now. This is where you are going wrong. You MUST get the hardware figured out first then worry about striping & block size. You don't need to worry about that until you first see if they ran degraded, or partial rebuild started, or whatever ..
Run an xor test
ASKER
dlethe:
All understood,again. And thank you.
Let's go with this: Same machine, four brand new drives. I run the Snow Leopard install disk, set up the RAID 5 on all four disks with no options for anything else being presented.
I fully install a working operating system on this array and all is verified and well. I pull these four drives and try to see them with R-Studio and I cannot recreate the RAID, yet I can put them back into the host machine with the same RAID card (blue card Mac) and it boots just fine.
If I can get the R-Studio product to see the known good drives as a viable Mac RAID 5, then it might indicate the settings to work with the other. But, I cannot see the known good RAID set with R-Studio.
I am bouncing here off of a post at the R-TT blog from another expert saying:
There are zillions of different RAID cards vendors, which are using different settings for different RAID configurations, with exception of ONE - Apple. Apple has only TWO cards, which support the INTERNAL MAC PRO RAID - black card (for Mac Pro early 2009 and later) and the original - blue card (for Mac Pro up to early 2008, which is Mac3.1).
Two simple questions: 1. Why there are NOTHING regarding Apple files restoration (templates, etc) in R-Studio for Mac? (!!!)
2. Who knows standard Drive order, Offset, Block Size, Block order for these cards? These parameters, opposite to other vendors, Apple establishes "by default", never asking us, users, what we want. It means these data are constant and we, users, will be so glad to get it from R-TT engineers or other R-TT Forum users.
The R-TT moderator agrees and says they should work on it, but so far, no solution. This is from a thread started last October (2010) and one that is still alive. I've written to the poster to see if he has a solution.
It seems to me that if the Mac blue and black card parameters were known, we could use this tool to do a recovery on a perfect set of images (and a known perfect set, as I have created). But, when you go from one set of block sizes and order schemes, you take a 24-possible combination to one of tens-of-thousands.
It's a given that R-Studio doesn't have the ability to "learn" the parameters from even a good RAID 5 set.
And the Runtime product can't do it either -- though it can do it on a healthy set of Windows/NTFS RAID 5 drives.
In the Windows RAID environment with perfect images of all drives, a Runtime Raid Reconstructor can get you there -- or -- with a bit of trouble -- or a few hundred bucks to the Runtime folks will get the answer.
Here, we have the Mac blue and black RAID cards with no public information about the parameters.
I thought I could reverse engineer with a perfect set of drives, but apparently not without thousands of combinations in the R-Studio software.
Still searching.
Thanks again for your input.
DataICU
All understood,again. And thank you.
Let's go with this: Same machine, four brand new drives. I run the Snow Leopard install disk, set up the RAID 5 on all four disks with no options for anything else being presented.
I fully install a working operating system on this array and all is verified and well. I pull these four drives and try to see them with R-Studio and I cannot recreate the RAID, yet I can put them back into the host machine with the same RAID card (blue card Mac) and it boots just fine.
If I can get the R-Studio product to see the known good drives as a viable Mac RAID 5, then it might indicate the settings to work with the other. But, I cannot see the known good RAID set with R-Studio.
I am bouncing here off of a post at the R-TT blog from another expert saying:
There are zillions of different RAID cards vendors, which are using different settings for different RAID configurations, with exception of ONE - Apple. Apple has only TWO cards, which support the INTERNAL MAC PRO RAID - black card (for Mac Pro early 2009 and later) and the original - blue card (for Mac Pro up to early 2008, which is Mac3.1).
Two simple questions: 1. Why there are NOTHING regarding Apple files restoration (templates, etc) in R-Studio for Mac? (!!!)
2. Who knows standard Drive order, Offset, Block Size, Block order for these cards? These parameters, opposite to other vendors, Apple establishes "by default", never asking us, users, what we want. It means these data are constant and we, users, will be so glad to get it from R-TT engineers or other R-TT Forum users.
The R-TT moderator agrees and says they should work on it, but so far, no solution. This is from a thread started last October (2010) and one that is still alive. I've written to the poster to see if he has a solution.
It seems to me that if the Mac blue and black card parameters were known, we could use this tool to do a recovery on a perfect set of images (and a known perfect set, as I have created). But, when you go from one set of block sizes and order schemes, you take a 24-possible combination to one of tens-of-thousands.
It's a given that R-Studio doesn't have the ability to "learn" the parameters from even a good RAID 5 set.
And the Runtime product can't do it either -- though it can do it on a healthy set of Windows/NTFS RAID 5 drives.
In the Windows RAID environment with perfect images of all drives, a Runtime Raid Reconstructor can get you there -- or -- with a bit of trouble -- or a few hundred bucks to the Runtime folks will get the answer.
Here, we have the Mac blue and black RAID cards with no public information about the parameters.
I thought I could reverse engineer with a perfect set of drives, but apparently not without thousands of combinations in the R-Studio software.
Still searching.
Thanks again for your input.
DataICU
ASKER