Any Compaq/HP Smart Array Experts? Smart Array 5302

Hello and thanks for looking at this problem.  

We took a server down for a scheduled upgrade - adding another 4254R Duplex 14 drive canister - to act as a drive to hold backups of a large multi array system whose tape library was malfunctioning.  

While rearranging the array drives, a RAID5 4-disk (call it logicalBroken) volume was accidentally split on two separate controllers, and mixed with two other logical  volumes.

This of course is the data that was not being backed up due to a bad AIT drive in the tape library.

The server was booted with the logicalBroken drives split in half – two of the four logicalBroken drives on two separate smart array controllers, one 5302, one 5304. This mixed the logicalBroken drives into to other logical drives.

As soon as the array controllers initialized – the error messages came up, i.e. arrays missing drives, drives at the wrong IDs, ECT.

The server was shutdown immediately.  

The logicalBroken drives were put back into their original positions, in the original cage, on the original controller and the server re-started.  
 
logicalBroken, of course, did not come back as a valid logical drive.  So here is where I am :

From the Array Diagnostic Utility:

SLOT 4 SMART ARRAY 5300 CONTROLLER ERROR REPORT:

   Swapped cables or Configuration error detected.
      An unsupported drive rearrangement was attempted.  SOLUTION: Power down
      system then move the drives back to their original location.
   Controller is not configured.
   Drive at Port 2, ID 0 indicates it should be located at Port 1, ID 1
   Drive at Port 2, ID 1 indicates it should be located at Port 4, ID 1
   Drive at Port 2, ID 2 indicates it should be located at Port 1, ID 0
   Drive at Port 2, ID 3 indicates it should be located at Port 4, ID 3
   RIS copies between drives do not match
   No configuration for array accelerator board

I do not want to do anything else to the drives till I can get an expert on this subject to help. I already called HP support and was incredibly unimpressed with the support group available today.

The drives have not been wiped, re-initialized, or booted into an array.  So the data **should** still be there.

1) Do we have anybody here that has dealt with this before? Can you edit the RIS Hex and put this back together?
2) Are there any data retrieval companies that may be able to handle this?
3) Does HP have a priority or super support group for smart arrays?  Today (Saturday) must not be a good day for support experts  at HP . . .  

Thanks in advance,
Tom

 
tvurtAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

sirbountyCommented:
"While rearranging the array drives, a RAID5 4-disk (call it logicalBroken) volume was accidentally split on two separate controllers, and mixed with two other logical  volumes."

If you don't mind my asking - how did this 'accidentally' occur?

As for HP support - I either hang up and try calling back or simply ask them to escalate the problem...
andyalderCommented:
I think you need a third party recovery company; HP won't even give me the tools to re-write the metadata and I've been a master ASE in StorageWorks since Compaq bought DEC and an ASE in ProLiants since they came out. If they have expertise the third party recovery company will have reverse-engineered DAAD/ADU to work out how to rewrite the RIS and other gunk at the begining of the drive. HP's standard solution for "An unsupported drive rearrangement was attempted" is to use a later version of the diagnostic tool.

Suggest ask CS to zero/paq this unless someone has access to the real tools.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
tvurtAuthor Commented:
Thanks for the recommendation - I will not be touching these drives again myself.  I am reviewing the documentation on several companies that claim to be able to restore this data.  

I will update this forum on the results of the restore!

sirbounty - A 4254R Duplex 14 drive canister has a physical split between two separate SCSI busses.  Each bus was plugged into a separate 5300 controller.  The raid 5 was split along drive positions 7/8 - two drives on the SCSI bus1, two drives on the SCSI bus2.  When the 5300 initializes the drives - it re-wrote the meta data on the drives telling them how to access the logical RAID5 volume.

Tom
Your Guide to Achieving IT Business Success

The IT Service Excellence Tool Kit has best practices to keep your clients happy and business booming. Inside, you’ll find everything you need to increase client satisfaction and retention, become more competitive, and increase your overall success.

ns_germanyCommented:
Hi Tom,

maybe there's a way to recover for you:
I had this issue on a Smart Array 5i controller when trying to use a previously hot-unplugged disk to restore a former OS patch level. Error messages like yours came up and in the end 5i had lost array information of most actual disks, too. Uh.
Outcome of a long night with an HP expert from Finland on the phone:
- make sure your Smart Array Controller has most actual firmware
  (downloaded file from HP website + installed it using SmartStart CD)
- power down + cut power supplies (= REMOVE all mains cable(s) of this server for a minute!)
- unplug all disks
- power on: machine is complaining: no array info found [here Smart Array Controller deletes internal saved array info]
- power down + cut power supplies (= REMOVE all mains cable(s) of this server for a minute!)
- insert disks in desired slots
- power on: machine tells: found array info on disk, can be restored but data loss is possible
- chose F2: restore array config from disk
- reboot

After this all data was available again and no loss was encountered. (Sigh!)

Since then we're using some Compaq tools on a DOS bootdisk to save array configs to a file and keep it available for emergency. If you like to know details I can have a look when I'm at work again in a few hours.
Hope you will succeed with it.
ns_germanyCommented:
tvurtAuthor Commented:
Thanks for your comments ns_germany.  The drives are in the hands of "Drivesavers" which claims that they can re-build the array.  I will post the results of this effort!

Tom
ns_germanyCommented:
Uh, maybe they will do the job like HP showed it to me (see above), but probably it will be more expensive.  8-(

Looking foward to your report.
And if you like to save your array configurations proactively just use 'Compaq Array Configuration Utility'. This is a DOS that provides a DOS bootdisk to do the job.
Cheers,
Norbert
ns_germanyCommented:
Sorry, this is the missing letter in 'foward:  'r'.
And here is the last sentence as it should be: 'This is a DOS tool that provides a DOS bootdisk to do the job.'
Too much typos today.
tvurtAuthor Commented:
Ok just got the drives back online today.  DriveSavers in CA successfully rebuilt the drives.  It was pricey but less expensive than re creating the data!

I can highly recommend DriveSavers.

Tom

It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Storage

From novice to tech pro — start learning today.