Link to home
Start Free TrialLog in
Avatar of SINC_dmack
SINC_dmack

asked on

Unable to get 8th drive bay working on Smart Array p410i, ProLiant ML350 G6

We have a ProLiant ML350 G6 with an integrated p410i Smart Array storage controller.
We were looking to add a disk to the 8th drive bay to expand the array, but the 8th bay appears inoperative.
Drives inserted into bay 8 don't even spin up and show now activity lights.
We suspected this might point to an electrical issue, so we tried replacing the backplane, but no change.
Instead, the controller is detecting the internal tape drive as being connected to bay 8.

The tape drive is connected via a break-out connector on the miniSAS cable for bays 5-8 between the drive backplane and the controller ports on the motherboard.
We've tried just disconnecting the tape drive, but the controller still shows the tape drive (disconnected) as being on bay 8 and never see's the SAS drive.
We also tried switching to a miniSAS cable that doesn't have the break-out connector for the tape drive, but the controller reports that none of the drives in those bays is detected when we do so.
We tried this with a second miniSAS cable in case the first was defective, but had the same result.

We've confirmed the controller is on the newest firmware, and it does have a battery unit connected.
It also lists having a "Flex License" on the controller.

Is there some special step required to disable the tape drive and make the controller see the SAS drive in bay 8 instead?
Avatar of Member_2_231077
Member_2_231077

Could you generate an ADUreport with a disk in bay8 plus normal SAS cables connecting the backplane to the motherboard and upload it, the tape drive info may be on the disk metadata in which case it may not be able to be deleted.

Can you also boot the server with no disks except for one in bay8 and verify the drive is seen. The controller will pick up the config from the original disks when you put them back and boot normally. You have to put them back in the same slots though so label them up before removing.
Avatar of SINC_dmack

ASKER

Is it possible to generate an ADU report without booting the OS?
If I use a SAS cable that doesn't include the tape drive connector, all the drives on that cable aren't detected by the RAID controller, that would constitute 3 lost drives in the server's RAID 5 array.

I can certainly try powering up the server with only bay 8 occupied to see if the controller see's it then, I would have to schedule a time next week when the server can go down for this.
You canuse Smartstart CD to generate the ADU report.

>If I use a SAS cable that doesn't include the tape drive connector, all the drives on that cable aren't detected by the RAID controller

Must be a faulty cable then. Both cables from controller to  backplane are normally the same 4 lane cables.
I had thought the same, but we used two different standard mini-SAS cables on that connection, and had the same result on both.
One had been left in the case from manufacture, I'm assuming that's in case you remove the tape drive at some point.
The other we had ordered brand new not realizing one was already left inside the case.

I will attempt to generate the ADU from the Smartstart CD.
You did plug the extra disk in hot didn't you? Just thought that if you plugged it in cold and it was second hand and you ignored POST messages then it would be disabled in firmware.
We tried adding 8th drive hot and cold. We never even get an indicator to suggest the controller is checking the drive. I don't believe the drive even spins up.
ADU report?
We've been in touch with HP support and they're having us reset a dipswitch on the motherboard to clear out some settings.  We probably won't do any work on it until Friday (so we have the weekend in case something blows up) and will follow up with details early next week.
If it's the system maintenance switch switch 6 then it is very unlikely to have an effect as it clears down BIOS but the RAID controller settings aren't stored in CMOS.
In the meantime, attached are ADU reports from before and after we disconnected the tape drive in the server.
report-ccd971db-000074f8-00000000-3.zip
report-dd7934df-0000756c-00000000-3.zip
No mention of a tape drive in bay 8 after changing it.

Are you absolutely sure the drive is seated correctly? We've had instances of people not opening the handle before fitting it and also people using new-style caddies in old-style servers. I've even gone to site only to find a disk is mounted backwards.
The caddie is identical, and the drive is the exact model as the others.
To check for a DOA drive, we took the hot spare from bay 4 and swapped them. The new drive was detected in bay 4, and the known working drive didn't even spin up in bay 8.
HPE support has officially closed the case. They've decided that the integrated controller is somehow defective or faulting.
Their recommendation is to replace the motherboard or install an add-in controller to replace the integrated one.
I think HPE support are wrong and that the fault is in the metadata (RIS) stored on the disks even though we can't see it in the text. They have a dubious tool to examine ADU reports and I have many years of experience but I don't have access to the source code.

Simple test, power off and take the seven disks out labeled up with the slot numbers. Put the new disk in bay 8 and see what happens in ORCA and ACU on SmartStart CD. You can eliminate cable and controller (motherboard) port by cross-connecting the two SAS cables and you can also remove any cache DIMM in case the problem is stored on that. After the experiment you can put the seven disks back in and the controller will pick up the config from them and boot same as before (as long as you have undone the SAS cable crossover we did during testing).
In the event that issue is with the metadata on the disks, what would the resolution be?  I'm guessing that it would involve taking an image of the server, deleting the RAID, recreating the RAID, and then restoring from the image.
Yes, there aren't any tools to edit it so you would have to delete the array and recreate.
Andy, my tech's comments to your suggestion:
"Two issues with his recommendation. First, even before the controller firmware loads, the disk in bay 8 never even spins up. Second, we already know that trying to use any cable except the breakout cable on the disk 5-8 side of the backplane renders that entire side of the backplane inaccessible on the controller."

How does that affect your recommendation?
Maybe the bakplane connector is damaged, you didn't mention that all 4 bays stopped working when you used a normal cable. A new backplane is only £15.

The disks don't spin up on their own, the controller sends them a spin-up command. The LEDs are meaningless until the controller loads since it turns them on and off through the sideband part of the cable.
We've actually replaced the backplane as part of one of the early rounds of diagnostics, with no change.  (Mentioned in the first post.)  Any other suggestions?  Thanks!
You need to test with all disks out except for your new one in bay 8.
My tech says "I tried booting to the RAID config with only drive 8 installed, the drive never even spins up."
ASKER CERTIFIED SOLUTION
Avatar of Member_2_231077
Member_2_231077

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Wow, P410s ARE really cheap--under $10 shipped for one on Ebay.  We ordered one last week and will do some in-house testing with it (create a RAID, write data to it) to ensure that it's working properly, and then give it a try in the server.

I don't know if my tech has attempted to get drive #8 working with the cache module removed, but we can give that a shot during the same onsite visit when we install the RAID controller, just for the sake of knowing.
After testing and confirming it was properly functioning on delivery, we had the new controller taken onsite this morning and installed.
We went ahead and used the extra MiniSAS cables we had and left the tape drive breakout cable disconnected.
The array was detected and imported automatically, and the drive in bay 8 was detected without any issues.
We also transferred over the cache/battery module and the feature license on the controller without issues.

It looks like the integrated controller was the issue all along. I appreciate all your help and recommendations.
Odd that just a single SAS lane failed although the breakout cable could have damaged it and really odd that four lanes got disabled when you used a normal cable. Glad it got fixed cheaply though.