Solved

Possible Server Failuer

Posted on 2015-01-26
64
109 Views
Last Modified: 2015-02-26
HP Proliant DL380 Server with 6 72GB disks in an array. The first disk has failed and the server won't recognize any disks. Message was that the array failed and then another message telling me which disk. If I take that disk out and insert a different 72GB disk in the same slot and then try to install Windows Server 2008 no disks are recognized. If I attempt to repair my computer from the Server 2008 disk the only disk is sees is the external HDD with the backup image on it. Do I have a failed server, failed disks or both? Do I need to remove all the disks in the array and insert the blank disk and try to image it? Please help with this as this is a school server and we need to get back up and running ASAP.

Thank you!

Robert
0
Comment
Question by:RobertEhinger
  • 30
  • 14
  • 9
  • +3
64 Comments
 
LVL 46

Expert Comment

by:noxcho
ID: 40571799
You need first to load the driver for a RAID controller only then will the Windows install process see your RAIDed drives.
0
 

Author Comment

by:RobertEhinger
ID: 40571811
Where do I get that driver? This entire array used to work until the first drive in the array failed.
0
 
LVL 34

Expert Comment

by:Seth Simmons
ID: 40571886
you must have something else going on; the system shouldn't die like that with 1 failed drive
guessing it's raid 5?
0
 
LVL 24

Expert Comment

by:VB ITS
ID: 40571893
What RAID level was the array configured for?

A single disk failure shouldn't break your RAID array unless you had them configured in RAID0.

You could very well have a faulty RAID controller or controller battery.
0
 

Author Comment

by:RobertEhinger
ID: 40571895
The drives were all installed in an array. The message was the array failed and then a message telling me the first disk in the array failed with the location of the disk. When I put in a known good drive it still did not see any drives. RAID 5 - yes.
0
 

Author Comment

by:RobertEhinger
ID: 40571899
So if I have a faulty RAID controller or controller battery am I better off  replacing the server?
0
 
LVL 24

Expert Comment

by:VB ITS
ID: 40571907
So if I have a faulty RAID controller or controller battery am I better off  replacing the server?
No, not unless you really have to. It's probably cheaper to just replace the RAID controller if it is indeed faulty.

When you see the error message regarding the failed drive, you should be able to still boot into Windows if it's a RAID5 array.

Is this not the case for you? If not, can you please provide us with the message you see on the screen?

A RAID5 array can survive a single drive failure. If you have two failed drives though then your array is gone and you'll need to recover your data through backups.
0
 
LVL 46

Expert Comment

by:noxcho
ID: 40572249
The drivers can be found on the official website of the hardware provider. Or you have a driver CD which came with the server.
0
 
LVL 87

Expert Comment

by:rindi
ID: 40572286
It is likely that one or more of your other disks were also failing or at least had errors on them. That would cause an array to fail.

Most older HP servers came a SmartStart CD. It is used to boot the server, setup it's array, select the OS you will be installing, and then it prepares the system. Once ready the CD pops out and you are asked to insert your OS installation CD or DVD. It then carries on with the installation and you don't need to provide the drivers.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 40572516
If you boot SmartStart, then under diagnostics generate an ADU report you can attach it here and someone will have a look. Sounds like DL380 g4 or previous, raid controller is inbuilt on mobo although you can put one in a PCI slot instead of changing motherboard.
0
 

Author Comment

by:RobertEhinger
ID: 40572548
How do I boot SmartStart?

Why, if I replace the disk do no HDDs show up if I try to reinstall Server 2008?

I am attaching pictures of the errors I see as I boot the system. As you can see there is a red indicator light on the failed drive indicating failure.
Failed-HDD.jpg
Backup-ROM.jpg
Server-Error.jpg
Server-Hangs.jpg
0
 
LVL 87

Expert Comment

by:rindi
ID: 40572576
You put your SmartStart CD into the optical drive, then go into the server's BIOS and make sure the optical drive is the first in your boot order.

If there is no array setup, you won't see any disks even if you load the controller's drivers. If an array is setup and running, and you have loaded the drivers for it, the 2008 DVD will also see the array as a disk.
0
 
LVL 87

Expert Comment

by:rindi
ID: 40572578
Besides that, you say your disks are all 72 GB. But the one that is showing as failing in your photo is only 32GB. You need to replace your bad disks with another of at least the same size as the others.
0
 
LVL 24

Expert Comment

by:VB ITS
ID: 40572699
The 32GB label could just be from a spare hard drive caddy rindi.

From your pictures it looks like your server is not booting to the Smart Array controller. Did you go into the BIOS and change the boot order around or anything along those lines?
0
 

Author Comment

by:RobertEhinger
ID: 40572901
I have not changed the boot order and there are no empty HDD caddies in the server. I guess I didn't look quite closely enough. The other 5 are 72GB but that failed drive is 36GB.

The main question I have is whether AI need to replace the server or just the hard disk.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 40572944
Default boot order is USB before array controller so make sure there are no USB devices plugged in.

>the only disk is sees is the external HDD with the backup image on it

Yes, that's the one, unplug it and see if the machine boots.
0
 

Author Comment

by:RobertEhinger
ID: 40575032
SmartStart would not boot with the failed drive in the server. Once I took it out and installed a known good drive SmartStart booted but found no bootable device. Evidently when this server was setup there was no array and I am not really all that familiar with this process. I am sending a couple of images and asking what to do next. Do I use the array configuration utility?
NoBootableDevice.jpg
0
 

Author Comment

by:RobertEhinger
ID: 40575087
And the next question I have is - will I be able to reload my saved image once I have an array configured?
0
 
LVL 24

Expert Comment

by:VB ITS
ID: 40575338
Hold on. Sorry, but I'm a bit confused here. I thought you said this server had a RAID5 array originally, and only one hard drive had died? If this is the case then you should have been able to simply replace the faulty drive (as you have already done above) then restart the server without any USB drives plugged in.

It should then allow the server to boot. If it doesn't, go into the Array Configuration Utility via the SmartStart CD and just review what's going on.

If you're comfortable with wiping/rebuilding the array and performing a restore then by all means do so.
And the next question I have is - will I be able to reload my saved image once I have an array configured?
Through what program was this backup image created?
0
 

Author Comment

by:RobertEhinger
ID: 40575447
I thought it did have RAID5. I didn't set it up and I guess I was assuming that was how it was initially configured.

With the failed HDD installed the server will not boot to SmartStart. Like I said, if I replace it with a good HDD it boots and I am at the screen from which I sent the image. If I boot to the Array Configuration Utility will that show me what is already installed or will that just walk me through setting up a new array?

I now have six 72GB HDDs in the server.

The image was created from Windows Server 2008 R2.
0
 

Author Comment

by:RobertEhinger
ID: 40576569
Here are a couple more images. One is an error report that was run after I replacing the failed hard disk and the other shows the 6 physical drives. Please advise, where do I go from here. Do I run the wizard to configure an array? Is there something else I should be doing.
Config.jpg
ErrorReport.jpg
0
 
LVL 24

Expert Comment

by:VB ITS
ID: 40576666
Where did you get the spare drive from? Also, according to the ErrorReport picture in your post above, the hard drives appear to have been re-ordered physically. Did you actually move the drives around physically?
0
 

Author Comment

by:RobertEhinger
ID: 40576689
The drive is rom an old decommissioned HP Proliant server and the only change to the drives is that one that I put into the same slot as the failed drive.
0
 
LVL 24

Expert Comment

by:VB ITS
ID: 40576692
Is there data on this server that you need to recover? Do you have a backup of this server before we go ahead and recreate the array?
0
 

Author Comment

by:RobertEhinger
ID: 40577082
There is an image on the external drive that I tried to restore but Server 2008 couldn't find disks ready to accept the image. There is data on this server.
0
 
LVL 55

Expert Comment

by:andyalder
ID: 40577146
If the drive is from an old server it must be fitted hot since the controller will read the metadata from it and as that does not match the metadata on the current disks it will get confused and refuse to play, that's why it doesn't work properly with it fitted. Taking it out, powering on, waiting a minute and then plugging it in will resolve that. You should have had a message during POST about that, you have to disable the splash screen or press a key to see POST messages on some models.

As already mentioned SmartStart will install Windows for you and provide drivers but you need to create an array and logical disk first, you can do that with the ACU under SmartStart maintenance or by pressing F8 during POST to configure the controller.
0
 

Author Comment

by:RobertEhinger
ID: 40579567
I want to start working on this over the weekend so what will be my best course of action?
0
 
LVL 24

Expert Comment

by:VB ITS
ID: 40580032
There is an image on the external drive that I tried to restore but Server 2008 couldn't find disks ready to accept the image. There is data on this server.
Please be aware that if you create a new RAID array, you will lose all of the data on the disks. This is why I asked if you have a recent backup of the server.

I don't want to be responsible for the loss of any data so please verify if you have a recent backup or not. Can you also please clarify what you mean by 'image'? Are you referring an image file created through some sort of backup software? If so, what is this backup software?
0
 

Author Comment

by:RobertEhinger
ID: 40580672
Its a Windows Image.
0
 

Author Comment

by:RobertEhinger
ID: 40580942
What are my other options?
0
 

Author Comment

by:RobertEhinger
ID: 40583991
Is there a way to recover anything from the failed drive? Can I install the operating system on the replacement drive without losing the data on the other five disks?
0
 
LVL 24

Expert Comment

by:VB ITS
ID: 40584005
Sorry Robert, missed your other replies.
Is there a way to recover anything from the failed drive?
If you don't have backups of the data on this server then I'm afraid I don't like  your chances of recovering the data on there.

Can I install the operating system on the replacement drive without losing the data on the other five disks?
No, not if you do this through SmartStart as it doesn't detect your array in the first place and wants to build a new one. Have you tried pressing the Esc at this screen to see if it will continue booting into Windows?: http://filedb.experts-exchange.com/incoming/2015/01_w05/894603/Server-Error.jpg

The fact that SmartStart is seeing all your drives as Unassigned concerns me. Do you have access to another similar HP server that has an external SmartArray controller card?
0
Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

 

Author Comment

by:RobertEhinger
ID: 40584042
" Do you have access to another similar HP server that has an external SmartArray controller card?"

Unfortunately, no.
0
 
LVL 46

Expert Comment

by:noxcho
ID: 40584101
Just to summarize your problem: the RAID failed but you have an image of the configuration taken with Windows Server Backup, right?
0
 

Author Comment

by:RobertEhinger
ID: 40584262
Correct.
0
 
LVL 46

Expert Comment

by:noxcho
ID: 40584364
Then to check if your image is correct I would recommend you restoring it to a test machine which has a driver in size similar to one you took the image from.
If the system restores and starts after this ok then you can reguild the RAID on old server and perform restore over newly rebuilt RAID.
The restore ca be done to a virtual machine as well.
0
 
LVL 24

Expert Comment

by:VB ITS
ID: 40585364
Sorry Robert, I think we're getting a bit mixed up here. Do you or do you not have a backup of the data on this server?

I assumed earlier that when you referred to an 'image' of the server, you were talking about the windows install disk. Is this correct or do you actually have a backup image of the server taken from Windows Server Backup or some other third party software?
0
 

Author Comment

by:RobertEhinger
ID: 40587787
I have an image on my external hard drive that was created from Windows. When this first happened I tried to restore the image using the Windows the restore option from the installation disk. The image was located but could not be restored because no formatted drives were found.
0
 
LVL 87

Expert Comment

by:rindi
ID: 40588073
That is normal as you first need a working, good array to restore to, and you probably also need the drivers for the RAID controller If it isn't already included with the OS DVD.
0
 
LVL 46

Expert Comment

by:noxcho
ID: 40588116
As said already, try to restore the image over a single HDD. The capacity should be the same as the capacity of the RAID was.
This way you can bring the server online and check if the image is ok.
After that resolve the problem with RAID, or configure new RAID with new controller and restore the image over it.
0
 
LVL 87

Expert Comment

by:rindi
ID: 40588278
You could also mount the VHD that was created on another PC using Windows disk-management. Then you can browse the contents of your backup and check if all data is there. You could then even restore critical files, not just the complete backup.
0
 

Author Comment

by:RobertEhinger
ID: 40589276
How do I format just one hard drive so that I can restore the image?
0
 
LVL 87

Expert Comment

by:rindi
ID: 40589314
Why just one disk? Create a new array and restore to that array, or restore the system to another PC, or just mount the VHD and copy the important files to some other location.
0
 
LVL 46

Expert Comment

by:noxcho
ID: 40589569
Restore will format it itself.
0
 

Author Comment

by:RobertEhinger
ID: 40596391
I will be attempting the restore tomorrow. I will post the results.
0
 

Author Comment

by:RobertEhinger
ID: 40597505
This is what happened today when I went in to try an restore the image. I have 4 other 36GB HDDs from the previously decommissioned server that I thought one would work to replace the failed 36GB drive. I tried each one in turn and that all showed that they failed with the same red light as I saw initially.  I then installed a new 72GB HDD that I had on hand and it gave me the green light so I then tried restoring the image. It failed with the error I am including with this message.
I then thought I wouldn't be any worse off if I tried restoring with one of the 36GB drives and the restore succeeded. It appears all my settings and data were restored and I have a working server with one failed hard drive.
I guess I don't understand why the 36GB drives are all failing and I definitely don't understand why the image restored with a failed hard drive.
Now what will happen if I hot swap the failed 36GB drive with the 72GB? What If I hot swap with a new 36GB?

Thank you for you help.
72GB-Error.jpg
0
 
LVL 46

Assisted Solution

by:noxcho
noxcho earned 400 total points
ID: 40597831
According to the error - it did not find the drive to restore to. Means the drive was not initialized (offline).

As for the 36GB drive over which you could restore - the failure can be in a bad sector which is excluded from I/O operations during restore.  Or the failure was not an important one, RAID controllers do report sometimes the failures which then disappear if you remove and reattach the drive.

As for the hot swap. As long as you have a backup - just replace it. In your situation it is hard to predict if it will accept it. Theoretically is should work.
0
 

Author Comment

by:RobertEhinger
ID: 40598033
So, how do I initialize the drive? Is that done through computer management?
0
 
LVL 46

Expert Comment

by:noxcho
ID: 40598039
Actually the initialization should have been done automatically.
I have some questions:
1)Is the 72GB drive of the same or bigger size as the drive which was imaged? Note, for successful backup it must be similar or bigger size. This is limitation of VHD backup. So in case you imaged in Windows 32GB drive (or RAID seen by Windows as single 32GB drive) then the drive over which you want to restore it must be also 32+GB drive.
2)If the drive is ok (similar or bigger) then another question. As you booted into Repair mode, loaded driver and detected the drive - did you format it? Note, drive must be empty and unallocated. If in commandline you create on this drive a partition and then deletes it (diskpart.exe) then the drive is initialized already.
0
 

Author Comment

by:RobertEhinger
ID: 40598172
It was my understanding that the drive would be formatted during to restoration of the image so, no I did not format it. I didn't do anything to the new drive except physically install it. The original drive was 36GB. The current used drive is 36GB. The new drive I tried to use is 72GB.
0
 
LVL 46

Expert Comment

by:noxcho
ID: 40598175
Is it a single drive or a part of RAID?
0
 

Author Comment

by:RobertEhinger
ID: 40598194
That is the $64 million question. I don't really know for sure. When this problem started the first drive failed and I could not boot the server into Server 2008R2. I am not sure how to tell if it is Part of RAID. I did not initially set this up so I am not sure what the original configuration was.
0
 
LVL 46

Assisted Solution

by:noxcho
noxcho earned 400 total points
ID: 40598207
But you can go into RAID Utility as server startup and see which drives are in RAID and what type of RAID you have. After POST screen press F8 and get to RAID utility. There you should be able to see what you have.
Or you should be able to see the RAID setup in HP SmartStart CD.

Have you already tried hot swap?
0
 

Author Comment

by:RobertEhinger
ID: 40598226
Have not tried hot-swapping yet. I will check the RAID utility later today.
0
 

Author Comment

by:RobertEhinger
ID: 40602967
OK, it looks like I have RAID 5. I am attaching a screen shot from what I saw when I went into the RAID utility this morning.  It also appear to me that only the 5 72GB Hard Disks are included and the 36GB must not be recognized possibly due to its failure. So that brings me back to the question about hot swapping and how to initialize a disk.
Array.jpg
0
 
LVL 46

Assisted Solution

by:noxcho
noxcho earned 400 total points
ID: 40602998
Take backup of your current configuration which as I understand is running quite ok.
Backup it using some third party backup tool like Shadow Protect or Paragon Hard Disk Manager. They are easier and more simple to use in comparison to Windows Server Backup.
Try to hot swap the drive. If it fails - rebuild the RAID with good drives and simply restore the image from a Boot CD of your backup software.
0
 

Author Comment

by:RobertEhinger
ID: 40609203
I will be back at the school Monday morning and will give the hot swap a try.
0
 
LVL 46

Expert Comment

by:noxcho
ID: 40609626
Ok.
0
 

Author Comment

by:RobertEhinger
ID: 40612852
I did the hot swap and everything kept running. I am sending a picture of the computer management screen taken right after installing the new hard drive.
drives.jpg
0
 
LVL 46

Expert Comment

by:noxcho
ID: 40613104
Looks good. How does the ACU tool see it? As part of the RAID?
0
 

Author Comment

by:RobertEhinger
ID: 40613294
So my question now is, with six 72GB HDD, why am I not seeing in excess of 400GB HDD space?
0
 
LVL 87

Assisted Solution

by:rindi
rindi earned 100 total points
ID: 40613311
Because if you use RAID 5, you loose the equivalent of 1 disk to parity. So your actual capacity is about 5x72GB (360 Total, unformated). Besides that, manufacturers advertise their disks to as having 1000 MB to make 1 GB, while for the OS 1024 MB's make 1 GB, so that also reduces the usable space.
0
 
LVL 46

Accepted Solution

by:
noxcho earned 400 total points
ID: 40613681
Yes, exactly as rindi said - you must forget about 1 disk and there is usually only about 68GB space from 72GB available. So all is shown correctly for you.
0
 

Author Comment

by:RobertEhinger
ID: 40614070
I was aware of the fact that the usable space is always smaller than advertised but I had forgotten about losing a disk to parity. Thanks for reminding me.
0

Featured Post

Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

Join & Write a Comment

Patch Management is administrated by page 129 of  the following document:     http://support.dell.com/support/edocs/software/smdmc/11/en/ug/ug.pdf However, there are issues with Patch Management with FTP (for instance, a slow Internet connectio…
Every server (virtual or physical) needs a console: and the console can be provided through hardware directly connected, software for remote connections, local connections, through a KVM, etc. This document explains the different types of consol…
It is a freely distributed piece of software for such tasks as photo retouching, image composition and image authoring. It works on many operating systems, in many languages.
This video shows how to remove a single email address from the Outlook 2010 Auto Suggestion memory. NOTE: For Outlook 2016 and 2013 perform the exact same steps. Open a new email: Click the New email button in Outlook. Start typing the address: …

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now