Solved

slow server after raid crash fix.

Posted on 2013-11-13
17
962 Views
Last Modified: 2013-12-10
Hi.
Have a HP ML350 G6 server With SBS 2011 installed.
Server have a Raid 5 With 6 drives.
Found one disk failled an 2 more predictive failure (!)
Hot swaped one disk at the time starting With the failed and leting it rebuild over night before swaping the Next.
After that it booted fine and worked well for a day.
After a day it became very slow. And would not boot until we change the cach battery..
Now, server is up but is very slow.
In aplication log it gives event id 823 (want m to run dbcc checkdb on sbsmonitoring.mdf)

What is the best stratigy to find what's wrong and to fix this.
Backup have been runing but gues it's something wrong\ files corrupted during the unstable disk situation that will be on the backups as well..

H
0
Comment
Question by:Tore Jacobsen
  • 6
  • 5
  • 5
17 Comments
 
LVL 5

Assisted Solution

by:warddhooghe
warddhooghe earned 334 total points
Comment Utility
check the array rebuild status, might still be rebuilding, or a failure.
in any case, while running in a redundant state, things are much slower. might be that one of your 2 predictive failures is already failing.
0
 
LVL 47

Assisted Solution

by:dlethe
dlethe earned 166 total points
Comment Utility
It is highly likely the array is still rebuilding.  This may take days. I'd ordinarily say just to do nothing, but in your case, take a full backup while you still can and consider replacing the other drives as well.
0
 
LVL 1

Author Comment

by:Tore Jacobsen
Comment Utility
Hi.
Guess I was not Clear regarding the drives. All 3 has been replaced. First the failed and then the predictive ones. One day apart so the raid could rebuild inbetween.
Have checked that the raid is fully rebuild (Hp Raid config utility)
0
 
LVL 5

Expert Comment

by:warddhooghe
Comment Utility
as long as the rebuild is 100% I very much doubt the slowdown is caused by these failures.
you might want to run some diags, check disc queue length, fragmentation, etc.

I also recommend upgrading all drivers and firmwares, since you have HP it can be done very easy, just download and install the latest HP SPP. Ofcourse not to be done during office hours. Will need reboot and best to execute on the ILO or server concole, etc.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
The slowdown is also further slowed down by the certainty that you have a large number of recoverable read errors.  Depending on the make/model of drive, it can take up to 10 seconds to get just one unreadable block. Those SMART errors on several drives indicate you have a statistical certainty that surviving disks have these errors.

By any chance are these consumer disks, or disks w/o the HP firmware?  If so, then you are destined to have more of the same problems.
0
 
LVL 1

Author Comment

by:Tore Jacobsen
Comment Utility
I am installing the SP for Proliant now. Takes hours (sloooow server)
Drives are HP.
After SPP is Complete, plan to restart, but assume it will be just as slow..
Any other suggestions?
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
goto the support.hp.com site and download HP's ACU program and look at the event log to see what is happening.

There is no reason to have to guess when HP spends millions developing a program that tells you exactly what is going on.
0
 
LVL 1

Author Comment

by:Tore Jacobsen
Comment Utility
Needed to run chkdsk -r
Was working fine after that
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 1

Author Comment

by:Tore Jacobsen
Comment Utility
I've requested that this question be closed as follows:

Accepted answer: 0 points for TelehusetMoss's comment #a39695601

for the following reason:

found solution
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
So what was the solution, especially in light of the fact that the array was rebuilding at the time as I pointed out, so of course it will be slow until it completes.
0
 
LVL 5

Expert Comment

by:warddhooghe
Comment Utility
Some credit in directing you to the solution?
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
Agreed, points should be awarded equally, unless the problem is still there and the array is no longer rebuilding.
0
 
LVL 5

Expert Comment

by:warddhooghe
Comment Utility
3) http:#39644852 and http:#39644930
Both comments advised running various diagnostics, which in the end fixed the issue as the owner claimed "chkdsk -r" was the solution.
0
 
LVL 47

Expert Comment

by:dlethe
Comment Utility
The chkdsk would have made no difference. The rraid was already doing a full media repair.  Time too complette the repairr was the only thong necessRy. Has chkdsk note have been run, it. would haave completed sooner.
0
 
LVL 5

Accepted Solution

by:
warddhooghe earned 334 total points
Comment Utility
Agreed that the solution could have been "just wait for the rebuild/repair", But still, a server without the ACU tool or without PSP/SPP is just totally nuts. I'm sure the owner learned something from us on how to diagnose his raid and some points would be appreciated.
0
 
LVL 1

Author Closing Comment

by:Tore Jacobsen
Comment Utility
Raid was fully rebuilt.
Only after chkdsk /r was it usable.
0

Featured Post

Too many email signature updates to deal with?

Do you feel like you are taking up all of your time constantly visiting users’ desks to make changes to email signatures? Wish you could manage all signatures from one central location, easily design them and deploy them quickly to users? Well, there is an easy way!

Join & Write a Comment

You may have discovered the 'Compatibility View Settings' workaround for making your SBS 2008 Remote Web Workplace 'connect to a computer' section stops 'working around' after a Windows 10 client upgrade.  That can be fixed so it 'works around' agai…
You might have come across a situation when you have Exchange 2013 server in two different sites (Production and DR). After adding the Database copy in ECP console it displays Database copy status unknown for the DR exchange server. Issue is strange…
This tutorial will walk an individual through the steps necessary to join and promote the first Windows Server 2012 domain controller into an Active Directory environment running on Windows Server 2008. Determine the location of the FSMO roles by lo…
This tutorial will show how to configure a single USB drive with a separate folder for each day of the week. This will allow each of the backups to be kept separate preventing the previous day’s backup from being overwritten. The USB drive must be s…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now