Solved

slow server after raid crash fix.

Posted on 2013-11-13
17
1,077 Views
Last Modified: 2016-12-08
Hi.
Have a HP ML350 G6 server With SBS 2011 installed.
Server have a Raid 5 With 6 drives.
Found one disk failled an 2 more predictive failure (!)
Hot swaped one disk at the time starting With the failed and leting it rebuild over night before swaping the Next.
After that it booted fine and worked well for a day.
After a day it became very slow. And would not boot until we change the cach battery..
Now, server is up but is very slow.
In aplication log it gives event id 823 (want m to run dbcc checkdb on sbsmonitoring.mdf)

What is the best stratigy to find what's wrong and to fix this.
Backup have been runing but gues it's something wrong\ files corrupted during the unstable disk situation that will be on the backups as well..

H
0
Comment
Question by:Tore Jacobsen
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 5
  • 5
17 Comments
 
LVL 5

Assisted Solution

by:warddhooghe
warddhooghe earned 334 total points
ID: 39644625
check the array rebuild status, might still be rebuilding, or a failure.
in any case, while running in a redundant state, things are much slower. might be that one of your 2 predictive failures is already failing.
0
 
LVL 47

Assisted Solution

by:David
David earned 166 total points
ID: 39644640
It is highly likely the array is still rebuilding.  This may take days. I'd ordinarily say just to do nothing, but in your case, take a full backup while you still can and consider replacing the other drives as well.
0
 
LVL 1

Author Comment

by:Tore Jacobsen
ID: 39644755
Hi.
Guess I was not Clear regarding the drives. All 3 has been replaced. First the failed and then the predictive ones. One day apart so the raid could rebuild inbetween.
Have checked that the raid is fully rebuild (Hp Raid config utility)
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 5

Expert Comment

by:warddhooghe
ID: 39644852
as long as the rebuild is 100% I very much doubt the slowdown is caused by these failures.
you might want to run some diags, check disc queue length, fragmentation, etc.

I also recommend upgrading all drivers and firmwares, since you have HP it can be done very easy, just download and install the latest HP SPP. Ofcourse not to be done during office hours. Will need reboot and best to execute on the ILO or server concole, etc.
0
 
LVL 47

Expert Comment

by:David
ID: 39644930
The slowdown is also further slowed down by the certainty that you have a large number of recoverable read errors.  Depending on the make/model of drive, it can take up to 10 seconds to get just one unreadable block. Those SMART errors on several drives indicate you have a statistical certainty that surviving disks have these errors.

By any chance are these consumer disks, or disks w/o the HP firmware?  If so, then you are destined to have more of the same problems.
0
 
LVL 1

Author Comment

by:Tore Jacobsen
ID: 39645624
I am installing the SP for Proliant now. Takes hours (sloooow server)
Drives are HP.
After SPP is Complete, plan to restart, but assume it will be just as slow..
Any other suggestions?
0
 
LVL 47

Expert Comment

by:David
ID: 39645644
goto the support.hp.com site and download HP's ACU program and look at the event log to see what is happening.

There is no reason to have to guess when HP spends millions developing a program that tells you exactly what is going on.
0
 
LVL 1

Author Comment

by:Tore Jacobsen
ID: 39695601
Needed to run chkdsk -r
Was working fine after that
0
 
LVL 1

Author Comment

by:Tore Jacobsen
ID: 39695824
I've requested that this question be closed as follows:

Accepted answer: 0 points for TelehusetMoss's comment #a39695601

for the following reason:

found solution
0
 
LVL 47

Expert Comment

by:David
ID: 39695825
So what was the solution, especially in light of the fact that the array was rebuilding at the time as I pointed out, so of course it will be slow until it completes.
0
 
LVL 5

Expert Comment

by:warddhooghe
ID: 39696028
Some credit in directing you to the solution?
0
 
LVL 47

Expert Comment

by:David
ID: 39696055
Agreed, points should be awarded equally, unless the problem is still there and the array is no longer rebuilding.
0
 
LVL 5

Expert Comment

by:warddhooghe
ID: 39702400
3) http:#39644852 and http:#39644930
Both comments advised running various diagnostics, which in the end fixed the issue as the owner claimed "chkdsk -r" was the solution.
0
 
LVL 47

Expert Comment

by:David
ID: 39703174
The chkdsk would have made no difference. The rraid was already doing a full media repair.  Time too complette the repairr was the only thong necessRy. Has chkdsk note have been run, it. would haave completed sooner.
0
 
LVL 5

Accepted Solution

by:
warddhooghe earned 334 total points
ID: 39703720
Agreed that the solution could have been "just wait for the rebuild/repair", But still, a server without the ACU tool or without PSP/SPP is just totally nuts. I'm sure the owner learned something from us on how to diagnose his raid and some points would be appreciated.
0
 
LVL 1

Author Closing Comment

by:Tore Jacobsen
ID: 39708594
Raid was fully rebuilt.
Only after chkdsk /r was it usable.
0

Featured Post

Optimize your web performance

What's in the eBook?
- Full list of reasons for poor performance
- Ultimate measures to speed things up
- Primary web monitoring types
- KPIs you should be monitoring in order to increase your ROI

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I was prompted to write this article after the recent World-Wide Ransomware outbreak. For years now, System Administrators around the world have used the excuse of "Waiting a Bit" before applying Security Patch Updates. This type of reasoning to me …
Resolving an irritating Remote Desktop connection that stops your saved credentials from being used.
This tutorial will show how to configure a single USB drive with a separate folder for each day of the week. This will allow each of the backups to be kept separate preventing the previous day’s backup from being overwritten. The USB drive must be s…
This Micro Tutorial hows how you can integrate  Mac OSX to a Windows Active Directory Domain. Apple has made it easy to allow users to bind their macs to a windows domain with relative ease. The following video show how to bind OSX Mavericks to …
Suggested Courses

630 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question