VMWare ESXi v4.1.0 hardware sensors on RAID 10 rebuild on a Dell PowerEdge 2950 with a Perc 5i RAID controller.

I'm running VMWare ESXi v4.1.0 on a Dell PowerEdge 2950 with a Perc 5i RAID controller.  Member HDD 5 failed on a RAID 10 array.  I put in a new SATA drive and it started to rebuild.  I'm monitoring via the hardware sensors on the VMWare vSphere client.  It still shows it rebuilding.  I refreshed the sensors but have not reset all sensors yet.  It seems that it's still rebuilding for disk I/O is slower than normal.  Each member (4 members) is 1 TB.  It's been a little over 24 hours now...  Is it possible or seem normal that the rebuild is still going on?  Do the VMWare ESXi hardware sensors refresh and update automatically?  Also, I have a DRAC board but it just shows the drive being placed into the server.  It doesn't show rebuild status.

Now,

I just checked the VMWare hardware sensors and I don't see rebuild anymore... I see a parity check status.  Does the parity check come after the rebuild status?
gopher_49Asked:
Who is Participating?
 
Andrew Hancock (VMware vExpert / EE MVE^2)Connect With a Mentor VMware and Virtualization ConsultantCommented:
RAID rebuild can take some time to rebuild the RAID array. Yes, after rebuild, comes parity checking.

Are you using the Dell version of ESXi, and you would be better checking using Dell OpenManage, here's my EE article, of how to install on ESXi.

HOW TO: Use Dell OpenManage Server Administrator to Create and Manage a Virtual Disk (RAID Array) on a Dell PowerEdge Server with VMware vSphere 5.0, ESXi 5.0 installed.
0
 
SandyCommented:
yes after rebuild.

Cheers,
SA
0
 
gopher_49Author Commented:
Great.  Sounds like the sensors are showing good results.  I'm using the Dell ESXi v4.1 image.  I recently found another article in regards to installing Dell OpenManage Server.  I plan to install it in my lab and if all goes well I'll do it in production.  

Thanks!
0
Improve Your Query Performance Tuning

In this FREE six-day email course, you'll learn from Janis Griffin, Database Performance Evangelist. She'll teach 12 steps that you can use to optimize your queries as much as possible and see measurable results in your work. Get started today!

 
gopher_49Author Commented:
It's still on the parity check and we're taking a huge performance hit.  I'm surprised we're taking such a performance hit during the rebuild of the array...
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Any RAID rebuild, affects performance, until the procedure is completed.
0
 
gopher_49Author Commented:
It's been going on for 2 days now... Horrible...  I'll open up a different ticket in regards to selecting a different RAID controller / drive selection for my next install to help reduce the impact during a RAID 10 rebuild...
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
RAID rebuild depends on size, speed of disk, type of disk, number of disks, controller, and during this time of rebuild, you are also at high risk of another disk failure, if all disks were purchased at a similar time.

think about RAID types, hot and cold spares, and more importantly have good valid backups.
0
 
gopher_49Author Commented:
All of my backups are good and saved to another mirror/appliance and also offsite.  I take application level backups and VMWare snapshots once a week.  So, we're good there, however, in the future I want a stronger RAID controller that doesn't take such a hit... I would also like drives to rebuild quicker... This is just too long of a rebuild and too much of a performance hit.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
I hope you do not use VMware Snapshots as Backups!?

see my EE Article

HOW TO: VMware Snapshots :- Be Patient

Snapshots are not backups, and if you have VMs running on snapshot disks, there is a performance overhead, and VMs can struggle.
0
 
gopher_49Author Commented:
I use application level backups 6 days a week.  Once a week I shoot off a VMWare backup using the VMWare backup/storage API.  I should of not used the word 'snapshot'.  But.. Both the application level backup and the VMWare backup save to a different appliance that is a mirror.  They are also sent offsite to another RAID 10 array.  My backup design is solid.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Just make sure you test your "Backup Design is solid".
0
 
gopher_49Author Commented:
I normally do with VEEAM Sure Backup...  This environment doesn't not have that... I do randomly download SQL and various documents to make sure they restore correctly...  Sometimes I even mount their Exchange in a recovery group.
0
 
gopher_49Author Commented:
The rebuild and parity check is complete, however, I'm having serious disk I/O performance issues still.  My exchange server event log even mentioned the below...  Users say every application accessing the file is system extremely slow...  Of course this makes sense since all VMs are sharing the same RAID 10 array.. But...  Something is wrong for sure.  The array controller is not reporting anything via the hardware sensors in ESXi - vSphere client..  Any idea?  Should I install Dell OpenManager?

Event Type:      Warning
Event Source:      ESE
Event Category:      Performance
Event ID:      507
Date:            6/9/2013
Time:            9:01:25 AM
User:            N/A
Computer:      BACKEND1
Description:
Information Store (2880) A request to read from the file "\BACKEND1\$dd623858-6171-4a2e-bec0-79ef2e870fcb\$CopySrc000000005564EC3A$" at offset 1441792 (0x0000000000160000) for 65536 (0x00010000) bytes succeeded, but took an abnormally long time (70 seconds) to be serviced by the OS. This problem is likely due to faulty hardware. Please contact your hardware vendor for further assistance diagnosing the problem.

For more information, click http://www.microsoft.com/contentredirect.asp.
0
 
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Check the VM is not on a snapshot disk?
0
 
gopher_49Author Commented:
It might be right this second for its being backed up..  But...  These symptoms existed when backup where not running.  I'll see if the snapshot is being removed when its done.  All VMs are having performance issues.  I'll check the snapshots.
0
 
gopher_49Author Commented:
I just checked and snapshots where not present.  I'll see what users say in regards to performance tomorrow when they log on..  I'm working remote so maybe the performance I'm noticing aren't as bad as they truly are.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.