Solved

vsphere disk latencies server down

Posted on 2011-09-26
10
1,055 Views
Last Modified: 2012-05-12
every few weeks our vsphere 5 server goes on the run.
all applications time out, vsphere client shows that disk latencies on all services go sky-high (see screenshot).
my only solution is a reboot of the physical server and restart each virtual server.
unix servers usually need a fsck in single user mode to get rid of errors.
what could be cause of these extreme latencies ?

we use vsphere 5 on a raid-5 of 6 x 500gb local sata disks.

vsphere-latencies.jpg
0
Comment
Question by:Lieven Embrechts
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 4
10 Comments
 
LVL 121
ID: 36600973
Are the 6 SATA disks 7,200 RPM?

6 SATA disks, RAID 5 - is going to have performance issues.

How much ram in the HOST server?

How much ram is being used by guests?
0
 
LVL 9

Author Comment

by:Lieven Embrechts
ID: 36601001
yes, 2.5 inch 7200rpm.

enough ram in server (12 of 24gb used), guests look ok for ram usage when server runs normal.

i'm not an expert on vsphere but it feels like the IO queue is overloaded somehow, and when that happens the complete vsphere environment goes on the run ...
0
 
LVL 121
ID: 36601043
It's the 7,200rpm SATA disks.

What's the server and RAID controller?
0
NFR key for Veeam Backup for Microsoft Office 365

Veeam is happy to provide a free NFR license (for 1 year, up to 10 users). This license allows for the non‑production use of Veeam Backup for Microsoft Office 365 in your home lab without any feature limitations.

 
LVL 9

Author Comment

by:Lieven Embrechts
ID: 36601551
cisco ucs c210 m2 server
lsi 6g megaraid 9261-8i - 512wc card
0
 
LVL 121
ID: 36601603
Have you got the RAID cache set to 75% Write 25% Read?

On a production machine I would not use 7,200rpm disks, the IOPs would be very poor, and worse in a RAID5 configuration.
0
 
LVL 9

Author Comment

by:Lieven Embrechts
ID: 36601665
>>Have you got the RAID cache set to 75% Write 25% Read?

if it's not default, why would i do that, where would i do that ?

>>On a production machine I would not use 7,200rpm disks, the IOPs would be very poor, and worse in a RAID5 configuration.

consultant used raid 6 i believe, he told me all IO would be distributed over the disks so better performance with raid 5 or 6.
what would be your suggestion for disks and raid-config ?

is the graphic you see (see screenshot) something seen before ? reason ?
0
 
LVL 121
ID: 36601684
SATA disks are very slow, poor performance, should only be used for Labs, Research, or Archives/Backup.

Your issues, is your poor SATA disks.

RAID 10 is faster for read and writes.

RAID Cache Configuration can be changed in the RAID BIOS.
0
 
LVL 42

Expert Comment

by:paulsolov
ID: 36646016
I would agree with hanccocka, SATA drive can pose problems.  Take a look at what is happening on the guest side such as a SQL server doing maintenance work on a database.  I would also ensure that the 75% write, by default I think the UCM servers are 50/50 which can slow down the write performance and as you can see it's thw write latency you're having an issue with.  SAS drives would be preferred.
0
 
LVL 121

Accepted Solution

by:
Andrew Hancock (VMware vExpert / EE MVE^2) earned 500 total points
ID: 36647581
0
 
LVL 9

Author Comment

by:Lieven Embrechts
ID: 36895834
had a look at the cache settins several were referring to:
apparently the lsi controller is by default full write cache, there is no read cache.
as for the sata disks, i don't see a problem with light apps & light load.

write latencies go up all of a sudden every 2 days at random time,
i need a method to find what is causing it.
0

Featured Post

Simple, centralized multimedia control

Watch and learn to see how ATEN provided an easy and effective way for three jointly-owned pubs to control the 60 televisions located across their three venues utilizing the ATEN Control System, Modular Matrix Switch and HDBaseT extenders.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

HOW TO: Upload an ISO image to a VMware datastore for use with VMware vSphere Hypervisor 6.5 (ESXi 6.5) using the vSphere Host Client, and checking its MD5 checksum signature is correct.  It's a good idea to compare checksums, because many installat…
In this step by step tutorial with screenshots, we will show you HOW TO: Enable SSH Remote Access on a VMware vSphere Hypervisor 6.5 (ESXi 6.5). This is important if you need to enable SSH remote access for additional troubleshooting of the ESXi hos…
Teach the user how to edit .vmx files to add advanced configuration options Open vSphere Web Client: Edit Settings for a VM: Choose VM Options -> Advanced: Add Configuration Parameters:
Teach the user how to use configure the vCenter Server storage filters Open vSphere Web Client:  Navigate to vCenter Server Advanced Settings: Add the four vCenter Server storage filters: Review the advanced settings: Modify the values of the four v…

729 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question