Solved

Disk I/O capacity

Posted on 2013-11-20
7
684 Views
Last Modified: 2013-11-20
We are experiencing some poor performance on our web application during a small stress test.  We would like to determine the IOPS consumption of the application so we can evaluate the storage system.   It is a Red Hat Linux/Oracle server running in VMware ESXi 4.1.  What tools can I use to capture the necessary data needed to analyze ?
0
Comment
Question by:NytroZ
  • 2
  • 2
  • 2
  • +1
7 Comments
 
LVL 40

Accepted Solution

by:
coolsport00 earned 250 total points
ID: 39664213
If you are thinking it is simply disk latency the VM resides on based on disk in & of itself (i.e. a misconfigured datastore for RAID, etc.), you can SSH into the Host the VM resides on, type "esxtop" (no quotes), then press 'v' & view the DAVG statistic for the VM. If it is above 10-12ms, your underlying storage is having issues. You need either more spindles in the RAID, a different RAID, or different disk types (i.e. SAS or SSD rather than potentially SATA).

~coolsport00
0
 
LVL 118
ID: 39664228
0
 

Author Comment

by:NytroZ
ID: 39664236
The current setup is 4 10K SAS disks in a RAID 10.  I was told this can do about 300 IOPS/sec.  Looking at the performance tab on the vSphere client it shows the IOPS around 1000 during a load.  Is this accurate?  If the disks are only capable of 300 IOPS how is it recording 1000 IOPS?  I will try the extop command.
0
IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 118
ID: 39664268
Doing the calculations, approx 400 IOPs.
0
 

Author Comment

by:NytroZ
ID: 39664304
The esxtop command showed me the reads/writes per second at it verified what the performance chart showed me.  The IOPS are around 1000 and disk latency is 4ms.  this runs for a bit but eventually the database comes back with TNS listener timeout error.  Would it be safe to say that the disks are being overloaded even though latency is still low?
0
 
LVL 30

Assisted Solution

by:Duncan Meyers
Duncan Meyers earned 250 total points
ID: 39664314
When you're preparing sizing, each 10K drive can handle 140 IOPS average. They'll peak to about 2.5 x that, so 350 IOPS per drive absolute maximum. Beyond that, response time will go through the roof
Your RAID 1/0 set should be able to produce 1400 read IOPS peak assuming your RAID array is smart enough to distribute reads across all 4 drives. For writes, it'll handle a maximum of 700 host write operations. You haven't described what the stress test is and what the I/O distribution is, but I'd say that at 1000 IOPS, you're at the limits of what 4 drives can provide so you need to add more drives. Depending on what your shared storage is, you could add more drives or add SSDs to improve performance.

Incidentally, beware of chasing benchmark numbers. What's important is the user experience, not what the load generating or benchmarking software says. If you have 10 concurrent users hitting your web app, then I'd imagine performance of your existing configuration is fine. If, on the other hand, you're getting 10,000 concurrent users, well, you're in the poo and you need more disk
0
 
LVL 30

Expert Comment

by:Duncan Meyers
ID: 39664608
Thanks! Glad I could help.
0

Featured Post

Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Join & Write a Comment

Suggested Solutions

Last article we focus in how to VMware: How to create and use VMs TAGs – Part 1 so before follow this article and perform the next tasks, you should read the first article how to create the TAG before using them in Veeam Backup Jobs.
In this article, I show you step by step with screenshots to assist you - HOW TO: Deploy and Install the VMware vCenter Server Appliance 6.5 (VCSA 6.5), with some helpful tips along the way.
Teach the user how to install and configure the vCenter Orchestrator virtual appliance Open vSphere Web Client: Deploy vCenter Orchestrator virtual appliance OVA file: Verify vCenter Orchestrator virtual appliance boots successfully: Connect to the …
This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now