Solved

how to measure IOPS on a current server.

Posted on 2010-08-19
19
2,295 Views
Last Modified: 2012-05-10
I want to monitor the IOPS on average that a server is using so I can estimate my storage needs on a virtualization project. However when I used perfmon at home to test setting it up, I was getting IOPS reported of around 800 for Disk Writes/sec. This is for a 1 TB SATA drive at 7200 RPM, which should be more around 100 IOPS if I am not mistaken.

Am I doing something wrong or am I jsut getting bad readings on that one computer?
0
Comment
Question by:ryan80
  • 8
  • 8
  • 2
  • +1
19 Comments
 
LVL 11

Expert Comment

by:gikkel
ID: 33477500
Yeah, doesn't sound right...there is only 1 drive connected, correct?  Are you sure it isn't measuring Disk Write kb/Sec?  What OS are you using at home?  What are you running during data collection? If you have a default system performance template, use it - much easier...
Try setting up a new data collector set.  Set your sample time to 1 sec and duration to one minute.  Add physical disk, all instances     \PhysicalDisk(*)\*  
0
 
LVL 47

Expert Comment

by:dlethe
ID: 33478041
Any particular operating system?  If running windows, you can measure it from the O/S side via perfmon, which is a native windows program. Just enter command from DOS prompt, and it is easy to config with GUI.  

P.S. no way are you getting 800 IOPs from a SATA drive, so you were obviously measuring cached I/Os
0
 
LVL 12

Author Comment

by:ryan80
ID: 33478202
Never mind, it was a mistake in settings on IOmeter. I had it set to 0% random and it was gettting 300-400 IOPs. I changed it to 16K, 67% read, 100% random and now i am getting more realistic numbers. I am getting about 110 IOPs on the drive.

Thanks for the help
0
 
LVL 12

Author Comment

by:ryan80
ID: 33478294
Is there a bootable or portable program that I can use to test IOPs performance of a server array? I dont really feel like loading something on a server.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 33478916
IOMETER is a good tool, just load it on any system that has enough horsepower to generate the I/O, that is physically attached via same type & speed of connection, and you are good.

Remember, important .. the results are only as good as your configuration.   67% read, 16K 100% random is not even close to a real-world scenario.
0
 
LVL 12

Author Comment

by:ryan80
ID: 33479235
What would be a good config for a real world situation on a DB server?
0
 
LVL 30

Expert Comment

by:Duncan Meyers
ID: 33481041
In Perfmon, monitor Disk Writes/sec. and Disk Reads/sec. Multiply Writes/sec x write penalty (2 for RAID 1/0, 4 for RAID 5 and 6 for RAID 6, 1 for a single disk) and add that to Disk Reads/sec for total IOPS (aka Throughput). Divide the total IOPS by 80 for SATA drives and that gives you the number of drives you'll need for the workload.
0
 
LVL 30

Expert Comment

by:Duncan Meyers
ID: 33481094
Best to use IOmeter as an indicator of relative performance rather than absolute. That is; use it to create a performance baseline, then check to see if changes you make improve matters or make things worse.

As you've seen, the IOmeter workload makes an enormous difference to the performance figures you get, and to be completely frank about it, unless you really understand the underlying storage and the workload you are creating with it, the numbers it produces are meaningless. I work as a solutions architect for storage solutions and *nothing* frightens me more than a server administrator appearing with IOmeter stats. That's half a day gone right there...  :-)

Sorry if that sounds rude... On the other hand, we're here to help demystify storage and the results that IOmeter produces.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 33481130
Actually real load can be measured at the storage itself, but not with all makes/models.

IOMeter is a good tool for creating synthetic workloads, but you don't need a synthetic workload. You have a real, live one, so appropriate thing to do is measure using perfmon.  Even if you have external RAID, then perfmon is good, because it measures I/O at the LUN level.  

Note that under VMWare, you throw out a lot of rules, as it handles I/O at the VMWare differently then individual computers do, so keep in mind you will not be able to nail this, but get "good enough".

VMWare will read/write 64KB chunks at a time for each VM, but your RAID configuration needs to be set up to write in much larger blocks.
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 12

Author Comment

by:ryan80
ID: 33484797
I just wanted to test a server that I have that I am planning on using as a VMware host. It has 4 15K SAS drives that I was going to put in a RAID 10. I just wanted to get an idea of how many hosts I can put on the server based on its IOPS.


dlethe - why does the RAID need to be set higher than 64K blocks? I dont know enough about changing the black size for performance.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 33485181
The controller has to know how many blocks to read/write from each disk at a time.  As example,  let's say you have a 4-disk RAID 10, configured with block size of 256kb (so each disk read/writes 256KB at a time).

So if you are copying large files, it will scream.  Now make that your boot disk.  NTFS natively reads/writes 4KB at a time.   If you so much as OPEN up a a file, don't change a thing, but open it ... then NTFS has to update the last-access time in the file system.  This means that your RAID now has to write 512kb worth of data.   (As writes have to be the full block size, and it needs to be written to both disks).

By optimizing for throughput, you can see you KILL I/Os per second.   If you are doing piddly little 4KB reads/writes at a time from your O/S, but are tuning your raid for throughput benchmark performance, then you are hurting real-world performance.  
0
 
LVL 12

Author Comment

by:ryan80
ID: 33485259
Is there a place that would have common settings for real world scenarios?
0
 
LVL 47

Expert Comment

by:dlethe
ID: 33485562
VMWare reads/writes chunks of 64KB at a time from each virtual machine.  So in 4-disk RAID10, you want block size=64KB, stripe size = 2x64KB = 128KB.   This will provide optimal IO.  VMWare is effectively a random I/O engine, so setting larger blocksize will hurt overall performance.

Now will 4 disks give you enough IOPs?   Run perfmon on your computers now, and see how many IOPs they are asking for (if you have queue depth > 1 then that means your storage on the physical machine can't deliver what it needs anyway, so whatever number of IOPs it reports it obtains, is less than what it really needs.

It gets tricky on block size calculations. If your current environment is single drive, then you are using 4KB I/Os per second.  VMWare is going to R/W 64KB at a time to the RAID, and the RAID is going to be doing 128KB for every write request ... that is the nature of the beast, and why lots of RAM is good because your VM engine is going to do it's best to combine as many of those 4KB IOs into a 64KB chunk.

Sorry cant be more clear w/o writing a looong article, gotta run, but this should be enough. You know how to configure the RAID at least, will it be enough?  No matter how the IOPs line up, remember that throwing memory at your VM hardware will probably be helpful.  Since I/O is the limiting factor, then no matter what hardware you get, make sure you get a system that allows you to add more disk drives. Bottleneck is rarely CPU. Save money by buying fewer cores, and spend it on RAM & disk.  SAS is vital for performance.  Not only are all the benchmarks higher, but one big difference that most people are not aware of, is that SAS is dual channel.  You have a dedicated write AND dedicated read signal paths.  SATA is single-channel, so reads compete with writes.   Even if you had a 7200RPM SAS vs 7200RPM SATA, with same buffer size and all same specs, the SAS drive will significantly outperform SATA if you are mixing reads & writes.


0
 
LVL 12

Author Comment

by:ryan80
ID: 33485838
Thanks for the help. The servers that I have now are only used by 5 people at the most so they are not using a lot of performance. I looked at perfmon on the one of the current servers and it was only using about 20 read IOPs and 10 write. The other servers are going to be under similar load, but I am going to benchmark those as well. With a RAID 10 that means that it will be about 50 IOPs to the disks?

So with (4) 15K SAS 3.5 inch drives in RAID 10 I should get around 400 IOPS?

So 6 servers with about the same load should be good and leave room for some spikes in disk access correct?
0
 
LVL 47

Expert Comment

by:dlethe
ID: 33486093
It looks good .. but write IOPS is actually quite different ... since each disk has to service each write request, then write performance of RAID10 will always be 1/2 of the perfect-world READ performance.  Make sure your controller has a battery backup for the write cache.  This will insure data integrity in both power failure and drive failure.

Also you are going to be doing 64KB I/O requests.   perfmon is reporting 4KB IOPS.  So do not expect 400 64KB IOPS.  If you want to know what the disks will do with 64KB IOPS real-world, then you need to start benchmarking controller/drive combinations.  tomshardware.com has good benchmark info.     You just don't add up manufacturer's specs x # of disks to get IOPS in a RAID config.  That only works for JBOD.

Bottom line, however, with 20 read 10 write X 4, you should be fine even in this RAID10 config, as long as you have those SAS disks, write cache enabled, and battery backed up cache, and a decent controller.  no intel matrix fake-raid controllers.  A real SAS controller with onboard processor and cache.

0
 
LVL 12

Author Comment

by:ryan80
ID: 33486294
just so I am not mistaken on the IOPs of the RAID array let me run this by you. I thought for a Raid 10 array that you would only count the IOPs from 1/2 of the drives. Then you count the read IOPs from perfmon once and the write twice.

Is this correct, is it you count all of the drives and then read IOPs once and write 4 times?
0
 
LVL 47

Accepted Solution

by:
dlethe earned 500 total points
ID: 33487111
Apples and oranges.  Perfmon is reporting 4KB IOPs, the RAID needs to be configured for 64KB.  
Vast majority of controllers will reorder and distribute read requests to the disk that can service it first.  Also most controllers will do partial reads.  So if stripe is 128KB, and you only need 64KB that starts/ends at  block 0-63 on a selected disk .. then you can potentially get 100% efficiency on reads.   (THis is why partition alignment is important).    But on writes, most controllers require all disks to write that 64KB.

So that 1/2 % rule is basically ... oversimplified to the point where it is incorrect.
 
0
 
LVL 12

Author Comment

by:ryan80
ID: 33491370
I had perfmon open still on my home computer and it was doing a virus scan. It was getting an average of 300 IOPs over a 3 minute period. Is this just because it is linear and not random?
0
 
LVL 47

Expert Comment

by:dlethe
ID: 33517661
That is one factor, the other is simply that perfmon is measuring I/Os that are cached as well as non-cached.  In order to properly obtain the benefit/cost of caching vs non-cached I/Os, you need to set an appropriate baseline by measuring random I/Os over many GB, to insure that there are no cached I/Os.  Also your O/S is doing I/Os that are measured; I/O size can be 4KB and or much higher numbers, and perfmon does not differentiate.

Moral ... Benchmarking is easy to do wrong, hard to do right.
0

Featured Post

Complete Microsoft Windows PC® & Mac Backup

Backup and recovery solutions to protect all your PCs & Mac– on-premises or in remote locations. Acronis backs up entire PC or Mac with patented reliable disk imaging technology and you will be able to restore workstations to a new, dissimilar hardware in minutes.

Join & Write a Comment

Solid State (Hard) Drives aka SSD began to evolve in the computer industry recently. As the name suggests, there are no moving parts in the drives. The drive uses microchip memory store the data, as opposed to the spinning disks of a traditional HDD…
This article is an update and follow-up of my previous article:   Storage 101: common concepts in the IT enterprise storage This time, I expand on more frequently used storage concepts.
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now