How do you calculate IOP's needed in your environment?

Hi, I am trying to find the best way to calculate IOP's required in my environment, especially as I prepare to go from P2V in the very near future.

So, what tool(s), techniques, etc. do you use to accurately or pretty good estimations of IOP's your environments required as you set the stage for required equipment for your virtual environment storage pools, etc.?

Virtual Environment Platforms
VMware ESXi vSphere 6.5
Microsoft Hyper-V (2012 R2 & 2016)

Thanks in advance.
rsnellmanIT ManagerAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
so you have physical workloads at present ?

and want to know what IOPS they produce, or you want to look at the current workloads and see what IOPS they are creating - Performance Monitor.
Bill BachPresident and Btrieve GuruCommented:
As you move from physical to virtual, though, you will run into a horrible truth: I/O operations per second are not a very good metric!  For many workloads (especially database workloads), the actual latency of getting blocks to and from the disk will far outweigh the actual throughput or operations per second.  Further, you can trade memory for disk I/O's by caching more data, eliminating the need for I/O operations in the first place.  

Having said that, here is a good starting point: Grab an I/O test tool, such as SQLIO or DiskSpd from Microsoft.  Now, set up an I/O benchmark that is closest to your "real" workload.  For example, with SQLIO, you can use something like this:
    sqlio -kW -s30 -frandom -o8 -b64 -LS -Fparam.txt
This will do a system write test for 30 seconds, writing randomly using 8 concurrent threads and 64KB blocks.  Run this test and check your current server's metrics.  Now, experiment a bit -- change it to 4K blocks instead of 64K, and see how the totals change.  Is the server any different?  No, the workload is different.

Anyway, if you can get to something that looks like your "real" work, then you have solid numbers as a starting point.  Now, run the same test on the NEW server and see where it comes in.  It may be vastly faster or slower -- but this gives you a RELATIVE performance metric to work with.  You can now estimate how disk operations on the new environment will work.  Be sure to note the latency numbers provided, as well.  As you move from physical to virtual, you will see a 30% increase in latency (due to the virtualization layer).  Don't let someone tell you that this doesn't matter, because for database or other sequential workloads, this has a profound impact on overall system performance.  It means less on a file server environment, where massive disk reads are the norm.  The problem gets MUCH worse if you move to a iSCSI SAN, when there are no HBA's to cache disk requests.  (I've saw one company move from a server that was pumping out 2600 MB/s of disk write throughput to a "new" server that maxxed out at 300 MB/s -- all because of the iSCSI latency.  Needless to say, performance was not as good as their old server!)

Of course, that is NOT the entire picture.  If you are adding memory, then you'll have more RAM on the server to cache data, and therefore eliminate the need to even do I/O operations in the first place!  As such, having lower throughput numbers doesn't mean that the system will be slower -- it could just mean that you have to optimize memory and caching instead.

Once you have good metrics, you then augment this information with the data from Performance Monitor.  It's not a bad idea to monitor each physical volume and look at things like Disk Read/Second, Disk Read Bytes/Second (and the Disk Writes as well) and other values.  Remember that if you are migrating several physical servers to a single VM Host, you need to add up the numbers to get an expected throughput requirement.  Remember, though, that latency changes the calculations by a LOT, so adding 30% might be in order.  Or, it could be that SUBTRACTING 30% may be in order -- because the higher latency may mean that you are simply not able to issue as many requests within a given time period any more!  Again, each metric you look at will show you only one part of the picture -- but actually deploying the workload in a test environment will give you a lot more data to work with.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Philip ElderTechnical Architect - HA/Compute/StorageCommented:
As mentioned, PerfMon (Performance Monitor) can be set up to examine and log the metrics on an existing system.
 + Write sizes
 + Read sizes
 + Throughput
 + Latency

For testing new storage setups we use IOmeter to start with.

We run a gamut of tests from 100% Read/0% Write to 0% Read/100%Write in 25% increments each way. In those increments are also sub-increments for thread count, read & write sizes, and disk queue depth.

Some posts I've done on the subject:
 + Storage Configuration: Know your Workloads for IOPS or Throughput
 + Hyper-V Virtualization 101: Hardware Considerations
 + Hyper-V: Disk Queue Length Can Kill Everything
 + Hyper-V Performance: Appropriate Disk Queue Length
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.