Shell script / command to generate IO / IOPS load

Can anyone share any scripts / command to generate IO, ideally
one that I could control the amount of IOPS generated.

This is for Linux RHEL 6 & Solaris 10 x86

If need to run 2 times the command to get 2x the IOPS, do indicate

Ideally don't require to install any packages, just use the built-in commands / tools
sunhuxAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

arnoldCommented:
Are you looking for benchmarking/load testing tools?
sunhuxAuthor Commented:
I'm not working on Outlook nor AD.

Just test loading to see at what IOPS, we'll feel slowness in the servers.

Heavyload is one tool I've used for Windows but it's only for CPU & RAM
only, not IOPS
Carlos IjalbaIT Systems DirectorCommented:
Îf you don't want to install any software to test this, we need to know first a couple of things:

-Your servers are physical or virtual?
-local storage or attached storage (provided via a SAN, NAS, NFS, etc.)??

Because the ways to see IOPs are different in each case.
-For example, you can go to the storage cabinet and see IOPs performance in a LUN, that way you don't need to check on the server itself, and it is more reliable than a local OS.
-If they are VMs under VMware, you can check from VMware performace tab the IOPs in each.
OWASP: Threats Fundamentals

Learn the top ten threats that are present in modern web-application development and how to protect your business from them.

gheistCommented:
IOPS is rate not amount
dd if=/dev/zero of=whavever bs=4k oflags=sync count=10
should illustrate iops good enough...

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
sunhuxAuthor Commented:
What gheist gave is close to what's needed: perhaps I could
run the dd commands in background (ie dd ..... &) and this
will increase the IOPS

>Your servers are physical or virtual?
both

>local storage or attached storage (provided via a SAN, NAS, NFS, etc.)??
Mostly SAN
sunhuxAuthor Commented:
Is /dev/zero   the same as   /dev/null   in Linux?

In Windows, I've just tried to channel a huge Outlook pst file to nul & the IOPS shot up as shown in
Task Mgr's Resource Monitor:
   type myOutlook.pst > nul

Running multiple of the above "type ..." commands do not increase the IOPS further though, looks
like my 64bit laptop has hit the limit even with one single command
sunhuxAuthor Commented:
Looks like the limitation lies with "type" command: it appears to get the IO up to the
max of about 29Mbps only.

If I issued "copy myOutlook.pst  test1.pst", IOS go up to 250Mbps (for 1 copy) or
350Mbps (if 2 copy   commands are running concurrently)
arnoldCommented:
Usher you copy you have the reads and the write iOs.

Similar to network speed test, your IOPS are an average IOs during a period of time....

mkfile is one where you can test write io. Your cat file >/dev/null will have read IO.
The iostat tool can be used to collect the data and then used to perform the calculation.
gheistCommented:
/dev/zero is read only device that emits plain stream of zeroes. Probably it shares man page with /dev/null.
sunhuxAuthor Commented:
Does running multiple "dd ..." increases the IOPS ?


Carlos,  for Windows CPU & RAM utilization, I have HeavyLoad but not for disk IO;
    what are the tools out there for Linux (esp ESXi) if I'm installing one?
arnoldCommented:
The issue with Dd is that some io is read and someho is write depending on source and destination of what you are writing.
In the example gheist provided, the outputfile (of) will have 0's written to it, IO.

While you are running Dd, you will need to run/capture iostat data and you will get an average IO based on the interval of iostat output. And will only deal with DISK IO

IOPS stats can be computational in terms of IO on CPU/MEMORY to perform tasks.

Look at CPU, Some include the IOPS rating on their spec.

Meaning you have to define which types of IO you are trying to calculate IOPS for.
Disk reads, disk writes, computational.....
Total system performance for some tasks which is a combination of the multiple components.
gheistCommented:
running multiple dd will not increase IOPS.
say rotational disk can make one write per revolution.
NCQ/TCQ may or may not queue and reorder  those, but with SYNC it writes directly to disk platter, so it measures physical media bypassing caches.
Next technique employed by Linux is write clustering, if it sees two requests to neighbouring sectors it does just one bigger write.
Also most OS-ses do disk request sorting in case queuing does not do it;s job at minimizing disk head moves.
Carlos IjalbaIT Systems DirectorCommented:
If you want to check ANY performance parameter on vSphere, on a ESXi host, or on VMs, then you need to check from the performance tabs of vCenter, not on the OS itself, as vSphere has performance improvements on the virtual HW like memory dedup, ballooning, thin storage, etc, which won't show up on the OS measurements.

Therefore, you don't need to install anything.

You can always use performance monitor on windows and topas, sar, etc on Linux to check the local OS response. But as I say, better off looking from vCenter.
gheistCommented:
memory deduplication is disabled in vmware for a year now via security patch, and it was ineffective since XP SP2 came out.
sunhuxAuthor Commented:
> dd if=/dev/zero of=whavever bs=4k oflags=sync count=10
Ok, running multiple dd's is not going to increase the IOPS;  is there any way at all
to increase IOPS, say by changing the blocksize (presume it's "bs" in the above) ?
Any other tools/commands (eg: cp, tar will give higher IOPS ) ?
arnoldCommented:
IOPS is a computed value based on IO during a time frame.

What are you trying to benchmark?
Look at lbs.sourceforge.net
See if that provides the info you are after.
sunhuxAuthor Commented:
UnixBench which has a function to gen IO activity is not there at the
URL anymore.

IOZone : appears that after installing, it only gives 1 option ie "Uninstal" & 
can't launch it
gheistCommented:
Question is exclusively about UNIX. If you want to install windows benchmarks ask another question.
Carlos IjalbaIT Systems DirectorCommented:
If you really want to check from ESXi, you can use iometer, i/o analyzer, and esxtop /this is by far the easiest, as you don't have to install anything).

For detailed info on how to use esxtop, look at Duncan Epping Webpage:

http://www.yellow-bricks.com/esxtop/

But basically, run esxtop, and press "d" for disk performance details, then run the dd command inside the VM and you're ready to go.
gheistCommented:
Those add to latency of virtual disk:
1. enabling snapshots (not even taking them)
2. Having dynamic VMDK, or any other sort of thin/delayed provisioning
3. Having any sort of IO barrier inside guest
4. Having too many VMs per rotational disk (remember - 1 write per revolution, you need 30IOPs at storage per VM, which may be just 15 IOPs inside VM)
petersystems engineerCommented:
I know this is an old thread, but I was researching this at this time as well and found this site which has a script to create the iops metrics, hope it helps
https://benjamin-schweizer.de/measuring-disk-io-performance.html
script is here:
https://benjamin-schweizer.de/files/iops/
gheistCommented:
Buffered read has nothing to do with the disk.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.