To keep pace with competitors, businesses must keep employees productive, and that means providing them with the latest technology. This document provides the tips and tricks you need to help you migrate an outdated PC fleet to new desktops, laptops, and tablets.
COURSE of the MONTH
7 days, 23 hours left to enrollBecome a Premium Member and unlock a new, free course in leading technologies each month.
Solved
OS IOPS value Vs Storage IOPS value
Posted on 2012-12-27
How do we differentiate the IOPS value from OS to Storage. For an example I'm writing 8K block of data using dd command in linux
dd if=/dev/zero of=/opt/rg/perf/testing123
245760000000 bytes (246 GB) copied, 334.014 s, 736 MB/s
IOPS value is 30000000/334= 89820
but in Storage I'm only seeing around 2000 IOPS. IF I understand correctly it converts to different block size, and then do the calculation?
dd if=/dev/zero of=/opt/rg/perf/testing123
245760000000 bytes (246 GB) copied, 334.014 s, 736 MB/s
IOPS value is 30000000/334= 89820
but in Storage I'm only seeing around 2000 IOPS. IF I understand correctly it converts to different block size, and then do the calculation?
[X]
Welcome to Experts Exchange
Add your voice to the tech community where 5M+ people just like you are talking about what matters.
- Help others & share knowledge
- Earn cash & points
- Learn & ask questions
3-
3
7 Comments
Active 6 days ago
dd does NOT write data to the HDD in 8KB I/O chunks, or for that matter, the way you are running it, it won't even write only 8KB x 30000000 bytes of data.
Your filesystem, kernel, and device drivers are doing a heck of a lot of things underneath the covers such as dealing with journaling, aggregating I/O, and potentially doing some compression depending on some settings.
It is even queuing I/O, so if you kill the dd, you will note that the HDD still blinks for a while after dd is killed.
If you want to measure how fast data gets to the disk, with reasonable accuracy, you must do a dd but set the of= to the raw physical device (which destroys file system). If you want to insure that the block sizes are correct and when you do a dd to write chunks in 8KB and exactly 8KB of data gets written and nothing else, then you have to write a program that uses the pass-through /dev/sg driver and send out the WRITE(10) and/or WRITE(16) CDBs. The raw SCSI method is the ONLY accurate means of having complete control.
Your filesystem, kernel, and device drivers are doing a heck of a lot of things underneath the covers such as dealing with journaling, aggregating I/O, and potentially doing some compression depending on some settings.
It is even queuing I/O, so if you kill the dd, you will note that the HDD still blinks for a while after dd is killed.
If you want to measure how fast data gets to the disk, with reasonable accuracy, you must do a dd but set the of= to the raw physical device (which destroys file system). If you want to insure that the block sizes are correct and when you do a dd to write chunks in 8KB and exactly 8KB of data gets written and nothing else, then you have to write a program that uses the pass-through /dev/sg driver and send out the WRITE(10) and/or WRITE(16) CDBs. The raw SCSI method is the ONLY accurate means of having complete control.
Active today
Same goes for Windows, the device driver will even re-order writes to try to make bigger I/Os to cut down the IOPS requirement especially if it's a clever RAID controller.
Active today
Thank you very much. If I'm writing to raw device, then how do I much up IOPS value with storage? Does it need to be equal or depends on the storage and stripe size?
Active 6 days ago
There is no such thing as stripe size when sending raw WRITE commands to the HDD. If you send it 2,618 bytes, then that is what it writes. IF you send 32,768 bytes, then that is what it writes.
Now what i think you want to do is send raw SCSI commands to the RAID volume, and that process depends on whether you are doing hardware or software RAID .. so be specific, what is the method you use for RAID?
Now what i think you want to do is send raw SCSI commands to the RAID volume, and that process depends on whether you are doing hardware or software RAID .. so be specific, what is the method you use for RAID?
Active today
No, I mean in storage side, does it convert to different stripe size and write it to disk?
Active 6 days ago
If you send the raw scsi commands to the RAID volume, then the controller will slice and dice as necessary to get it to the disk. Example, if stripe size is 64KB and you have a 3-disk RAID5, then if you send just one byte to the raid volume, then this will generate approx 256KB worth of I/O.
But if you read 1 byte, then that could generate anywhere from 1 byte to 64KB worth of physical disk I/O.
It is all dependent on understanding the RAID architecture.
But if you read 1 byte, then that could generate anywhere from 1 byte to 64KB worth of physical disk I/O.
It is all dependent on understanding the RAID architecture.
Featured Post
![[Webinar] Lessons on Recovering from Petya](https://filedb.experts-exchange.com/files/public/2017/7/11/dcb3272a-62a8-4274-a34e-d1771610ec3b.png)
Skyport is working hard to help customers recover from recent attacks, like the Petya worm. This work has brought to light some important lessons. New malware attacks like this can take down your entire environment. Learn from others mistakes on how to prevent Petya like worms.
Question has a verified solution.
If you are experiencing a similar issue, please ask a related question
Fine Tune your automatic Updates for Ubuntu / Debian
This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting.
This w…
In this video, Percona Solutions Engineer Barrett Chambers discusses some of the basic syntax differences between MySQL and MongoDB.
To learn more check out our webinar on MongoDB administration for MySQL DBA: https://www.percona.com/resources/we…
Suggested Courses
Course of the Month7 days, 23 hours left to enroll
765 members asked questions and received personalized solutions in the past 7 days.
Join the community of 500,000 technology professionals and ask your questions.