Solved

Strip Size..? Clusters..? RAID..?

Posted on 2012-03-12
36
2,112 Views
Last Modified: 2012-03-16
Hello Team,

I have a silly yet very puzzling question.
I've done a reasonable amount of research on the subject but I cant seem to get my head round it. Since most of the research is coming from forums, alot of the questions are situation specific. What I really want answered is a general.

Anyway.. What is all this Strip Size subject along with Windows Cluster Sizes along with the best configuration. Sites say it depends on your IO and some sites say stripping at 64kb all the way up to 256kb is old school nowadays it is about stripping at 1meg or even 2meg.
When the RAID controller strips the data what I understand its actually dividing the data chunk if you say and writes it to the disks but NTFS is 4kb so why should 1meg or 64kb even make a difference. I mean it has to spread it over 4kb cluster sizes. Then you have arguments saying reads should also be around the size of the files you are going to store on the disks?

Then you have smaller strip sizes for RAID 5 as it has to calculate the parity bit? But it still needs to cut it down to 4kb? If the file is 10meg, it still needs the same reads? I mean is it reading the 10meg file in 1 meg chunks?

All this to save reads and writes? But I cant make sense of it. I mean when its reading in 1meg chunks it still has to read it from 4kb clusters so why having 1meg strip size beneficial? Same with the writes?

I understand writeback, write through, read ahead and all those technologies.

I just cant make sense of how these strpping sizes and clusters make sense?
I mean doesnt it all depend on the file structure..? What am I missing?

Can someone explain how all this works?

I'm sure its simple but I just need a helping hand?
0
Comment
Question by:dqnet
  • 12
  • 10
  • 8
  • +1
36 Comments
 
LVL 47

Accepted Solution

by:
dlethe earned 200 total points
ID: 37713323
Those sites that just say to stripe at 1MB or higher are wrong.   i've been writing RAID controller firmware, software, drivers, etc since late 90s and everything is relative.

There is no universal "best" other than the "best" way to set the stripe size is closest to whatever method generates the least amount of physical disk I/O.

While gross generalizations can be made by RAID level, in the real world things like whether or not your controller (or RAID software) does variable block size I/O; read load balancing;  queue depth; how much I/O reordering it can do;  cache/buffering (in drives, and controller, and operating system).

What about your I/O mix?  Are you random, sequential, reads, or writes (percentages of each).  What about block sizes, 4KB,8KB,16KB ... 2MB?  What if they are mixed?

One can easily make a 4-disk RAID5 array outperform a 2-disk RAID1 array, or vice-versa if you are free to choose the benchmark.

Throw in a file system and then you change things also.  

In general, and I emphasize general, you want to tune the RAID to match the file system and the I/Os that the applications cause the file system to cause the device drivers to issue.  You want to have to go to media as little as possible.   If you need to save 64KB then you want to just save 64KB in such a way as it completes ASAP.   You don't want to write 8KB 8 times because add the parity, and you will end up creating over a dozen i/Os, when you could set it up in such a way to only do 2 I/Os, in parallel.

If doing SQL server, then I/Os by the application are going to be 64KB at a time.  so if you are doing a lot of random stuff, or using index files, and your RAID is tuned to have to write  1MB at a time then performance will be awful.  Conversely, if you are video streaming and have to maintain 300MB/sec for video streaming studio quality HDTV, then if you use 64KB stripe, then you may have problems.  If you use 4KB, then it will simply be impossible to maintain the speed.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 37714129
If I understand this right, and since dlethe writes this code he can definitely correct me if I am wrong.

Windows NTFS has it cluster size, which depending on the partition size defaults anywhere from 4K to 64K.  This is the unit that NTFS allocates disk space when creating a file.  So, if you have a 4K cluster and a 1 byte file, you get 4K.  You just waste 4095 bytes worth of storage.  The bigger the cluster size the more space you waste on smaller files.

RAID has strip size, again if I understand this right, this is the maximum amount of data that will be read or written in one I/O operation.

Say you have a file that is 1MB, or 245 clusters.  If you have a strip size of 64K you can fit 16 clusters within a single stripe and so it would take 16 I/Os to read the file.
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 37715610
To understand all of this, you need to know the path from the application to the drives:
Application may use NTFS or raw drives (rare, mainly DBMS because this avoid to use the NTFS cache and it's slightly faster) to issue its io queries. Read/write functions has many option allowing the app to notify the storage subsys of some "preferences" (read ahead, priority, block vs stream, synch/asynch, large/std, ...

NTFS has its cluster size which is 4KB by default (for drive larger than 2GB) but you can set it up to 64KB
The io controller use the RAID stripe size as a basic unit for each io request, obviously good controllers will have an optimized firmware allowing for many optimization. Although, any parity raid (5/6/50/60) array will terrible suffer from a random io stream because of the read/xor/write penalty that is required for all non full stripe io
A large write back cache backed by a battery allows the io controller to reorder the ios for avoiding read/xor/write...but it can not render a random io stream as a sequential one !
Alignment of OS partition with drives boundary (stripe or sector size) is a mandatory optimization (Windows before Vista/7/2008 were reserving 31.5KB before the data that misaligned any default partition, Vista/7/2008 aligns at/reserved 1MB which is compatible with most stripe, cluster, sector size.

So, if your expecting mainly sequential streams with large files (like Videos), you obviously have to define all stripe and cluster to the maximum allowed.
If you are defining a DBMS storage subsystem, you should take care of the DBMS io size to define cluster (if not using "raw drives") and stripe to the io pattern behavior you are expecting on your DBMS (I mean OLAP is very different than OLTP)...data, index, logs, temps may be optimized with different cluster/stripe values.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37715759
To add to what BigSchmuh elegantly wrote, and to clarify what I wrote ... we are dealing with some generalizations.  Going back to your earlier question about if you have a stripe size of 64KB ... it takes 16 I/Os.

Not necessarily true.  I/Os can be of different sizes, and the RAID controller + filesystem + filesystem settings + disk cache + RAID cache + O/S buffers + outstanding I/Os and a few more variables ultimately decide how many I/Os are required.

Even the # of I/Os required isn't all that important sometimes, if you can do things in parallel.

You will get 100MB of data faster with 1600 x 64KB I/Os if you are doing it in parallel on multiple drives, with software RAID5, then if you have 100 x 1MB I/O to a solid state drive in some cases.

Everything matters, and the more you know, the more you know that what you know isn't enough to know what you don't know ;)  

You know enough to be dangerous, and that is pretty much all you can hope for, as even people who have been doing this since the 90s are still learning things.
0
 

Author Comment

by:dqnet
ID: 37716450
All your answers have been outstanding with great level of detail. Thank you for this. I have absorbed most of the information but I have a few more questions :(

let's take our ERP system for example. Every report outputs a pdf file the equivelent of anywhere from 64kb up to 200kb. The server is virtulised and the Parent OS is running Windows. The strip size is 64kb with a "full strip size of 128kb [whatever that means]" Both the host and the child are running Windows server 2008 r2 (so no alignment issues here) and are running in RAID 5. Would you say this is a good setup?
(HP Proliant DL380 SmartArray P400i, dual port sas drives)

With regards to this statement:
"Windows NTFS has it cluster size, which depending on the partition size defaults anywhere from 4K to 64K.  This is the unit that NTFS allocates disk space when creating a file.  So, if you have a 4K cluster and a 1 byte file, you get 4K.  You just waste 4095 bytes worth of storage.  The bigger the cluster size the more space you waste on smaller files.
" Does this mean you are actually wasting the remaining 3k of space for every empty notepad file meaning you can fill a drive with this method?

Thirdly, when the disk is writing or reading from windows, it is still having to read it in 4k chunks, I mean what difference does it make? Whatever the raid controller strips at, it still has to spend time distributing it across 4k clusters? Doesnt that mean regardless of what you strip at, it will always take the same time? Thats what I cant get my head round? The RAID controller would cut it into 64, 128, 256, etc. but windows or something inbetween would still have to distribute this in 4k pieces too? Isnt this the bottleneck?

With regards to the alignment, now I don't understand this issue. It seems way too daunting and nobody seems to know. My friend's system is running Windows XP and is not using the new 4k drives, Does this mean he has been running a messed up setup for the last 3 years? I mean if this alignment issue was true, why hasnt everyone formatted and started again if there is such a huge performance gain? Nobody even seems to know about this? If so, how would you even set the alingment? Would you need to use a specific drive tool before you install windows?

I know, there are tons of questions here and I apologise but I have no where else to take these questions. Everyone keeps arguing about what the best setup is and nobody seems to know why. 1meg is the best, 2meg is the best, use more spindles, etc.

Many thanks.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37716856
Well, now at least for me it is getting into to many layers and too many unknowns to give you the correct answer.

So here is how YOU can derive the correct answer.  

Learn how to use perfmon.  (The windows performance analyzer).   This can tell you I/O sizes, queue depth, throughput and many things that we have mentioned, from the perspective of the O/S.

If you make sure the source data is contiguous and defragged as much as possible (look for wininternals contig.exe at the msft site, which makes files contiguous, and use it in combination with the defrag a few times then you will have good data that is close enough for a source).   Also make pagefile and everything else as contig and defragged as possible.

Then use the RAID and create a fresh new RAID1 for an R:\ drive,  using a 64KB NTFS, do a full initialization format to zero everything out, then copy all those PDF files, and look at permon, and a timer.   Look at the total bytes used and record it.

Repeat to insure results are consistent.

Then reformat the file system, again, not a quick format, you want it all blown away, but use default 4KB NTFS.
Repeat tests above.

Compare.

Now you have a baseline.

Build a 3-DISK RAID5 with a few chunk sizes like 16KB , 64KB, 256KB,  compare with 4KB & 64KB NTFS settings.  So that is 6 combinations.  

If you are good with scripting (well, really good), then this can be all automated to run over a weekend, assuming your RAID controller has a command-line configurator.  

Or, if your RAID controller does not have one, you can get a good approximation if you just add the drives and tell the RAID controller that they are JBOD disks that should be exposed to the O/S.

Then you can configure the disks as windows dynamic drives, and then use native windows software RAID, and automate all of this with a powershell script.

That is what we do.  (Well, actually over time, we've developed quite sophisticated programs that can do much more, because we have to look at more, but you get the drift).
0
 
LVL 57

Assisted Solution

by:giltjr
giltjr earned 100 total points
ID: 37716861
--> Does this mean you are actually wasting the remaining 3k of space for every empty notepad file meaning you can fill a drive with this method?

Yep.  In XP, not sure about the newer  Windows, if you right click on a file and select properties it will show you two sizes.  One size is the file size, this is how many bites the file actually contains. The other size is the size on the disk, this is how much disk space the file actually takes up on the disk, which is always a multiple of the cluster size.

On avg. the amount of disk space wasted is number of files times 1/2 of the cluster size.  If you have 4K clusters (4096 bytes) on avg. you are wasting 2048 bytes.

--> Whatever the raid controller strips at, it still has to spend time distributing it across 4k clusters?  

No, Windows is reading/write 4K "blocks" of data, but the RAID system (be is software or hardware) is physically writing it in "strip" size.  

With RAID the OS does not see what is happening at the physical disk level, in a sense what the OS sees is a virtual disk (not to be confused with virtual volumes).

RAID sits in-between the OS's normal IO functions and the hard drive.  In the case of software RAID sits just below the OS's normal IO functions, in the case of hardware RAID the drivers sit below the OS's normal IO functions and passes info to the RAID controller.

Remember, the OS sees a single drive, when in reality there are multiple physical drives.

--> My friend's system is running Windows XP and is not using the new 4k drives, Does this mean he has been running a messed up setup for the last 3 years?

No.  If you were to assume he was running messed up for the last 3 years, then you would have to assume everybody has been running messed up since the mid-1950's.

The 4K you are talking about is the physical sector size on the disk.  That is how data if physically written on the platter.  Today it is 512 bytes and has been since the 1st hard drives were creates back in the mid-1950's.

A cluster/stripe is nothing more than a collection of sectors.

The 4K standard has just been recently standardized, 2010.  The biggest thing that 4K drives give you is less wasted space.  There is a bit of addressing on each sector on the drive, with 4096 byte sectors you have 1 set addressing for 4K bytes.  With 512 byte seconds you have 8 sets of addressing for 4K of bytes.  So by using 4K sectors, you waste less space per cluster.
0
 

Author Comment

by:dqnet
ID: 37718481
---> The 4K standard has just been recently standardized, 2010.  The biggest thing that 4K drives give you is less wasted space.  

Does this mean that only the new 4k drives need aligning with older Windows versions..?
Also does this mean Windows 7/2008 checks the checks the type of drive and aligns accourdingly..?

I guess what I am really trying to ask is, how do you know you have this problem? Let's say I purchased a drive pre-2010 (SAS, SATA, IDE) whatever it may be and installed Windows Server 2003 or XP, would I have this problem?

Same question with 4k cluster drives?


Nearly there :)
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37718627
In most cases, the 4K drives are effectively running in an emulation mode so as far as the host O/S knows, the disks are still using 512 bytes per block.  Some benchmarks can even be contrived to make either the 512 or 4KB disks run faster, depending on what the desired results need to show.

All this wasted space is still there.  The benefit has nothing to do with saving space. It has to do with efficiency and getting past the 10-Byte CDB issues, and addressability.  

If you need to write 512 bytes then it takes one I/O to do so, each will write a single block, provided he engineering work has been done (which it hasn't for most operating systems).

If you write 4096 + 1 byte, it will take the same number of I/Os  (1) on either disk, but the I/O sizes will be different.

The way these disks work in the windows world is that they emulate 512-byte address space, so if the I/Os are NOT a multiple of 4096, which is often the case, then you will end up doing more work with the 4KB disks.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 37719696
Well everything I have read said that 4K sectors came about because it was tougher and tougher to get more data in the same physical space.  The 4K sector was to reduce the amount of space used for each sector (gap, sync, address, and ECC). Ref:

  http://www.anandtech.com/show/2888
  http://en.wikipedia.org/wiki/Disk_sector
  http://www.seagate.com/docs/pdf/whitepaper/tp613_transition_to_4k_sectors.pdf

Yes, there is still wasted space, but with the large size of disks today and each sector taking 50 bytes alone for ECC you have a lot of wasted space. Now 4K sectors use a 100 byte ECC, so you save at least 300 bytes of space going from 8 512 byte sectors to a single 4K sector.  When you figure that a 400GB drive has close to 800 million 512 bytes sectors (ref. the segate doc from above), that adds up to a LOT of space.

However, that just means there is less wasted space in the overhead of sectors on the disk.  You still waste space within the sector for any file that is not a perfect multiple of what ever your cluster size is.

As for performance.  Well, if the OS was aware of the the 4K sectors and used them, and you had large files, and were accessing them sequentially, 4K sectors would perform better.  

However, if you mainly have small files (less that 512 bytes) or had large files that were you accessing randomly (like a database), then maybe not so good.
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 37719871
Apart from the misalignment problem which can really lead to a performance drop, I would say that ALL usages are faster using 4K sector vs 512 bytes sectors. Database use small pages ... but I do not know any DBMS using smaller page than 4KB, Oracle defaults at 8KB, DB2 uses 4, 16 and 32KB pages, MS-SQL mainly uses 8KB.
0
 

Author Comment

by:dqnet
ID: 37720116
yea but does this mean that only the new 4k drives need aligning with older Windows versions..? Also does this mean Windows 7/2008 checks the checks the type of drive and aligns accourdingly..?

I guess what I am really trying to ask is, how do you know you have this problem? Let's say I purchased a drive pre-2010 (SAS, SATA, IDE) whatever it may be and installed Windows Server 2003 or XP, would I have this problem? How do you know you have this problem? as well as the millions and millions of people that have these predated 2010 disks and even the newer 4k drives?

this is an interesting read to:
http://forums.storagereview.com/index.php/topic/30611-why-would-a-cloned-4k-aligned-drive-be-slower-than-un-aligned/page__p__272671__hl__alignment__fromsearch__1#entry272671
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37720124
If you are using Win7, answer is NO, because win7 does it anyway.  If using previous windows, answer is YES, but then ALL drives need aligning if < Win7.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37720161
All uses  are NOT going to be faster with 4K alignment.  I can certainly set up a benchmark to prove otherwise by setting up a write benchmark that uses random I/Os of some multiple other then 4KB.

If your computer needs 64KB of data, then the O/S, device driver, RAID controller, buffers, and disk itself will do it's best to make that into a single 64KB I/O request, especially if the data is contiguous.  

Don't forget, whether ATA or SCSI spec, there are READ & WRITE commands that specify the NUMBER of blocks you want.  Just because a HDD has a minimum I/O size internally, doesn't mean that non-4KB drives can't do a minimum of 4KB at a time anyway.
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 37720240
Using an old OS (Ex: XP/Win2003), one can align his partition explicitly at any boundary using Diskpart.exe MS utility...but all partitions created on those OS using the defaults parameters are aligned at 31.5KB (=63 sectors) and will have a small performance drop using a 4KB sector drive. This is very well known on the SSD market for example.
==> Many partition utility can re-align your existing partitions...this is a bit dangerous (backup backup backup) but works perfectly.

4KB sector drives performs always better than 512 b sector drives...because :
-the 4KB sector HDD are using a more dense storage (less checksum and overhead bytes)
-reading 4KB requires only 1 checksum validation (usually only SAS drives performs a checksum at read io time)
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37720309
The checksum validation (well, there is no such thing, it is ECC) is done in the hardware, MUCH faster then the data can get to/from the media.   It is done in nanoseconds.   I/Os take milliseconds.

Conversely, thought experiment ... What if you need 512 bytes?  Which takes longer, the checksum/ECC to verify the data is good, or needlessly being forced to read an additional 3584 bytes, and then validating ECC on 3584 bytes that are going to go into the bit bucket?

-- And then while your 4KB drive is reading that 3.5KB it doesn't need, my disk will NOT be standing by and doing nothing, I'll do a few I/Os elsewhere ;)
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 37720355
You may doubt...up to reading the below review where a 1TB drive from WD is reviewed for its 512b and 4KB editions EADS vs EARS series...
http://www.tomshardware.com/reviews/wd-4k-sector,2554-2.html
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37720530
Ahh, vindication

"The Performance Trap

If 512 byte data has to be written across two physical 4KB blocks, the hard drive will have to read the 4KB blocks that are affected, introduce the modifications, and write them back onto the drive. This process is referred to as read-modify-write (RMW). While this doesn’t hurt much if it happens here and there, it becomes a significant issue if the alignment of eight 512 byte sectors into only one physical 4KB block is wrong. Therefore WD offers either its Alignment software (information and downloads can be found here), which rearranges all data on a hard drive to fit the logical 512 byte sectors into the physical 4KB sectors."

=========
The weakness with the WD realignment software is that it certainly won't have desired affect if you are using a RAID controller, or are using virtualization, or are not using windows.

It does prove my point, these drives are slower unless you can do all I/O aligned, and 4KB multiples.  Let's face it, they wouldn't spend all that $$$ writing and maintaining such software if this was a no-brainer everything-is-faster-with-4KB-drives as you originally posted ;)
0
Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

 
LVL 18

Expert Comment

by:BigSchmuh
ID: 37720618
...but, as stated before, you ARE doing all io 4KB aligned as soon as you have a correct alignment !
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37720700
But alignment is NOT guaranteed if you have a RAID controller, or any other kind of virtualization or multiple O/S partitioning.  Nor is there any guarantee of I/Os of 4KB in size.  So performance gain is ONLY guaranteed in both situations.

So you can't just assume the disk is going to be faster as a general rule, which at least IMHO you were making with that comment about the disks always being faster.
0
 

Author Comment

by:dqnet
ID: 37720844
If you are using Win7, answer is NO, because win7 does it anyway.  If using previous windows, answer is YES, but then ALL drives need aligning if < Win7.


That's ridiculous, so for the last 12 years since Windows 95 all our drives or so to say all the idiots have been running at a performance drop without a single article explaining we were doing it wrong all along? I mean how did the world stay quite about it?
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 37721016
4KB sector drives are very recent HDD, same case for SSD.
==> So this was not ridiculous from dlethe and there many articles about that misalignment problem (mainly for SSD because there were suffering from that before any 4KB sector HDD hit the market) !

Misalignment is a clear drop performer for pre Vista/7/2008 OS and most Linux
0
 

Author Comment

by:dqnet
ID: 37721780
==> So this was not ridiculous from dlethe and there many articles about that misalignment problem (mainly for SSD because there were suffering from that before any 4KB sector HDD hit the market) !

Where did I say dlethe is ridiclous????? If antything, the education I have gained from him is incredible!


4KB sector drives are very recent HDD, same case for SSD. Misalignment is a clear drop performer for pre Vista/7/2008 OS and most Linux

With regards to the recent 4k drives,this is what I have been trying to ask. Is this problem resident in older drives and not the new 4kb drives.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 37721913
O.K. looking at some of the benchmarks from Tom's Hardware, unless I am missing something nobody will really notice the difference.

Access time is not really any faster, and avg. throughput is within a few MB/sec of each other.

I don't think the avg. Joe running Windows 7 opening up a 100K word doc will notice.

Maybe I'm missing something in the performance tests.
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37722085
Tom's hardware knows nothing about real-world benchmarking. That is the best and most polite thing I can say.

It isn't possible to properly asses this technology when you constrain it to being the boot disks in a home computer.

Get a rack of  SAS-2 enclosures with them where you are maintaining servers that are streaming a few GB/sec and/or working on a database that is pushing a quarter petabyte then  differences get pretty obvious.

It is a waste to even worry about such things for the "average Joe".
0
 
LVL 57

Expert Comment

by:giltjr
ID: 37722163
I understand that we don't need to worry about this for the average joe, but dqnet was:

"My friend's system is running Windows XP and is not using the new 4k drives, Does this mean he has been running a messed up setup for the last 3 years? I mean if this alignment issue was true, why hasnt everyone formatted and started again if there is such a huge performance gain? "

The answer is that because for the average joe 4K drives and alignment doesn't matter.

It only matters for for servers that attempting to move "a ton" of data.
0
 

Author Comment

by:dqnet
ID: 37723504
Tom's hardware knows nothing about real-world benchmarking. That is the best and most polite thing I can say.

...Agreed

It is a waste to even worry about such things for the "average Joe".
...Also agree, unless your one of those gamers who feel running RAID 0 has a huge gain!

But the real question is, how do you resolve this issue? Do kids with the older drives have the same problem or is this specific to the newer 4k drives?
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37723653
Think about what you wrote ... how do you resolve a problem where some aspects of your configuration isn't optimal?

You can't even measure the effects of this without running a benchmark designed for that purpose.  In other words, the issue only manifests itself when you are doing something (running a benchmark) that you never would run otherwise.  

As such, there is no "issue to resolve", because there is no "issue" that needs resolving.  Plenty of more important battles to worry about, like whether or not write cache is turned on or the queue depth isn't optimal.
0
 
LVL 18

Expert Comment

by:BigSchmuh
ID: 37724095
As of today, alignment is not supposed to be a problem at all :
It's aligned by default with Vista/7/2008
It's not a problem for 512 byte sectors HDD
It's not a serious problem for RAID 1/0/10 arrays

Old OS servers requires some work to get their partition aligned with their drives:
Diskpart MS utility does the job (you can even script it)
Boot partition can't be anything else than on a 4KB cluster
Drive, array cluster alignment is a mandatory point to take care of BEFORE defining a partition
There are many partition utility (Ex: Partition Wizard) that allows to align a misaligned partition
0
 

Author Comment

by:dqnet
ID: 37724490
As such, there is no "issue to resolve", because there is no "issue" that needs resolving.  Plenty of more important battles to worry about, like whether or not write cache is turned on or the queue depth isn't optimal
.

Well that isn't entirely true.. The issue is present if we like it or not.. It's just a question of how us people choose to deal with it and how the importance of it applies in your environment


It's not a serious problem for RAID 1/0/10 arrays
How comes? Aren't they based on the same principle therefore can be affected just as much?

So 80% of the people out there who still use the old 512 byte drives don't have this problem with older versions of windows? That's a relief!
0
 
LVL 18

Assisted Solution

by:BigSchmuh
BigSchmuh earned 200 total points
ID: 37724796
Please see below some examples of misalignment drop performance :

1/ No RAID or RAID 1/10/0 on 4KB sector drives
Assume we have 2x HDD and a single RAID 1 array partitioned using Win2003 defaults
31.5KB are reserved by the partition header which means the first 4KB data cluster are stored on the last 512 bytes on the 8th 4KB sector and the first 3584 bytes on the 9th 4KB sector
When writing, you suffer from 1 read io penalty (reading 2 sectors) which is a real problem only when your application are issuing a large random write io (Ex:Database). This penalty is lowered by any write back cache
When reading, you read 1 more 4KB sector which is negligible

2/ RAID 5/6/50/60 on 4KB sector drives
Assume we have 3x HDD and a single RAID 5 array (64KB stripe) partitioned using Win2003 defaults
31.5KB are reserved by the partition header which means the first 4KB data cluster are stored on the last 512 bytes on the 8th 4KB sector and the first 3584 bytes on the 9th 4KB sector both on the first drive 64KB stripe
When writing, you suffer from 2 read io (data + parity) + 1 write io penalty which is a terrible problem when your application are issuing a random write io (Ex:Database)
When reading, you read 1 more 4KB sector which will be on the next drive in 1/16th time - a 6% drop -
0
 

Author Comment

by:dqnet
ID: 37726879
Perfect - explains it spot on.

If this wasnt an issue with older drives, why is everyone making such a big deal of this issue. The 4k drives only became available recently and Windows V/7/08 all solve the problem.
Why would anyone even instal xp or 2003 now anyway, let alone using the new 4k drives?
0
 
LVL 47

Expert Comment

by:dlethe
ID: 37726934
Maybe only the people you know, or trade rags you read make a big deal about it.  Certainly it is not a big deal from the likes of the storage subsystem manufacturers and engineers I deal with on a daily basis.  

As for why people still use XP/W2K3 is easy. It isn't worth the expense & effort to change, and virtualization makes it unnecessary.  An O/S is a means to an end. I can just as easily say why would anybody even install Win7, or even pay the microsoft tax at all, if their app runs under one of the *nix operating systems.

Perspective.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 37727767
--> If this wasnt an issue with older drives, why is everyone making such a big deal of this issue.

Who is ever body?  I never even heard of this issue until this thread.

--> Why would anyone even instal xp or 2003 now anyway, ...

I agree with dlethe.  Why upgrade when what you have works?  Does Windows V7 make the Internet go faster?  Does it make me type any faster when using Word?

Do you get a new car every year.

I use Windows because my company makes me.  My favorite OS is IBM z/OS, but I can't afford an zSeries server to put in my house so I use my second favorite OS at home, Linux.

I know of places that are still running Windows 95.  Why?  Because they have special hardware or custom written programs that won't work on anything newer.  They don't want to pay the cost to have the programs modified or they can't get drivers for the hardware.  It works, why change?  I mean does Windows V7 make that PC that is acting like a cash register work any better?  Can Windows V7 add 1+1 better than Windows 95?

--> .... let alone using the new 4k drives?

Well, some people always want the latest and greatest.  Manufacturers are making it and shipping it, soon that will be the only think you can get.
0
 

Author Comment

by:dqnet
ID: 37728334
Fantastic thread.. Very educational.. Thanks to all..! :)
0
 

Author Closing Comment

by:dqnet
ID: 37728344
I would have liked to give 500 to each.
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

Data center, now-a-days, is referred as the home of all the advanced technologies. In-fact, most of the businesses are now establishing their entire organizational structure around the IT capabilities.
In the modern office, employees tend to move around the workplace a lot more freely. Conferences, collaborative groups, flexible seating and working from home require a new level of mobility. Technology has not only changed the behavior and the expe…
This video teaches viewers how to encrypt an external drive that requires a password to read and edit the drive. All tasks are done in Disk Utility. Plug in the external drive you wish to encrypt: Make sure all previous data on the drive has been …
This Micro Tutorial will teach you how to reformat your flash drive. Sometimes your flash drive may have issues carrying files so this will completely restore it to manufacturing settings. Make sure to backup all files before reformatting. This w…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now