Calculation formula for usable disk storage

Is there a formula I can use in Excel to determine the usable space that I'll achieve with various hard drive sizes, raid configs, stripe size, filesystem type, and allocation/cluster sizes and virtual files.

For example, trying to figure out how much usable space I'll have without having to buy the system of the following:

8 x 600GB SAS drives
RAID10
Stripe Size 64K (RAID)
NTFS filesystem
Windows allocation/cluster size: 64K
Then creating a 200GB hyper-v disk file on that volume.

And then figure out the same for 2 x 2TB drives, or with different RAID types/configs.

Thanks!
MrVaultAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

DavidPresidentCommented:
It depends on how accurate you need to be.   It will be impossible (virtually) to obtain exact numbers because you are missing other variables, some are controller/vendor/product specific.  Then you have to deal with "hidden" / reserved areas that are functions of the number and sizes you put on a specific file system based on additional filesystem parameters and the number and degree of fragmentation of files.

With 8x600 in R10 config you get RAW usable of 4x600GB.  With a 2x2000 RAID1 you get raw usable of 1x2000Gb.


0
MrVaultAuthor Commented:
thanks. yeah, I knew the raw numbers (since RAID10 and RAID1 use give half the space). A department here needs 2150GB free space on a volume in RAID10. I don't know if I buy 8 600 GB drives if that will give enough.
0
DavidPresidentCommented:
yes, it will be enough. You left a sufficiently large fudge factor
0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

MrVaultAuthor Commented:
besides experience and going based on gut, how does one come to this conclusion? I think you're right, but what if the department said they needed 2350 free space, or 2300? see what I mean?
0
Duncan MeyersCommented:
Don't forget the binary (computer) v base 10 (marketing) factor...

A 600GB drive is marketed as 6,000,000,000,000 bytes. Your systems look at it multiples of 1024 (1024 x 1024 x 1024 = 1,073,741,824), so what the system displays is 558 GB (or 558.7935447692871 if we're going to be precise) - so what you'll see in your RAID 1/0 set is 4 x 558 = 2235GB. Which is really 2400GB if you're doing the sums in base 10.

It's all a little confusing really...  :-)
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
DavidPresidentCommented:
Well, 20 years experience designing RAID controller, subsystems, drivers, management software;  for most of the manufacturers at one time or another, and the line between gut & experience gets fuzzy :)

I really just factored in a half dozen reserved/unusable areas but didn't do the math. we're talking a few GB.  They probably came up with 2150 rather arbitrarily as well, and if you went back and ask them if this was Gigabytes or Gigibytes you would get a lot of confused looks, and if you asked them individually to tell you how many bytes they needed, you would get some telling you 2150 x 1000 x 1000 x 1000 and others saying they needed 2150 x 1024 x 1024 x 1024.

Just say that drives come in multiples of 600GB, or is it really GiB , and to have it protected they need to buy twice what they need, and to allow for a fudge factor for unit conversions , metatdata, and things like file system overhead that changes depending on how many files you have, and that performance dies once you start getting close to being used up.

As such, you need a 10% fudge factor.  The SAS disks come in multiples of 600G, so if you add plenty of room to insure performance doesn't die when file systems fill up, figure 500GB at a time.   So then just ask if they want 4 sets of 500GB disks or 5 sets.
0
MrVaultAuthor Commented:
Unfortunately getting this department to really explain why they need XYZ usable space is like pulling teeth.

Thanks for the info. I realized most of this (that 100GB is not going to be really 1,000,000K
0
DavidPresidentCommented:
Well you could always try a different technique to give you a wide, comfortable fudge factor.  As them to define "usable".  Do they mean that if you give them a system with exactly 2,350,000,000 bytes, that they can read and write all day long, that they are willing to ignore that the O/S will probably crash and whatever applications they run will blow up if all 2.3TB was actually written?  


Ask them how much free space every program they plan on using requires, in worst case scenario, down to the byte to insure that programs don't die or slow down to the point they are unusable when the 2,350,000,000.   Then ask anybody foolish enough to even try to respond, how many users does that represent, and does it change based on the size of the amount of data they have. Ask if you can have the equation based on the number of data files each application has, so you can give them the precision they desire using a spreadsheet.  Do they want to print files? Ask for the list of files and say they all have different amounts of overhead.

 Somebody will take the bait and tell you to estimate, then you can fire back with  buying in multiples of 500GB and saying it is a decent estimate.

0
kevinhsiehCommented:
One thing to consider is the number and size of files. If you files are 4K and you are using 64K clusters, then you are only going to be able to use 1/16 of your space because each 4K file will consume the minimum allocation unit of 64K.
0
MrVaultAuthor Commented:
Thanks. I can only imagine the looks I'd get if I went that route with asking them to estimate. However I can also vividly see the guy who's head will explode if in fact he gets in there and see less than 2400GB free. I'm going to bank on letting his head explode instead. I'll just claim it was impossible to get more without degrading performance significantly (RAID 10 to RAID 5).

as for the file size, that's good point. Although in our case this volume will hold hyperv VM files, so the file size will definitely be greater than 64K. I always wondered what I should format the drive in the VM as that's inside of the file.

So on host, V Drive is 64K formatted with VM files on it. Each VM file represents a disk presented to a volume (either boot or data disks). When I create a 200GB VM disk as the E drive for the VM and boot up the VM, do I format that E drive as 4K or 64K? The E drive will host SQL databases. I realize virutalized SQL cannot perform as well as non-virtualized, but I still need to get the best performance I can. Typical SQL recommends 64K allocation size for SQL databases and log files, but since inside a single VM disk file, does that even matter anymore? I realize we're getting a bit off topic - but it's fun! :)
0
DavidPresidentCommented:
64KB for sure.
0
kevinhsiehCommented:
For SQL, 64K for both the host and VM formatting. Hopefully you ar using Hyper-V 2008 R2 or R2 SP1, in which case the VHD can by dynamic. Keep in mind that is you get close to filling up the host volume that it will take the VM offline, so don't put a 200 GB VHD on a 200 GB volume and expect that everything will be okay. I would make sure that at least 10 GB is free onthe parent volume.
0
MrVaultAuthor Commented:
if I make the vhd dynamic, will it impact performance as it grows? if I set it to fixed at 200GB and MBT size, can I expand it later with diskpart if needed?
0
DavidPresidentCommented:
it won't make much of a difference, just give you slightly more random I/O, less opportunities for read caching, but you probably won't have a lot of that anyway, so go for it.
0
MrVaultAuthor Commented:
do you know if I can I expand a fixed disk that's formatted as mbt?
0
DavidPresidentCommented:
yes or no. it depends on where the new space comes from.  it starts with the raid, you deal with the 2tb limit also.   lots of rules, but possible in many situations but not all
0
MrVaultAuthor Commented:
so besides not wasting space up front is there any other benefit to doing dynamic? if it can be expanded later, I'm inclined to give it 200GB up front so that I don't think there's more space than we can afford long term.
0
DavidPresidentCommented:
Look, if you have to expand, you have to add the free space to the RAID controller, and then expand the 2TB limit logical devices, then you have to expand those, then you have to expand the partition, then you have to expand the logical volume.  Way to much work.  It isn't practical, IMO to even worry about it.

If you bust out of space, add a pair of 600GB disks, make it into a new RAID1, give it to ESX, then migrate the VM to the RAID1.   Much less operator time.  Dynamic disks are convenient if you DON'T have flexibility that VM provides. In a VM environment, they create more work.
0
MrVaultAuthor Commented:
Not sure if we got confused somewhere. I'm not using ESX. I'm using HyperV on Windows Server 2008 R2. also, the total space for the host volume needs to be be 2TB+. The VM disks don't need to be that large. Only 200GB to start.
0
kevinhsiehCommented:
When using VHD and Hyper-V, you can expand the VHD by shutting down the VM that it is attached to, expand the VHD file, boot the VM and then use diskpart.exe or Disk Management to expand the volume. The VHD itslef can be MBR or GUID. Since you're at 200 GB, it's not likely that you're ever going to expand it beyond the MBR limits, so no harm is doing regular MBR for the disk type.

The downside of using a fixed 200 GB fixed VHD is is they're only using 50 GB and you need to copy the VHD or make a backup of it, because you will have to copy the entire 200 GB, instead of just the 50 GB they're using. A VHD can be converted from fixed to dynamic and back if you have the free space on the host volume to do so.
0
MrVaultAuthor Commented:
thanks kevin. just the clarification I was looking for. we have plenty of free space for now, but don't want them to see that free space and didn't want to over subscribe so we did it fixed. thanks everyone!
0
Duncan MeyersCommented:
Thanks! Glad I could help.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2008

From novice to tech pro — start learning today.