what % of data drive can you fill up before performance hit happens?

if you have a dedicated drive (not system drive) for archive files, is there a standard formula: that at special % when the drive is full, then it's performance will be hit or you should make the drive bigger?

Let's say the drive is 500GB, would you concerned if it reached 60/70/80/90% full? and what will be your next strategy for that kind of situation?
LVL 5
25112Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

dbruntonCommented:
>>  Is there a standard formula: that at special % when the drive is full, then it's performance will be hit or you should make the drive bigger?

No standard formula.

I'd be concerned with backups first.  Are they occurring and current.
Next I'd look at how fast my data is growing and plan for an upgrade if I'm going to get near some limits (keep reading).
You need to allow empy space for defragging purposes (if you've got a spinning hard disk).  Say 20 to 30% to make the job easier.
You may have your temp files being stored on your data drive.  Word for example creates these while creating documents.  Your browser does this too.  Browser caches can easily exceed a Gb in size.  Allow space for those and run an application such as CCleaner to remove these occasionally.
Roughly at over 60-70% of a full disk I'd be looking for more storage but I wouldn't be panicing about it.
0
Gary CaseRetiredCommented:
Sectors on a hard drive are allocated starting at the outermost cylinders and moving inward -- so the highest sector numbers, which are generally the last ones used by your file system, are on the innermost cylinders => where the read and write speeds are the slowest.    The exact difference between the speeds on the outermost  and innermost cylinders depend on the specific drive, but in general there's about a 2:1 difference in the speeds.

So Yes, there's a performance difference due entirely to the physical characteristics of the drive.   But that's not generally a major consideration -- and indeed the file system may not allocate in sequential order (NTFS does not, for example), so there's no guarantee that only filling a drive to a specific % will eliminate any use of the innermost cylinders.

If you want to ensure no data ever uses the slowest, innermost cylinders (say, perhaps the last 25% of the drive); then the way to accomplish that isn't to watch how full the drive is => it's to "short stroke" the drive ... i.e. when you create your data partition on a new drive (or an empty one with no partitions), make the size of the partition 75% of the size of the drive.    Then the partition won't include the slowest part of the drive, so you don't have to worry about the physical performance slowing as much.

Beyond that, there's also the file system's characteristics.   NTFS is pretty good at not slowing down on full drives, as long as there's space for the file you want to write that's not excessively fragmented.    Some file systems (notably Reiser in Linux) get VERY slow on full (90% or so) drives, as they spend a lot of time "analyzing" where they should write the file -- the actual write is at full speed, but it doesn't start for a bit.

With Windows and NTFS, I simply wouldn't worry about it -- the only performance "maintenance" you might want to do is keeping the drive reasonably defragmented ... and if you're running Windows 7 or 8, that happens automatically once/week unless you've changed the settings.

Even that may not be an issue if you're using the drive just to archive files (as you implied) ... if you don't do much deletion/changing of the data, it's likely the drive never gets fragmented anyway.
0
nobusCommented:
i would also look at what you want to archive ;
if you are putting files or folders on it  which size takes GB's - you need obviously more than this, i believe the double of free space
if you store mostly small files there's less of a problem there

is this a theoretical question - or do you have an actual problem?
0
Cloud Class® Course: CompTIA Healthcare IT Tech

This course will help prep you to earn the CompTIA Healthcare IT Technician certification showing that you have the knowledge and skills needed to succeed in installing, managing, and troubleshooting IT systems in medical and clinical settings.

Dr. KlahnPrincipal Software EngineerCommented:
Agree with garycase's comment above re fragmentation.  A drive can be 95% full but if it has only a few big and unfragmented files there will be no performance hit.  On the other hand, if the drive is only 10% full but the files are many, small, and fragmented per-block the performance hit will be severe due to the drive constantly seeking.

If you give us more information regarding the size of the files, how often they are used, whether read or write dominates, we can probably give you feedback more applicable to your specific situation.
0
nobusCommented:
you can put millions of jpg's on the drive  -if you never access them -  there's never a performance problem
but if all are in the same folder, and you try to open that - you'll have to wait long

so best organise your files in a decent folder structure with subfolders if needed; instead of a couple very big folders
0
25112Author Commented:
Thank you..

the backups are happening. yes, i will remember the 70% rule dbrunton mentioned.

Gary, is this example appropriate to what you said (75% rule suggestion): if the drive is 500GB, then only partition it for 375GB and leave the 125GB empty? good to know about NTFS doing better on this.

  nobus, they are small files (MBs not GBs)
 
   right now, the C drive is 1 TB, 200GB user data and rest free.


one correction, i would like to make.. i said 'archives'.. yes, it will be mostly, but it will be hard to distinguish totally which ones are totally archives, as the work may require use of any of those files, and in strict sense, it may not be actually called archives.

But the system is slow often because there are too many files (MSOffice documents, pictures, videos, photoshop work etc), everytime C drive is opened... i understand the reasons you mentioned, and yes, the defrag is happening every week.
   
   read & write both happens a lot. my bigger aim in the question is how should I partition the 1TB.
     
 there are 2 factors:
 the system is little old (4 year old dell pc). there are two primary uses: one is GIS mapping software which is more resource intensive, and other photoshop.. other stuff are all MS-office based.
 
 There are 2 users.. Either the GIS or the Photoshop user.
 
 When the photoshop user is on, they will need access to the 200GB data and that will grow maybe additional approx 100GB usage a year.
 When the GIS user is on, they don't need any access to that data, but their data folder is relatively very small (max 2 GB).
 
 So, thinking of more efficient solution for both users. The GIS user needs all of CPU/Memory/DiskSpin dedicated to him. The photoshop user work is less critical.
 There will be a reboot between the time the 2 users use the system. GIS user at beginning of week and Photoshop user later in the week.
 
 We need a properly balanced drive planned as the system is ready for a fresh reinstall.
 
 Should I have something like 250GB-750GB partition between OS & data drive? or 500-500? The main installs on Win7 OS drive will be Photoshop, GIS program, MS-Office 2010 (very heavily used). thanks for any ideas.
 
 (This setup would be temporary, as in 3 months, likely there will be another system made available dedicated for the GIS user.)
0
nobusCommented:
could you post more details about the drive connection?  
is it internal, over usb, or lan?

**it may even be bad sectors causing this
0
25112Author Commented:
nobus, the 1TB is just internal.

the machine has never had a reinstall in 4 years.. so this is a good time to revisit it..
0
nobusCommented:
worth testing the drive once; just to be sure that isn't the cause
i'm sure you know where to find the diags?
0
David Johnson, CD, MVPOwnerCommented:
Just my 2 cents.. I've never noticed a drastic slow down as the drive fills up.. I suppose if one was to benchmark things one could see a difference but is it enough of a subjective difference or more of a theoretical difference that one actually would notice the slow down.. Can you tell the difference between 750 m/sec and 850 m/sec to retrieve a file? Disk fragmentation is the larger issue, so the more contiguous free space you have the better things are head seeks are the major time waster and if one has to seek from the outermost to the innermost tracks back and forth things will be really relatively slow compared to if the entire file was on the 'SLOW' innermost sectors.
0
25112Author Commented:
nobus, are you referring to CHKDSK? if more sophisticated tool, pl suggest.

Dave, are you referring to Gary's suggestion "make the size of the partition 75% of the size of the drive" (Disk fragmentation I believe is no issue due to auto defrag each week)
0
Gary CaseRetiredCommented:
" Gary, is this example appropriate to what you said (75% rule suggestion): if the drive is 500GB, then only partition it for 375GB and leave the 125GB empty? good to know about NTFS doing better on this. "  ==>  Yes, that's exactly what you could do.

r.e. the comment about opening folders with a lot of graphics taking a long time =>  This is true because the system needs to generate thumbnails for all the images (even if there's already a Thumbs.db file, it will still check to confirm it's current, which takes time).    You can completely eliminate this delay by setting your folder options to not show thumbnails.    This may or may not be something you want to do ... but many folks with very large folders full of pictures/jpegs find this very helpful.

From what you've outlined, I'd partition the 1TB drive as 300GB for the system;  400GB for a 2nd data partition; and leave the last part of the drive unallocated.   If the two users have their own accounts, I'd relocate the data for the photoshop user to the 2nd partition.    The GIS user could store his/her data on the C: drive if you want; or you could also move that to D:    [There's a very tiny performance benefit to leaving it on C:, but for the reasons I mentioned earlier you may want to move it to D:]
0
Gary CaseRetiredCommented:
One other thought to ponder:    SSD's are MUCH less expensive these days than they were just a year or so ago.    A 500GB SSD is in the $200 range.

With the amount of data you've noted, you could simply do the reinstall on one of these and the performance would be VERY nicely improved for both users.    You may still want to make two partitions -- one for the OS and programs; one for data -- but that's purely for organization reasons ... nothing to do with performance.    If you do that I'd probably just use half the drive for each.
0
25112Author Commented:
>> I'd partition the 1TB drive as 300GB for the system;  400GB for a 2nd data partition; and leave the last part of the drive unallocated.
OK.

>>but that's purely for organization reasons
Gary,

you would suggest not to bother about partitioning (not mandatory for SSD performance) and have one 500GB C drive for everything( and then keep the current 1Tb has an external/additional drive). In case of SSD, the rule of 75% does not apply, is that right?
0
Gary CaseRetiredCommented:
Correct, with an SSD there aren't any "inner" or "outer" cylinders ... access to anywhere on the drive is a matter of less than 1ms of settling time for the electronics (typically ~ 150 ns).
0
Gary CaseRetiredCommented:
Note that if you replace the hard drive with an SSD, then as long as this system has a reasonably recent processor (any Core architecture CPU ... especially if it's a recent i5 or i7) will "feel" like a new system to your users :-)
0
nobusCommented:
a diagnostic = the manufacturer's diag
if you don't post the brand -i can't suggest a correct diag
but you can find WD diags here, or on the UBCD : http://support.wdc.com/product/download.asp?groupid=612&sid=3
0
25112Author Commented:
thanks gary and nobus.

would you suggest Inspiron 620 (2011) is ready for SSD? (it is a i5)
also please suggest most appropriate diag for this.
0
nobusCommented:
any system with sata connections will be able to use SSD's
diags for SSD's are not very common - i know the intel toolbox for intel ones : https://downloadcenter.intel.com/download/18455/Intel-Solid-State-Drive-Toolbox

but for others -  the manufacturers have to make the tools available
0
Gary CaseRetiredCommented:
Both Intel and Crucial have good diagnostics for their SSDs -- they're also the SSDs I recommend the most, so I'd just buy one of those and use the appropriate diagnostic package.

But you likely won't ever need to bother, as they're exceptionally reliable and you don't really need to run diagnostics on them.
0
David Johnson, CD, MVPOwnerCommented:
if one only has sata version 1 or version 2 then you won't be able to get the maximum benefit of a sata version 3 (6GB) but the difference in speed is rather dramatic in comparison with a 5400 rpm green or even a 7200rpm hard drive.. all you need is a sata connector and power connector to use these drives..  even with an ide->sata adapter you will see benefits. Instead of the computer waiting on the hard drive the hard drive will be waiting on the computer
0
25112Author Commented:
a)
thanks.. and to really know what SATA provisions I have, i should run any diag out there?

b)
regardless of immediately apply SSD, regarding the partition of the 1TB when it is used as external drive perhaps- let's say i am partitioning it into two (300GB, 400GB and leaving rest empty), are they truly considered 2 drives (practically speaking).. would it be a literal or logical separation. example: if partionA is having very high reads (300GB), will it affect partitionB 400GB)
0
nobusCommented:
>>   to really know what SATA provisions I have, i should run any diag out there?   <<  not needed, just look up the specs of your mobo, or pc model
0
David Johnson, CD, MVPOwnerCommented:
if partionA is having very high reads (300GB), will it affect partitionB 400GB)

Yes since it is only a logical separation.  For this reason I tend to think and talk about spindles and not 'drives' as a physical drive can have an unlimited # of logical drives and one is limited to the performance of the physical drive. For the same amount of disk space many smaller drives will outperform lesser larger drives. Don't forget increased head seek distances that can be caused by partitioning a physical drive.
0
David Johnson, CD, MVPOwnerCommented:
External hard drives usually result in poor performance due to the interface.. USB2 is really slow.. USB3.x is acceptable. E-Sata is acceptable (the problem being that the E-Sata connector was poorly engineered in a real world scenario)
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
25112Author Commented:
thanks for the helpful guidance. appreciated.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows 7

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.