how can i defragment my mac os x hardrive without norton???

enginerd
enginerd used Ask the Experts™
on
i have recently gotten jaguar, mac os 10.2, and am very happy with it. however, i cannot now and never have been able to figure out how to defragment my hard drive. I do not have the norton utilities on cd, i have downloaded them and tried them without having a cd version and it does not work. I was wondering if there is anything else that i can do, is there another program or do i have to breakdown and buy norton utilities?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
To defrag your startup volume you do need to boot off a CD. You can do that with Norton Utilities which youre going to have to break down and buy. You should have a copy of norton anyway.
Commented:
You can defrag your hard disk with Norton Utilities, but I wouldn't recommend it. To understand why, consider two things: (1) What the word "fragmented" means, and (2) the operating system itself.

There are two kinds of fragmentation: file fragmentation and volume fragmentation. File fragmentation refers to what happens when a file gets broken into lots of tiny pieces so that every little nook and cranny of the hard disk can get used.

Volume fragmentation (the kind that Norton looooves to point out) just means that all of the files aren't crammed together in one (mostly) contiguous part of the hard disk. File fragmentation can be bad--volume fragmentation: not really.

In Classic Mac file systems (HFS and HFS+) and DOS/Windows file systems (FAT, FAT32, NTFS), an "optimized" disk means that not only are files themselves contiguous, but all those files are jammed together in a tidy block. It seems efficient and neat, however... from the second that a disk has been defragged, it becomes more and more inefficient. All that packing together of data means that the moment a temp file is deleted, part of your business proposal is probably going to get stuffed in it's place--instant fragmentation.

In UNIX, however (like MacOS X's underpinnings), typical filesystems use high degrees of _volume_ fragmentation to combat file fragmentation. Anybody who has had some sectors go bad on a hard drive knows that lots of file fragments mean the possibility of a much greater number of corrupted files.

In short (oops, too late). Defragging in the UNIX (and now MacOS) world is more likely to do harm than good.

To combat the potential problems there, there is an OS X Speed Disk profile floating around. As far as i know it was released by Symantec.
Ensure you’re charging the right price for your IT

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Author

Commented:
why then if it would do more harm than good in mac os x system does norton offer, and suggest defragmenting or "optimizing" the hard drive.? the way that a hard drive works it seems to me that fragmentation would only slow down the hard drive at a very minimal rate, especially with the speed of the reader arm and the RPMs of  of the disks now days. So, just to confirm it is best with the new mac os x system that i not defrag at all???
It partially depends on what youre doing. For example, if youre writing large ammounts of video to disk you want a large chunk of your disk clear. Then it can write that file without jumping the heads all over the place, and slowing the write process down. For general everyday use you probably dont need to defrag.

Commented:
Sorry for the delay in answering... To answer the question of "why would Norton offer this":

Well, as weed suggests, there are some cases where having huge tracts of empty hard disk space is essential (such as saving uncompressed video directly to disk or, to a lesser extent, ripping audio)

The bigger reason, in my opinion, is marketing. It's another 'feature' for Norton to offer, one which Windows people espouse doing regularly. If Norton didn't make this option conspicuous, it might suggest a "feature gap" between the Mac and Windows versions... something that we Mac folks see plenty of already.

I would agree with you that any speed gain from optimizing the disk would be pretty negligible. (depending on what you're doing, of course. If you're dumping raw video on your drive, you may want to eke out every bit of performance you can.)

To answer your question about "more harm than good," the most commonly-used "optimization" profiles tend to focus a lot on volume fragmentation which, as I described above, can actually lead to greater file fragmentation. If the hard disk becomes damaged or corrupted (even just a little), there is the potential of more damaged files, with greater difficulty of recovery. Unix filesystems actually use a large degree of volume fragmentation, which is less conducive of file fragmentation.

So the briefest answer to your question of "to defrag or not to defrag" that I can give is:

(1) You probably won't gain any benefit
(2) It could even lead to greater trouble if you experience data corrupton

Happy OS Xing!

Commented:
Excuse me if I am wrong, but by default, Mac OSX installs into an HFS+ partition on Apple hardware. Unless the user specifically repartitions their hard drive and creates UFS partitions in preparation for Mac OSX, the filesystem is the same as that used by older Mac OS versions. Eventhough the Mac OSX is BSD and Mach to the core, the filesystem used by default is not from the unix system and does not inherit any of the strengths of the unix file system.

That being said, defragmenting an HFS+ partition is usually a good thing to do on a regular basis for performance reasons. I haven't used a Mac in awhile, so that may have changed with the advent of OSX. Take the advice of others with more recent Mac experience, but I wanted to clear up the filesystem issue, I hope?

If you installed Mac OSX into a UFS partition, then defragmenting would be a waste of time since the unix file system handles file fragmentation just fine.

Commented:
Yup. 100% HFS+ from the factory. Otherwise, you wouldn't be able to boot into OS 9 (though you could still use the Classic compatibility environment). Also yup that HFS+ has no inherent benefits of the Unix File System. Disk defragmentation utilities report high degrees of volume fragmentation (though not file fragmentation) on hard drives in Macs straight from the factory and on drives just defragged and then upgraded from 9.2 to OSX.

Apple doesn't have anything to say about this (at least that I've been able to find), but it seems to be approximating one of the inherent benefits of UFS.

In short... I _still_ contend that (even on HFS+ volumes) any defragging on MacOS X is of dubious benefit, at best. It will be likely to increase file fragmentation and offer little (if any) perceptible speed improvement.
One explanation I've seen for finding high degrees of volumne fragmentation on drives right from the factory is that the image used to load the drives was based on a drive that had the OS 9 and OS X files separated by a lot of empty drive space - hence the volume fragmentation.  Sound logical (I'm no expert)?

Commented:
I have run my Norton Utilities from the Classic Mode where it shows that my harddrive is Severly fragmented. About 10% into defragmentation the harddrive freezes up and I have to reboot. After reading your comments above, should I presume that even though it suggests my harddrive is severly fragmented that I should ignore the whole process and continue without worrying that something terrible will happen to my harddrive? All I want is re-assurance that I should not worry about the state of fragmentation as suggested by my Norton Utility program.
Fragmentation wont cause anything bad, it just slows you down.

Commented:
I know this is only partially relevant, but I am still doing digital audio work in OS9 and am wondering if and how (i.e., with which software) I should defrag.

Any advice would be much appreciated.
Norton Speed Disk.

Commented:
Is there ANY benefit at all of me defragging my HD on a dual gig G4 running OS 9.2 which I've had for 2 years now. Btw anyone ever mention that Apple OS fathered all WIn OSs in a chatroom and watch the ensuing melee of irate PCers?   lol   Thnx to anyone responding to my Q.

Commented:
Oh yeah, forgot, any way to allot more memory to the Mac OS?
You should be running OS X on a dual GIG G4. For LOTS of reasons.

Commented:
I will in the future, but along with classic 9. I'm a musician and all of my studio software is OS 9. I'm just wondering if defragging would be a help or hinderance.
It cant hurt.
Norton is rubbish.  DiskWarrior 3.0.1 is far better in it's operations especially for defragmentation.  Go check out http://www.versiontracker.com/macosx/  to see what end users have to say about it.

Franky I have neve had anything but problems with Norton under 10.x.

And yes, defragmentation is a good thing to do on a regular basis ( say monthly depending on usage) because it is the writing/ reading to the harddisk which is also degraded ( i.e. the more fragmentation, the more work the HD has to do ( hence if you here the HD working overtime on your Mac it's time to defragment :)

In my experience though MS XP is worse when it comes to fragmentation especially with 'Restore' enabled.  It seems to really screw over the OS on a frequent basis.
In OS X.x all you have to do in boot into single user mode and run fsdk -y, in panther, fsdk -f and that's it.
Not sure where you got your info, but there is no such thing as fsdk. What you're probably talking about is FSCK, and no, that doesn't defragment ANYTHING.

Commented:
An often overlooked method of dealing with a fragmented disk is to use a free tool like CarbonCopy and  'clone' a bootable disk to an external firewire drive, then boot to that disk and make sure everything is running and your data is intact.  Next, run Disk Utility and erase your original disk, then rerun Carbon Copy and 'clone' a bootable disk back to your original disk -- viola - defraged with none of the liabilities of trying to defrag your production disk while it is running.

lystrata:  I see you like living dangerously... ;)  Personally I would ever erase my hard drive unless it was totally necessary.  This is especially dangerous given the flakeness of utilising Firewire devices under 10.3 ( given that many people suffer total failure of their Firewire device!).  Replacing the whole disk from a backup would be a good way to repair any potential bad blocks on your HD however ( You could do the same thing with Retrospect come to that).

FYI: FSCK = the unix File System Check utility. But with Journaling on as default (10.3.x)  there is little reason to do this. In fact according to Apple, running fsck on a journaled disk will generate false errors. See http://docs.info.apple.com/article.html?artnum=107250

Just to clarify my earlier post, it really all depends on how hard the HD is working ( i.e. lots of opening/ closing files; working on large files a lot, files which change a lot i.e. emails; and how much free space you have as the nearer you get to full the more the remaining free space gets fragmented).  If you just use it a home for light work then it would be a lot less frequent.

Commented:
I'm just going to throw in this explanation from mac.
http://docs.info.apple.com/article.html?artnum=17933

Commented:
I must say that I notice the effects of fragmentation most on laptops specifically because users tend to install and uninstall data more often and because everything -- applications and data all end up in the same partition.  On my UNIX workstations and Servers, I setup up the partitions and segregate static and dynamic data.  When I have more time, I am going to look into taking over more control of the Apple parititioning up front (if that is even possible) to lessen the impact of fragmentation on the system areas.  This is a real weakness of the current Mac Install.


With regards to LuNa's comment above,  I do no see cloning a disk as necessarily dangerous.   I have been a sysadmin too long to look for trouble.  I specifically see ANY defrag utility as dangerous.  The clone process (1) copies data non-destructively from a working/production disk to an avalible disk (it does not have to be firewire, in the case I used  -- it was -- it could easily be another drive on the chain), (2) the cloned disk can be throughly exercized, checked, etc. to assess its viability, (3) with a now know good disk, I can with a totally clear conscience "erase" the old fragged production disk, (4) I restore the defragged disk with a clone operation.  If for any reason there is a problem during the cloning process, I risk nothing -- the same can hardly be said for a disk defragmenation utility.

I have used this technique on my laptops, G4s and G5s, Xserves with out a single failure.  I am a very careful man -- I do need to make work for myself -- my users do more than enough of that.  I am just very distrustful of solutions like defragmentation utility when there are more tried and true methods.  
Just mentioning that Panther (10.3)  has a new feature, automatic file defragmentation and hotfile clustering.

This is from John Siracusa,  who imho gives a balanced view on this new feature:
http://wwwww.arstechnica.com/reviews/003/panther/macosx-10.3-5.html

quote

"A file is fragmented when the data it contains is spread over the disk instead of being contained in a single, contiguous set of disk blocks. Since moving the disk head from one location to another is one of the slowest operations possible in a hard drive, fragmented files take a disproportionately long time to read. It is difficult to avoid fragmentation entirely, especially for files that grow slowly or without bound. Defragmentation collects the data from a fragmented file and moves it to a contiguous region of free space on the disk (assuming there is one large enough).

Under certain circumstances, Panther will defragment files automatically. You can read the thread in Macintoshian Achaia for all the gory details, but the short answer is that fragmented files less than 20MB large on journaled HFS+ disks are defragmented automatically in Panther when they are opened, whether for reading or for writing.

This initially strikes me as a slightly wrong-headed optimization. First, it seems odd that up to 20MB of data could be written to the disk as a result of merely opening a file (note: not actually reading or writing any data either, just opening). Second, if fragmentation is such an issue on HFS+ disks, a better solution is to improve the allocation policy or, better yet, replace HFS+ with a better file system (which Apple is doing anyway as part of its filesystem metadata enhancement project . . . right? Right?)

Nevertheless, I am more than willing to give Apple's HFS+ developers the benefit of the doubt and assume that this optimization was created as a result of performance profiling, and was tested and shown to provide a net performance improvement in typical usage.

Hotfile clustering, which is also detailed in the forum thread, is another optimization that I instinctively view as premature. Panther tracks frequently accessed files and slowly migrates them to a "fast" portion of the disk. This only happens on the boot disk, and there are constraints on the number and size of the files that are eligible. As a side-effect, transferring files to the hotfile area also defragments them.

It seems to me that this optimization tries to out-smart the normal file caching mechanism. But as with the automatic defragmentation, I'm willing to trust that this change has proved itself in testing. I can imagine that it does actually play well with normal disk caching, since presumably all of the very frequently accessed files will be in a conveniently cacheable contiguous cluster (say that three times fast) in the hotfile area.

The proof is in the pudding: Panther thrashes less than Jaguar on the same hardware in my experience. Admittedly, this could be attributed to many things, not the least of which is the fact that this is a freshly installed OS. But remember that I did a normal "upgrade" install rather than an "erase and install", so I'd like to think that these two new features really are at least partially responsible for the decreased disk activity."

endquote


Commented:
I spoke with a Microsoft tech rep today about a different issue. However, defragmentation came up in the conversation and he maintained that defragmenting a hard disk "more than once a month" would "eventually damage the disk."

According to the MS technician frequent defraging (every night) wears out a drive like excessive milage wears out a car.
I'm no hard disk expert, but assume most hard drives today can handle the use. There are several issues here:

- physical "wear and tear" from use - some cheaper disks probably have a lower mean time before failure, but in general hard disk defragmentation should not be a problem (some hours of reshuffling at the most if done seldomly, some minutes of activity if done often). On a file server (not speaking mac spefically) I would go for defragmentation. On a non-Panther computer (e.g. Windows I would defragment on a regular basis if file integrity and access speed is important). On a Windows workstation or a file server my best bet has been Raxco PerfectDisk since it is geared for big disks and has a lower processing overhead than for example DiskKeeper.

- file integrity - files spread over many physical locations just sounds like plain scary. More fragments, more chances of faults in my opinion.

- response times - though new harddisks are fast, it would seem to me that fragmented files takes longer to read and write. Plain simple logic?

- mac advice? Well, since the Panther OS helps us out with defragmentation of accessed files of 20 Mb or more, and we are talking about a journaled file system at least on a file server, it seems Norton defragmentation is old-style, best for OS 9. Must admit that I enable journaling on all machines, since I believe it helps the OS to keep track of things (IMHO). I would avoid Norton in general on OS X - I have at least had stability problems with their utilities under OS X. DiskWarrior from Alsoft is a great tool for checking disks and rebuilding directory information.
Hmmm... mac advice and defragmenting under OS X Panther...

Must admit I got it muddled up in my last posting. Panther only defrags files under 20 Mb when accessed. So this does not solve fragmentation of media servers with lots of pictures/big files.

Any suggestions from anyone else?

Commented:
As I understand the 'defrag or not defrag?' issue, it only becomes really important to have a non-fragmented drive if either

i) you need to be writing large files fast (such as in audio/video recording, as has been gone into above), or

ii) if your hard drive is at all approaching being full. This second issue is more significant, because osx keeps a file allocation 'map' for each file (it's a big database) on each volume. The average osx installation has at least 50,000 files before you even start adding your own apps and documents - most people's startup volumes are more likely to be around the 100,000 file mark or higher. Now if a lot of those files are fragmented, that very quickly gets to a huge number of file allocation pointers (I don't know what the proper technical term is for this - I mean the record in the 'map' that tells osx where each fragment of each file is kept on the volume).

Once your volume is getting at all near to full, there is the possibility of disaster, because every time osx writes a new record in its allocation map, it needs a minimum chunk of contiguous disk space to do so. If it can't find one (because the free space is fragmented all over the place), it throws away an existing allocation pointer in order to be able to write the new one (insane but true). Disaster! At the very least, you lose a doc file. At worst, it is a system file whose location is lost, and your OS goes down. This isn't abstract theory - I've had this happen on a 30Gb drive that apparently had 5Gb of 'free space' on it, but had to be reinstalled from scratch, because it turned out that the 5Gb was in something like 12,000 fragments, and the file allocation map (of around 250,000 files!) got trashed.

I use TechTools Pro, which will not only defragment the files and free space, but can also 'optimise' the file allocation maps to make them more efficient. This speeds up the use of osx, and makes the above disaster much less likely to happen.

I agree that it's not something to be doing every night - it's very hard drive intensive, as well as taking up a fair bit of time - an hour to defrag and check a drive, against those little bits of time you save because it's then running faster, is a factor to weigh up when deciding how often to defrag. But it's really worth doing if you keep your drives at all close to full - Apple recommend keeping at least 25% free space on each volume.

And of course, I backup my drive before optimising it, don't you? ;)

Commented:
Get disk warrior. It never fails. Fixes just about any problem with a hard drive and system and works 95% of the time. I run two mac network and I couldn't live without it. Trust me you will not be disapointed.
Re Disk Warrior ... get it! It is the greatest!

Commented:
If you want to defrag a Mac volume, I would recommend Alsop's Disk Warrior as well.  I work as a Sys. Admin. at NASA Langley, and it's what we use a lot on our Macs out here.  You can boot to the Disk Warrior CD and it will remove overlapping and interlocked files that an "fsck -yf" in single-user boot mode (holding down Command + Option + S at boot-up) won't always remove.  It will allow you to build a "before" graph of the fragmentation, and you can compare that to how it looks afterwards with an "after" graph.  It also has some nice diagnostic utilities for all of your hardware (CPU, RAM, video, etc.) to use when booted to OS X after installing the program to the hard drive and running it there if you chose to do that as well.

Commented:
1) I'm amazed at the number of people here who believe that DiskWarrior deals with disk fragmentation. It doesn't. It simply optimises the directories. The remainder of your HD will remain  just as fragmented as it was at the beginning of the process.  Disk Warrior what it does extremely well but is is essentially a directory repair tool, not a defragmentation utility.

2) The "accepted answer" in this thread is partly right, but also seriously wrong in some respects. As yoxi has pointed out, in a balanced and accurate reply, free space fragmentation can become a very serious issue with Macs running under OSX when disk space becomes tight. OSX needs large hunks of contiguous free disk space (free space all in one "lump") for the writing of extents directories and swap files and the like. On my own Powerbook, at a time when it  had 5 gigabytes free, I found that the largest piece of contiguous disk space was well under 100 megabytes! This is very dangerous territory to be in.

3) While the built in defrag routines of Panther and Tiger work very well for small files, they don't apply at all for files over 20 megabytes. A file larger than this, written on a fairly full HD is likely to be scattered all over the place. (burn a DVD, do some video editing work , wok with some big graphics, even rip an audio CD and you will have created far larger files than 20 meg which you will probably immediately delete. Repeat the process a few times and the level of free space fragmentation will increase even further). At the very least you will experience slowdowns as the Mac begins having trouble writing swap files. At worst you will experience heavy crashes, if you don't have a large amount of empty real estate on your drive.  Your drive will also have to work much harder, with increased failure risk, and increased heat generation.

4) When crashes associated with these sorts of problems occur they than can be catastrophic. In my own case I had to actually rewrite some information manually with  Nortons fairly frightening "Disk Editor" to recover. Utilities such as DiskWarrior, TTP and NDD simply can't be used to recover from such situations. In most cases it will , at least, require a complete erasure of the HD and in some cases even this may not be sufficient.

5) While users with todays very large HD's may not run into such issues for quite some time, anyone running OSX on a disk of 80 gigabytes or less should at the very least be aware of it. I regularly find I have to warn people on the Apple discussion boards about these sorts of dangers when they install Panther or Tiger on iMacs and Powerbooks with puny little drives of 20 gig or less.  Any boot drive with 10 gig free space or less running these OS's should be defragmented regularly, and it is a good rule of thumb to NEVER let your free space get down to less than 5 gig (and considerably more if you regularly work with large files).  

So , to summarise:

YES, Panther and Tiger do a good job of defragging small files. NO, this doesn't mean that defragmentation no longer matters under OSX, especially if you only have a small or , full, drive.

YES, DiskWarrior is an excellent directory repair utility. NO, DiskWarrior is NOT designed to deal fix these sorts of issues.

I , personally, regularly  use an excellent "specific purpose" utility called "iDefrag" to deal with disk fragmentation on the little 40 gig drive on my PB12. Works a treat and doesn't have the same space overheads as TTP's massive 4 or 5 gig disk partition required for use of its full capabilities.  I always have a current back up when running it , though, and I always use either DiskUtility or DiskWarrior to repair directories before doing so. see http://www.coriolis-systems.com/iDefrag.php   The current version is fully compatible with Tiger (unlike Nortons Speed Disk and all but the very latest version of TTP4).

Cheers

Rod

I see absolutely no reason to defrag a mac running OS X. I myself have NEVER done it and have had no problems whatsoever. So, to answer your question, you don't.

Commented:
If DiskWarrior wasn't defragmenting the drive when the utility is ran and the graph displayed when using that function (granted it's been a while since I've ran it and don't remember what they call that function within the program to click on), what would be the purpose of showing the individual blocks being re-organized and re-colored?  I'd have to wonder that very seriously if it wasn't actually defragmenting anything, as it wouldn't have a point to be doing that if all it were doing was optimizing the drive.

I do have to agree with yoxi - I've seen a Mac or two that wouldn't boot after their drives became close to full (less than 5 GB free), and had to take another drive, image it, and put the old data onto the new drive, when nothing was wrong with the old drive except for the OS boot issue, which we didn't find out later until running some disk diagnostics.  I've seen drives that had 20 GB free one day have only 5 GB free the next day, and not because they had 15 GB of files they copied onto the computer or created themselves.  The only thing I can come up with is yoxi's theory of disk fragmentation, which makes complete sense the way he described it, to me.

Commented:
Diskwarrior is simply defragging the directories / catalog files, navyjax2. (which is a worthwhile thing in itself, but doesn't help with problems of free space  or regular file fragmentation). It is not uncommon, in fact, to get a message from DW that the directories can't be defragged because there is not enough contiguous (ie all in one block) space for it to perform this function , if the drive is close to full.

One difficult thing with the "defrag or not" arguments is that it is difficult to pin an absolute figure on the point where the problems occur. Large drives, for example, need more contiguous space for various essential catalog and extents files, than smaller ones.  User factors enter into the equation , too. If you regularly deal with DVD burns or video generally you will need more free space, and more ferquent defragging , to avoid problems than if you simply run a word processor and deal with email.  A good rule of thumb for most users, though, is to never let your drive get to the point where you have only 10%  free space, regardless of drive size, and to defrag regularly once you get below about 20% free space. Video / DVD pundits may need to double these figures, though.

Cheers

Rod



Commented:
Well, at least you agree with me that DW is a defragging tool now, if only the directories & catalog files :) .  Personally, I would defrag anytime I saw a lot of disk space going away (without actually installing programs or creating/saving files) after seeing how full a Mac OS drive can get without it, just from sitting there, having system files being written back & forth, and then seeing the amount of disk space that is restored following a defrag.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial