Hard Drives and Sensitive Data ?

ok in that case then what exactly does an unconditional format do ie :

format C: /u

If it was designed so that data could not be retrieved then why was it made if it does not work ?

Now I see:

This unconditional /u flag does something windows nt, 2k, XP don't have, I was already wondering about that flag:

See for yourself:

Previous to DOS 5, the format program completely formatted a floppy losing all data if previously formatted. As of DOS 5, Format creates a "safe format" by saving additional data on the disk. This takes a bit longer but allows the disk to be unformatted. Since you will not have to unformat blank disks, you can format "unconditional" and speed up the format process with the /u switch:

format a: /u

--
So the /u flag was used to prevent the creation of a special file, placed at the end of the medium, that contained unformat information.

But /u was never invented to wipe out the medium.

So it's of no use, today.

Tolomir

From the FAQ on the link I posted earlier:

Secure Deletion of Data from Magnetic and Solid-State Memory
http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html

Data Remanence in Semiconductor Devices
http://www.cypherpunks.to/~peter/usenix01.pdf

Basically, even if you write all 0's to a disk once, data can still be recovered.

Gecko,

add on to others, format /u - just says to format the drive unconditionally - ignoring any exception.  It's for writing another layer of bits on top of what exists....  All it does is to create another layer of bits on top of another one the other layer exists but weaker..... It's kind like this:

You write on a stack of paper (you press hard and make indentation impression on the layers below - note the word layers)
After you tore off the top sheet - the impression are still there for many layers of paper (just less visible the deeper you go),
As you write on the next sheet you make impression on the sheet below, the new impression is more visible but the first impression is still there...... Yes, overtime it would be very very difficult to read all the layers...... it would be difficult (but not impossible) that number of layers is seven - that's why the US DoD secure erase is seven passes.....  

cheers

...there is also a difference in what we mean by "format" today as opposed to not-so-many years ago (80's and early 90's).   Originally a "format" actually wrote data on every track for the entire track.   This included sector ID data, inter-sector gap data, and a string of 1's & 0's to actually initialize the magnetic structure on the disk.   So a complete format at that time was more "complete" in the amount of data that was written; and had a more destructive effect -- although the magnetic layer recovery methods that resulted in DOD standards for erasure were still an issue.  Modern disks use a separate servo track, and the actual low-level formatting is done at the factory.   There is, in fact, no way to "low-level" format a modern IDE/ATA disk.  The "formatting" that Windows does is simply writing structural data for the file system selected.   None of the sector marks, ID data, inter-sector data, etc. is written -- it couldn't be even if Windows "wanted" to.

Another point touched on above, but I didn't catch your question when I commented above:  the MBR is not even part of a Windows format (a true low-level format would overwrite it, because that's a DRIVE-level format).   Windows is formatting a PARTITION -- not a DRIVE.   The partition may "seem like" a drive, if the drive only has a single partition -- but the MBR is a record on an area of the disk that is never assigned to a partition.  My comments above regarding a DOD 5220.22-M compiant erase of an MBR would only be true with a sector-level security erasure tool that included an erasure of the MBR sector.  A tool that just does a 5220.22-M compliant erasure of a logical drive (i.e. a partition) would not even touch the MBR.   On the other hand, if a 5220.22-M compliant erase has been done on the data, a fully intact MBR is useless -- it simply identifies where a partition existed:  if all of the directory information and data has been securely erased, it's essentially unrecoverable (except by the methods discussed above -- which probably wouldn't even work with a 5220.22-M compliant erasure).

So from what you said here :

"...there is also a difference in what we mean by "format" today as opposed to not-so-many years ago (80's and early 90's).   Originally a "format" actually wrote data on every track for the entire track.   This included sector ID data, inter-sector gap data, and a string of 1's & 0's to actually initialize the magnetic structure on the disk.   So a complete format at that time was more "complete" in the amount of data that was written; and had a more destructive effect -- although the magnetic layer recovery methods that resulted in DOD standards for erasure were still an issue. "

And also some other things that were said , if myself or someone else used the hard drive scrub and or in combination with dban from the sourceforge site that leew mentioned as well as doing and fdisk, a format then an un conditional format and then formatting it and using fdisk or the repair console to re write to the mbr ( even though you said that the mrb is useless when the data has been securely wiped ) it would be a hell of a lot harder to recover any of the data ?

Anyway thanks for the info everyone !!

The response has been great and I will defintly split points when I get back this evening as I have to go into work today unforantly.

mrb = mbr , just a typo that I realised I made in the above post :P

First, your quote made me see an error in my post:  "...every track for the entire track..."   should have been "...every sector for the entire track..."

But the answer to your question is YES -- if you used Hard Disk Scrub (or dban) it would be effectively impossible to recover the data -- and absolutely impossible without an extremely expensive laboratory, an exceptionally talented technician, and even then a very large amount of luck.   As I noted above, even if you have this equipment & talent, the success rate is VERY small (and to my knowledge those that were successful were with disks that were NOT erased to 5220.22-M standards).

You don't have to bother with the fdisk and/or repair console -- as I noted, the mbr is useless at that point.  It simply identifies the partitions on the disk -- which no longer contain ANY data.  In fact, I believe Hard Drive Scrub & dban both have options to include the MBR in their secure erasure - so this may be a moot point (I've used them both, but it's been a while).

...one distinction you should note between Hard Disk Scrub and dBan.

Hard Disk Scrub scrubs (to 5220.22-M compliance) all free areas of a disk.   So if you've erased something, emptied the recycle bin, and then do a Hard Disk Scrub it is GONE -- nobody's ever going to recover it.

dBan is booted from a floppy & erases an entire disk (to 5220.22-M compliance).

So which one is better depends on your focus:  dban is easier to simply "kill" an entire disk;  Hard Disk Scrub is better if what you want to do is be sure files you've deleted are absolutely no longer accessible.

While a hard disk scrub tool will kill each single bit (rewrite it with different patterns) format c: /u is deprecated and did NOT kill every bit on the harddisk, but instead simply did't save unformat information. In that state every information was still on the harddisk readable to a simple diskeditor, just the directory structures were lost.

Btw. that unformat information was a nice tools from pc-tools called "mirror.sav" microsoft had bought. With windows95 we got a new filesystem (with long filenames) and that mirror.sav information was no longer compatible.

So in format the "/u" was dropped.

Tolomir

Thanks for all the great replies , very informative. The only thing that would be great is if someone could post a good website to explain the "DOD 5220.22-M (the "gold standard" for a secure erasure)."

So I can go away and do more reading to understand how it works or what it does etc, or even what or how they came up with the name "DOD 5220.22-M".

Kind regards

Gecko

Your wish is our command.

Quick overview:

http://www.killdisk.com/dod.htm

The whole enchilada (see chapter 8):

http://www.dss.mil/isec/nispom_0195.htm

A bit of light reading:  http://www.dtic.mil/whs/directives/corres/pdf/522022m_0195/cp8.pdf

Note in particular pages 8-3-5 and 8-3-6

@ willcomp - "Your wish is our command."

Alrighty then - I wish for a billion dollars, lots sports cars , my own place with a jacuzzi / hot tub , my own cinema, etc, you want me to carry on lol ?

@ garycase

Thanks for the URL and page refs , very much appreicated.

Sorry, you ran out of wishes.  Note that HDDs containing Top Secret information are destroyed.  They don't take any chances.

Yep -- I had quite a few destroyed when I worked in that environment :)

There was a discussion several months ago on one of the help sites where I participate related to this question.  Crux was whether or not a HDD could be read after a multi-pass wipe.  I'm trying to recall gist of discussion.  Apparently DOD spec is based on a research paper prepared several years ago that has not been supported by actual data retrieval.  Writer postulated the weaker underlying residual data layers could be read by extraordinary means but offered no supporting empirical data to support theory.

Theory was incorporated into DOD spec and taken as gospel ever since.  There are researchers that refute the theory and believe multi-pass wipes do render data on a disk uncoverable.  DOD has lots of money (yours and mine) to spend, most of us don't.  We can all feel safer knowing Top Secret data is really destroyed except for that carried out in someone's pocket.

Incidentally, 0s and 1s are written to drive since drive stores data in binary format.

I've had an opportunity in past years to use some of the equipment that actually does the retrievals.  As I noted much earlier in this thread, even with the right equipment the likelihood of retrieval is very small -- but NOT zero.  It has been done -- it's not just theory.  But from ANY pragmatic perspective, it's essentially impossible.  Especially if the erasure has been done with a 5220.22-M compliant program.  Don't understand your last comment (I know how data is written).

Wasn't directed at you.  Assumed an old goat like yourself would know :-)

Looking back through comments, found that LeeW had linked to paper in question.  Read through it and saw that author had posted an epilogue addressing modern large hard disks rather than old MFM/RLL small capacity drives.  Basically agrees that there is almost no liklihood of retrieving data from a modern drive after a couple of overwrites.  Does appear that they had empirical evidence to support their conclusions.  Didn't recall that being the case.  I'm old and forgetful too (and if you think I really believe that, got some prime oceanfront real estate in Memphis).

Since you had hands on, were you able to retrieve any usable data after a multi-pass overwrite?

If I told you I've have to kill you :) :)    (I was working with a 3-letter agency at the time)

... but the answer is that what I said earlier is correct:  you CAN recover SOME data -- but certainly not all; and the success rate of doing even that is VERY SMALL.   It is safe to conclude that if you do a 5220.22-M compliant erasure you can consider it essentially impossible to recover the data.