External Hard Drive Shows Unallocated in Disk Management, Using TestDisk to Recover

My two home PC's (Win XP) recently stopped booting (probably a virus).  That's not the topic of this question (separate question already out there #25019929) but it's probably important background.  My first step of recovering those hard drives is to try to recover data, which I planned to do with my external hard drive (model WD2500032).  But it isn't recognized by any of the Linux bootup programs (KNOPPIX) that I'm running from CD on my broken computer, or even my work computer (which works fine - what I'm using now).  When connected to my work computer, it shows up in Device Manager, but has no drive assigned in Windows.  I tried rebooting with it connected (USB) - no change.  In disk management it shows as "Unallocated."  I ran the Western Digital Data LifeGuard Diagnostics and did a full scan, which came back all okay (SMART Staus = PASS).  I called Western Digital and they said if LifeGuard could have repaired anything, it would have done it automatically and otherwise they have no tools.  I've seen suggestions to use testdisk so I downloaded and ran that.  This is where I'm getting stuck -- I don't want to do anything to the disk to over-write my files.  I selected Analysis and it says "Structure OK" -- I selected "Deeper Analysis" and it comes back with the following screen:
Disk /dev/sdb - 250 GB / 232 GiB - CHS 30401 255 63
       Partition                           Start                End          Size in sectors
* FAT32 LBA                        0   1  1  30400  254 63         488392002 [My Book]

Structure:  Ok.  
Then is says what functions the arrow keys provide, and says I can use the Left/Right keys to CHANGE partition characteristics:  *=Primary bootabl  P=primary   L=Logical  E=Extended  D=Deleted.  However, when I press those keys it toggles only between *, P, and D.  

What should I do next?  Thanks to whoever can help me use TestDisk to recovery my data and, if possible, get the hard-drive back in working order.  

Who is Participating?
Glad everything is working well!  

You can do a number of things with that .img file.  The easy one would be to run testdisk against the .img file as if it was a drive.  Then you can use test disk to try and extract files, or you can repair the partition on the .img file too (since it was imaged with the bad partition).   One cool way to browse and recover from the .img file is to use a program called Autopsy, it is available on some security boot cd distros, such as backtrack:


There are some good tutorials on Autopsy/Sleuthkit here:


With Autopsy, you can mount the .img file, view all files on it, and also use it for un-deleting files through a web interface.  Honestly, testdisk or photorec will be the easiest ways to recover data, but I found Autopsy interesting if you have some time or desire to delve in a little further.

Lastly, if you wanted, you could use dd or dd_rescue to write that .img file back to a disk partition.  Since we fixed yours, you won't need to do this, but it is a nice exercise in backing up and restoring disk partitions.
Do you have any kind of encryption on your external drive?
mrjlalAuthor Commented:
No, I just have used it right out of the box and have saved files to it from these two computers.
mrjlalAuthor Commented:
But forrrest, you reminded me -- I thought I read somewhere to right click on testdisk and run as "administrator".  Well, this is my work computer and that's not an option (probably by design).  My only other option is my wife's computer but it runs Vista and I was trying to avoid mixing OS's (Testdisk asks if there are any files that were stored by Vista).  Do I need to run as the admin?
Are you planning on using testdisk to recover files, or are you going to try and repair the partition table?  If you have the extra disk space, I would recommend taking an image of your hard drive using dd from a linux boot cd.

It might take awhile to run, as it will copy every single block on your hard drive as is, so the resulting image will be the size of your HD.

You will need to boot to your linux cd, make sure your backup storage is mounted and writable, and go to the terminal.

The command will essentially be:

 >dd if=/dev/sdX of=/path/backup/backup.img

where "sdX" is going to be your problematic HD, and /path/backup/backup.img will be the destination of the HD image on your backup storage.


Once you have that dd image taken, if anything happens, you will be able to write back exactly what was on your HD before.

Have you reviewed some of the recovery examples on the testdisk wiki?


If you think that you are still having permission problems, testdisk should be included on you Knoppix CD, so you can try running it from there.
mrjlalAuthor Commented:
thanks for your comment.  OK, I purchased another hard-drive because I have nowhere to go with the data -- probably won't hurt to have two, anyway.  

I need to clarify a few things:
1.  You asked if I'm using testdisk to recover files, or repair the partition table.  I think both.  I'd like to have the hard drive as a storage device.  But I thought before I repair the partition table, I need to recover the files first -- does that make sense?

2.  I understand the instructions to copy an image, but the hard-drive I'm making an image of is an external drive, which probably doesn't matter, but I want to make sure I identify the devices correctly.  How do I find exactly the right name for the problematic HD, which you've labeled sdX?  Since I haven't been able to mount it, I'm not sure what to call it.  (i can send a picture of any screen if that will help).

3.  I did look at some of the examples on the Wiki -- they are just a bit over my head.  I need to read up on Fat vs. NTFS vs. RAW, etc. and then maybe the examples will make more sense.

The easiest way to identify the drives is usually by size, assuming you didn't get the exact same drive as before.  Did you get another external USB drive?

Are you working in Knoppix right now?  If so, open a terminal and run:

 >fdisk -l

and that should output all your drives.

You can also fire up Gparted from knoppix, and this will list all your HDs and partitions in a nice GUI.

It's been awhile since I've been in Knoppix, and I think you need to set a root password first. If you are getting nothing from the fdisk command, or it is asking for root or sudo permsissions for either the fdisk or GParted, you will need to set a root password.  To set a root pass, in terminal type:

>passwd password

Where "password" is the root password.  It doesn't matter what this is, since you are running a live cd, it won't persist.  Then use this to open gparted, or run:

>sudo fdisk -l

the sudo command will instruct the fdisk command to be run as root, akin to running a program as administrator in windows.

Linux will list SATA drives as sdX, and IDE drives as HDX, where X is typically the channel of the hard drive.  Some times this is hard to determine on the motherboard, so yes, it is good to double check.  

Hopefully you will be able to easily identify your new HD and your old HD based on their size from these commands.  One addition step is that you will have to identify the mount point of your new HD in knoppix.  It should hopefully automount for you, and you can identify it from the properties.

Feel free to post screen shots if you need.
I decided to fire up knoppix and refresh myself.  Here are some screen shots to help you out.

The first screen is the commands to get the root password set, and the output of fdisk -l, and then there are two screenshots in GParted of my two hard drives.  sda is the hard drive on my MacBook, you can see both the total drive size, and also all the partitions.  The HFS+ is just the partition type for Apple, and the NTFS is my Boot Camp partition for Vista.

sdb is my external usb drive.  You can see that it is a 300 GB drive vs the 111 GB of my system drive, plus you can see a large NTFS partition, and the two unknown partitions are for Time Machine (another Apple thing, no need to worry about them).

The fourth screenshot shows that my sdb is mounted on the folder "/media/sdb1".  That means that partition 1 on drive sdb is mounted to the folder /media/sdb1 .  Instead of using drive letters to attach partitions like Windows, Linux can mount them to arbitrary file locations.  Knoppix uses the /media/ folder, followed by the drive and partition number.

So, if I wanted to image my 111GB HD on my laptop to a location on my usb drive, I would use:

 >dd if=/dev/sda of=/media/sdb1/baddrive.img

You are using "sda" without any numbers, because we want to image the whole sda drive, not just a partition.  Besides, sda shouldn't have any partitions right now.
mrjlalAuthor Commented:
Russell, you are awesome.  I'm about to try all this stuff.  Thanks!
mrjlalAuthor Commented:
OK, everything going well -- I'm trying to fire up Gparted to make sure I know the mountpoint, but I'm not finding it on the knoppix cd.  It could be staring me in the face....
Look for the disk icon on the desktop, it should automount if it came partitioned already.  Is it another USB drive?
mrjlalAuthor Commented:
Oh, never mind -- it is previous obvious that the new hard drive is under a folder /media.  My knoppix has "PCMan File Manager" so it looks slightly different than your screen shot, but I think I can go forward with the dd command.
mrjlalAuthor Commented:
OK, I ran the command -- the curser dropped down one line and is sitting, the bad hard drive is spinning away (quietly) and my new hard drive (a Seagate BlackArmor device) is now clicking away.  I suppose this might take awhile?
Yep, every byte is going to be copied, so now the wait.  Especially since you are using USB drives.
mrjlalAuthor Commented:
Going back to your question -- I think you think I was saying I couldn't find the drive.  You are right -- it automounted.  What I can't find is Gparted itself.  But I could tell from PCMan that is was in /media so I was able to go without Gparted.  

hey! I think it might be done!  Rats, it has an error message, but maybe it's okay.  After my command, it now shows:
dd:  writing to '/media/sdb1/mybookimiage.img': Input/ouput error
2189936+0 records in
2189935+0 records out   [Comment from Mark:  note the difference in the numbers)
1121246720 bytes (1.1 GB) copied, 281.908 s, 4.0 MBM/s

I'm guessing there was probably 10-20GB of stuff on the bad drive ("Mybook").  Both drives are still churning away, however, so I'm inclined to wait until they stop.
DD is going to copy all data, even blank space,  so a little overkill, but it will catch all data
mrjlalAuthor Commented:
Well, I'm about to go to bed (I'm Central time zone in the Americas...aren't you in the Central Europe Time zone?)  It's cranking away.  Assuming I have an image when I wake up, I thought I'd ask about the next step.  Do I use testdisk to attempt to fix the hard drive, or otherwise wipe it clean and start over (either way, I need some help).  I will bump the points to 500 before we are done -- you are giving me awesome help.  
mrjlalAuthor Commented:
It's still going - I hope it's actually making progress, since it went back to a prompt status ready for a new command.  Is there a status command or something that I can use to check?  At 4MB/s, and a 250GB hard drive, I guess it could take around 18 hours which hasn't elapsed yet....
Is it actually still running?  You can browse directly to the .img file and have a look at the size, and also check to see if the .img file is growing at all.

You can also try the following command:

watch -n 10 killall -USR1 dd

Haven't used that myself, just got it of the internet.

Assuming that dd completes successfully, there are two different things we can try.  First, I think we should try to repair the partition on your hd with testdisk.  If that fails, you can always run testdisk against your .img file to extract data.
mrjlalAuthor Commented:
Well, it is still running.  I tried the "watch..." command and the screen shows:
Every 10.0s: killall -USR1 dd
dd:  no process killed
huh?  I hate that word "kill" :-)
mrjlalAuthor Commented:
24 hours now -- still clicking away.  Being paranoid I went browsing for more on dd status commands.  Found an interesting discussion http://www.linuxquestions.org/questions/linux-newbie-8/dd-copy-running-for-4-days-can-i-check-status-somehow-707463/
the suggestion of using the comman pgrep -l '^dd$' doesn't return anything.
Then i found this:  http://www.groupsrv.com/linux/about156711.html
which suggests using a command
~$ ps aux|grep dd\ |grep -v grep
to determine the process id of the dd command.  It did give me a lot to look at -- I'm going to post is separately because I need to paste it from the computer with Linux....
mrjlalAuthor Commented:
Sorry for the length but here is the result of the ps command previously mentioned.  I scoured for the dd pid and it's not there (I pasted into notepad and searched for 'dd').  
But my drives are still at it.  Wonder what is going on?

root 1362 0.0 0.0 0 0 ? S< Jan08 0:03 [cloop0]
root 1519 0.0 0.0 2764 1404 ? S<s Jan08 0:01 udevd
root 1594 0.0 0.0 0 0 ? S< Jan08 0:00 [khpsbpkt]
root 1605 0.0 0.0 0 0 ? S< Jan08 0:00
root 3056 0.0 0.0 1644 404 ? Ss Jan08 0:01
/sbin/klogd -c 1 -x
root 3058 0.0 0.0 1692 584 ? Ss Jan08 0:00
/sbin/syslogd -f /etc/syslog-knoppix.conf
root 3068 0.0 0.0 1640 584 ? Ss Jan08 0:00 acpid
101 3079 0.0 0.0 2604 1100 ? Ss Jan08 0:00
/usr/bin/dbus-daemon --system
104 3101 0.0 0.1 5760 3884 ? Ss Jan08 0:00
root 3102 0.0 0.0 3216 1068 ? S Jan08 0:00 hald-runner
root 3181 0.0 0.0 3280 1020 ? S Jan08 0:00
hald-addon-input: Listening on /dev/input/event5 /dev/input/event4
/dev/input/event1 /dev/input/event0
104 3192 0.0 0.0 2136 900 ? S Jan08 0:00
hald-addon-acpi: listening on acpid socket /var/run/acpid.socket
root 3194 0.0 0.0 3280 1020 ? S Jan08 0:00
hald-addon-storage: no polling on /dev/fd0 because it is explicitly disabled
root 3202 0.0 0.0 3280 1032 ? S Jan08 0:11
hald-addon-storage: polling /dev/hdc (every 2 sec)
root 3204 0.0 0.0 3280 1128 ? S Jan08 0:05
hald-addon-storage: polling /dev/hdd (every 2 sec)
root 3248 0.0 0.1 13884 2496 ? Ssl Jan08 0:00
/usr/sbin/NetworkManager --pid-file
root 3291 0.0 0.0 2776 1480 tty1 Ss+ Jan08 0:00 bash -login
root 3297 0.0 0.0 2776 1476 tty2 Ss+ Jan08 0:00 bash -login
root 3302 0.0 0.0 2776 1476 tty3 Ss+ Jan08 0:00 bash -login
root 3307 0.0 0.0 2776 1476 tty4 Ss+ Jan08 0:00 bash -login
root 3322 0.0 0.0 4108 1344 ? S Jan08 0:00
/sbin/wpa_supplicant -u -f /var/log/wpa_supplicant.log
root 3324 0.0 0.1 6136 3132 ? S Jan08 0:00
/usr/sbin/nm-system-settings --config
root 3325 0.0 0.0 2684 1248 ? Ss Jan08 0:00 /bin/bash
/etc/init.d/knoppix-startx start
knoppix 3327 0.0 0.0 2380 1024 ? S Jan08 0:00 su -l -c
export STARTUP=startlxde ; exec /usr/bin/startx -- -br -noreset -nolisten tcp knoppix
knoppix 3330 0.0 0.0 2984 1392 ? S Jan08 0:00 /bin/bash
/usr/bin/startx -- -br -noreset -nolisten tcp
root 3334 0.0 0.1 6200 2168 ? Ss Jan08 0:00
root 3343 0.0 0.0 2056 860 ? S Jan08 0:00
/sbin/dhclient -d -sf /usr/lib/NetworkManager/nm-dhcp-client.action -pf /var/run/dhclient-eth0.pid -lf /var/lib/dhcp3/dhclient-eth0.lease -cf /var/run/nm-dhclient-eth0.conf eth0
knoppix 3352 0.0 0.0 2740 788 ? S Jan08 0:00 xinit
/etc/X11/xinit/xinitrc -- /usr/bin/X -br -noreset -nolisten tcp -auth /tmp/serverauth.qvmaHMeYjF
root 3356 23.0 1.6 42056 33680 tty5 R<s+ Jan08 388:26
/usr/bin/X :0 -br -noreset -nolisten tcp -auth /tmp/serverauth.qvmaHMeYjF
knoppix 3407 0.0 0.0 3372 928 ? S Jan08 0:00
/usr/bin/ck-launch-session /usr/bin/dbus-launch --exit-with-session startlxde
knoppix 3441 0.0 0.1 4204 2452 ? S Jan08 0:00
/usr/lib/libgconf2-4/gconfd-2 4
knoppix 3448 0.0 0.0 0 0 ? Z Jan08 0:00 [ogg123]
knoppix 3450 0.0 0.0 2856 1236 ? S Jan08 0:00
/usr/lib/gamin/gam_server --notimeout
knoppix 3455 0.0 0.0 4620 568 ? Ss Jan08 0:00
/usr/bin/ssh-agent /usr/bin/ck-launch-session /usr/bin/dbus-launch --exit-with-session startlxde
root 3458 0.0 0.1 16676 2264 ? Ssl Jan08 0:00
knoppix 3533 0.0 0.0 2568 900 ? S Jan08 0:00
/usr/bin/lxsession -s LXDE
knoppix 3536 0.0 0.0 2960 680 ? S Jan08 0:00
/usr/bin/dbus-launch --exit-with-session startlxde
knoppix 3537 0.0 0.0 2480 848 ? Ss Jan08 0:00
/usr/bin/dbus-daemon --fork --print-pid 5 --print-address 7 --session
knoppix 3538 0.0 0.0 3040 1436 ? S Jan08 0:00 /bin/sh
knoppix 3539 0.0 0.0 3656 1152 ? S Jan08 0:00
knoppix 3542 0.0 0.1 4572 2364 ? S Jan08 0:01
xscreensaver -no-splash -no-capture-stderr
knoppix 3547 0.0 0.5 19300 11608 ? S Jan08 0:09 lxpanel
--profile LXDE
knoppix 3548 0.0 0.5 25440 10564 ? S Jan08 0:03 pcmanfm -d
knoppix 3557 0.0 0.5 19852 11112 ? S Jan08 0:00 nm-applet
knoppix 3611 0.0 0.3 14280 7388 ? S Jan08 0:06
/usr/bin/gtk-window-decorator --replace
knoppix 3612 4.5 3.5 80840 74192 ? S Jan08 77:22
/usr/bin/compiz.real --ignore-desktop-hints --replace --indirect-rendering ccp
root 3614 0.0 0.0 0 0 ? S< Jan08 0:00 [scsi_eh_0]
root 3615 0.0 0.0 0 0 ? S< Jan08 0:00
root 3705 0.0 0.0 0 0 ? S< Jan08 0:00 [scsi_eh_1]
knoppix 3808 0.0 0.4 17600 9572 ? R Jan08 0:00
knoppix 3809 0.0 0.0 2768 756 ? S Jan08 0:00
knoppix 3810 0.0 0.1 4612 3188 pts/0 Ss Jan08 0:00 /bin/bash
knoppix 3823 0.1 0.0 2684 1120 ? Ss Jan08 2:26
/sbin/mount.ntfs /dev/sdb1 /media/sdb1 -o rw,nosuid,nodev,noauto,users,umask=000,uid=1000,gid=1000
root 3833 0.0 0.0 0 0 ? S Jan08 0:00 [pdflush]
knoppix 4241 0.0 0.0 1852 816 pts/0 S+ Jan09 0:01 watch -n
10 killall -USR1 dd
knoppix 6585 0.0 0.1 4556 3132 pts/1 Ss 01:25 0:00 /bin/bash
knoppix 6831 0.0 0.0 2296 900 pts/1 R+ 01:44 0:00 ps aux
How big is your .img file right now?

So dd isn't running, but the monitoring thread is still going:

knoppix 4241 0.0 0.0 1852 816 pts/0 S+ Jan09 0:01 watch -n
10 killall -USR1 dd

You can feel free to terminate the pid 4241, as it isn't doing anything right now.

I am pretty sure that dd encountered an error and stopped after the 1.1 GB point.  I'm not quite sure why your drives would still be chugging away though.  In hindsight, we probably should have started off with using dd_rescue instead of dd.  dd_rescue is basically the same as dd, but doesn't exit if it encounters any disk errors.

The syntax for dd_rescue is a bit different:

%>dd_rescue dev/sda /media/sdb1/mybookimiage2.img

making sure to rename the drive letters or image name as needed.
mrjlalAuthor Commented:
OK, I unplugged my spinning external drives and ran the command (had to add a slash before the first "dev" -- that freaked me out) and it started running showing a count of the bytes transferred.  Cool!  That ran for a few minutes and then my PC completely shut down.  Like I unplugged it, only I didn't.  Powering back up now....
mrjlalAuthor Commented:
Oh, and i found no evidence of an image file anywhere!
Ok, so looks like dd wasn't doing the job for whatever reason.  Let's hope that dd_rescue will do better, and you won't run into any power issues.  Make sure that you switch into super-user mode with:

sudo su

before running the command,  and also double check that your new hd is being mounted as writeable.  You can just try creating a new folder or file on your drive to verify this.

And sorry about missing that slash on the earlier command.  
mrjlalAuthor Commented:
Hey no roblem on the slash -- I'm just glad I could figure out something for myself.  Thanks for the softball. :-)
Well, I had already started the dd_rescue before seeing your suggestion about the sudo command.  But I just checked and I can create a folder on the new drive so it appears to be mounted as writeable.  dd_rescue is humming along -- for whatever reason it shows to be about 5 times faster than the initial message from dd -- it's about 20MB/s, and it has transferred 111GB  -- almost halfway after 2 hours!
I don't know what that power thing was -- just crossing my fingers that doesn't happen again.  The CD drive was going at a high right non-stop during the first try -- this time it acts more normal, with intermittent activity.
mrjlalAuthor Commented:
OK!  I've got a 233GB image on my new hard drive!  
You mentioned before that the next step was to use testdisk to repair the partition of my drive.  
I've run testdisk and it doesn't exactly hold your hand.  I feel like one keystroke could cause it to erase everything in sight.  It's located the bad hd and said the strucutre is ok.  Do I now "Write partition structure to disk"
When you did your partition search, did it find a partition? Can you post a screenshot of testdisk output after you have done the partition search?
mrjlalAuthor Commented:
It says
Disk /dev/sda - 250 GB / 232 GiB - CHS 30402 255 63
Current partition structure:
No partition is bootable
 "no partition is bootable"
mrjlalAuthor Commented:
Sorry, I didn't mean to submit that.  Ignore that last "no partition is bootable"
Do the deeper search that you did the first time, and you should get this output:

Disk /dev/sdb - 250 GB / 232 GiB - CHS 30401 255 63
       Partition                           Start                End          Size in sectors
* FAT32 LBA                        0   1  1  30400  254 63         488392002 [My Book]

Do the quick search, hit q to back up, and then do the deeper search.

Once you are at this point, hit the "P" key to try and list your files, and make sure that you can see actual files, and not just garbage.  If your files look ok, then hit the left/right arrows, and change the partition from the boot (*) to primary (P), since this isn't a boot volume.

Once everything looks ok, then you can proceed to write the partition table.  This is only modifying the partition table, and not the data, and you also have a backup of your data, so even if something goes wrong, you should have your bases covered.

mrjlalAuthor Commented:
OK - had to take a break to go see "Avatar."  Now I'm doing the deeer search.  Looks like that will be done in about 1 hr 15 mins and I'll start the job to write to the partition table  (after checking the files).  
mrjlalAuthor Commented:
OK, everything checked out -- I could see files I recognized.  When I selected "write" it immediately cleared the screen and said "you will need to reboot for the changes to take effect."  Is that the change to make it primary or is writing the partition table that quick?
mrjlalAuthor Commented:
Well, I can answer my own question -- it was that quick!  My external drive "My Book" is now readible again!  Everything looks as it was -- I checked our a few files (mostly pictures) and they are intact.    Thanks a million!
Any final words of wisdom?  I do have a question about the img file.  If I want to access that, how do I do that?
Bumping up the points to 500.  I'll accept your reply as the solution.  Thanks again!
mrjlalAuthor Commented:
FYI, i'm opening another question on my laptop drive:
Would love to have your guidance if you are available!
It's similar but there are some differences (mainly that it definitely has bad sectors) so I'm anticipating that I'll need som ehelp.
Also forgot to mention that you can mount the .img file itself as if it was a partition in linux, and browse it directly.

mrjlalAuthor Commented:
I really appreciated the extra effort to load screen shots to help me understand what I was seeing.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.