copy drive contents to a blank drive

Posted on 2006-11-01
Last Modified: 2008-01-09
I'm trying to copy the contents of a hard drive to another drive.  The source drive is dying (HW failure), but I'm able to read it via an external USB enclosure using Knoppix.  I don't know *nix that well, but I'm *fairly* comfortable working with it.  The plan is to install Windows XP on a fresh hard drive then, copy this backed up data (if it ever backs up) over to new hard drive.

I'm trying this as root, because I figure that gives me the most privileges.

so far I've done:
#chmod 777 source
#cp [-f] [-r] source destination

(I've tried both -f and -r independently and together)

The problem is that I keep getting errors such as:
cp: omitting directory ....

Please help - I've been working on this for far too long.  I'm getting burnt out on this one, and my client wants this done.
Question by:Geisrud
LVL 27

Expert Comment

ID: 17855055
cp -Rp source destination
LVL 14

Author Comment

ID: 17855105
after running command: #cp -Rp source destination
I'm getting errors:

cp: cannot create regular file 'some destination file': Operation not supported

cp: failed to preserve ownership for 'some destination file': Operation not supported

cp: connot create directory 'some destination directory': Operation not supported

getting MANY of each of these for multiple files.
LVL 27

Expert Comment

ID: 17855163
can you create any file on in destination directory manually (touch /dst/dir/filename) ?
LVL 14

Author Comment

ID: 17855188
Yes, created file with no problem
LVL 14

Expert Comment

ID: 17855714
try listing the contents through cat. preferably one from "cannot create regular file 'some destination file'"

cat some/destination/file

if the file scrolls through without giving errors then we know that the read is successful.  if there is a "error reading inode # xxxxx" we know that the disk has bad sectors or a corrupt inode table.
LVL 27

Expert Comment

ID: 17855765
I guess that destination filesystem has FAT32 type.
Then you may ignore 'failed to preserve ownership' message.
What about 'cannot create regular file'  and 'cannot create directory'

I see the only teo  possible reasons:
1) You have incorrect filesystem type when mounting (like MSDOS instead of VFAT).
2) You have  incorrect file/directory name for FAT32. Something very long or with special characters.
LVL 14

Author Comment

ID: 17857649
I'm reformatting destination drive, since it seems like that might be part of the issue.  Should I format as FAT32 or NTFS?
LVL 14

Author Comment

ID: 17857659
Nevermind.  NTFS it is.

Expert Comment

ID: 17858083
Hi there Geisrud,
One of the problems you might run into when copying from a linux file system, such as ext2, to a windows file system, is file name clashes.  On linux you could have

/path/something <- a file
/path/Something <- a different file
/path/somethinG <- a directory.

note that the files and directory would be able to all co-exist in the same location (/path/) on a case sensitive file system, but on a windows file system, you are going to effectively have multiple entries with the same name.  This could be causing your problems, and I will asume that it does, so here is the solution.

You can also have special files, such as block devices, character devices, soft-links and fifo pipes.  These you can probably just discard as they don't contain data in any case.

There are a few possible ways to solve this.  My first suggestion would be to just tar up the drive, example

tar cf /destination/archive.tar /path/to/source

This would however create a HUGE archive.tar file which will contain every single file on the hard drive.  This is not possible on FAT32 / DOS / etc if the to be backed-up disk contents is more than 2 GB.  Also writing to NTFS as the target is difficult under best of conditions, and not supported under most conditions, so my second suggestion:

You need to get rid of "clashing" file names.

first use the copy process as you have been doing above, but record the output into a new file, eg

cp -r /source /target 2>&1 |tee  /tmp/copy_process_log
or better:
find . | cpio -pdmv /destination 2>&1 | tee /tmp/copy_process_log

You will end up having probably most of your files copied.  You will also now have a log file of the failures.  This is where the beauty of Linux and Unix in general comes in.

Next extract a list of the failed files from the log: depending on how many, the easiest way might be to just edit the log file generated.  Remove everything from the file, but leave only the file names, one per line.  If many files has failed to copy, you could script this with "cleanup" using sed, but for that I would need to know more detail of the exact error messages.

Once you have a list of files, one per line, you can use this to automatically re-copy only those files.  So the next thing to do is to create a second directory so that these files cannot clash with your existing files.  Eg:

mkdir /destination/second_pass
and now copy the files in the list (This is why cpio is better)
cpio -pdmV /destination/second_pass 2>&1 | tee /tmp/copy_pass2_log < /tmp/list_of_failed_filenames

When you generate this list of files you may discover other things:
A small number of directories might each have multiple files with similar names except for case.
Some of the files you may not care about.  Delete these from the list.
Some of the files you might just copy by hand to specific locations.  Remove these from the list and handle them by hand.
Essentially: Simplify the list.

Note that on the second copy pass I again recorded the output of the process to a new log file.  This is in case a third pass is needed.  There are many other ways to solve this particular problem - you might for example tar just the problem files, or rename them by a script.  Here is a method which would use a rename process:

cat list_of_failed_filenames | while read LINE
  echo Moving \"${LINE}\" to \"${LINE}.$T\"
  mv "${LINE}" "${LINE}.$T"
  let T+=1

After running this script, which depends on the list you made above, re-run the copy process as you did initially, but be aware that every file will be renamed to get a number appended to the end of the file name, eg this number will be after the .jpg or .doc or whatever other extention you may have.

Also as a side-note:  You will probably discover that the cpio copy creates the files under /destination/source/...

This could be avoided by doing :
cd /source; find . | cpio ....

But don't do that, because you will probably re-use the lists of files and might move lists between cpio and cp, and not always be in the right directory, and and and....  Rather let the extra parent directory be created and remove it afterwards (It is very quick to remove this extra directory when all your copy work has been completed, even under Windows.)

P.S. good luck!
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

LVL 14

Author Comment

ID: 17858207
So much for drag and drop eh?  I'll try to follow those suggestions as best I can.  I won't be able to work on this until this evening, so please bear with me.  Any other suggestions, let em fly.  I'll let you know what happens.
LVL 27

Accepted Solution

Nopius earned 500 total points
ID: 17863108
> Nevermind.  NTFS it is.

That's not good news. The better it whould be to format the partition  as FAT32 (it's better supported from Linux then NTFS), then copy files, then (from Windows) convert it to NTFS if you like.

Expert Comment

ID: 17866250
You need FAT32, because Linux only has read status on NTFS by default, unless you use ntfs-3g or captiveNTFS (which I've never got working anyway)  CaptiveNTFS is a total waste of time IMO and ntfs-3g is a headache you don't need.

Use FAT32 on the destination drive and just use drag and drop, it shouldn't be that difficult as Knoppix puts a mount icon for each drive/partition on the desktop.

You just need to understand the Linux HD/partition naming system

The first partition on the dodgy HD will be hda1 (C in windows)
If the destination drive is the second HD, it will be hdb?  where ? is a number from 1 to whatever, depending how many partitions it has
LVL 14

Author Comment

ID: 17866447
I'll try to format as FAT32

Expert Comment

ID: 17871922
first and foremost, let me say this is dangerous if you don't do it right.  that being said....

you can make an exact image of the drive.  when i say exact, i mean exact.  partition table and all.

find out what your source and destination drives are.  assuming that the source drive is "/dev/hda" and the destination drive is "/dev/hdb"

using the command:
     dd if=/dev/hda of=/dev/hdb

When you do this, all data on /dev/hdb will be destroyed and overwritten with that on /dev/hda.  I wouldn't look at this as a permenant solution because you'll lose some of the space on /dev/hdb (the drive will think it's the same size as the original, hence if the original was 40 GB and the new one was 160GB that will be 120GB of lost space).

Hey, at least you'll have your data.  In the future though.....MAKE BACKUPS!
LVL 14

Author Comment

ID: 17873925
I was able to format as FAT32 after I created a partition <32GB.  I working on transfering files.  Updates soon!
LVL 14

Author Comment

ID: 17874243
Once I created a <32GB partition and formatted as FAT32, drag and drop copying went flawlessly.
LVL 27

Expert Comment

ID: 17885846
Thank you, Geisrud. Have a nice Windowz :-)
LVL 14

Author Comment

ID: 17906637
I'm just glad this job is done!  Thank you.

Featured Post

How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

Join & Write a Comment

This is the error message I got (CODE) Error caused by incompatible libmp3lame 3.98-2 with ffmpeg I've googled this error message and found out sometimes it attaches this note "can be treated with downgrade libmp3lame to version 3.97 or 3.98" …
Setting up Secure Ubuntu server on VMware 1.      Insert the Ubuntu Server distribution CD or attach the ISO of the CD which is in the “Datastore”. Note that it is important to install the x64 edition on servers, not the X86 editions. 2.      Power on th…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now