Solved

How to copy LARGE files using linux cp command ?

Posted on 2008-10-23
12
7,047 Views
Last Modified: 2013-12-16
I am very new to Linux ! I have been playing around with creating a bootable installation of PCLinuxOS 2007 on a usb stick. I have been using a LiveCD in a Windows host machine and have been trying to write the large livecd.sqfs file to a FAT16 formatted partition on a usb stick using 'cp'. This all starts and runs just fine for a minute or two, then always halts with the following error ...

cp: reading [file] input/output error

... or very similar. What causes this to happen ? and how can I resolve it ? I've looked at 'scp' but it really should not be necessary to encrypt a file just to copy it.

thanks RR
0
Comment
Question by:RetroRocker
  • 6
  • 3
  • 2
  • +1
12 Comments
 
LVL 12

Expert Comment

by:Heem14
ID: 22788143
how big? Fat 16 has a 2GB file size limit.
0
 

Author Comment

by:RetroRocker
ID: 22788433
Good point ! but not that big ... the file I mentioned originally was 'livecd.sqfs' and it is 692 MB.
0
 
LVL 12

Accepted Solution

by:
Heem14 earned 125 total points
ID: 22788506
I've never had a problem copying large files with cp - up to 8GB at times. Maybe try rsync? it has similar syntax to cp for simple copies.

I'd say your problem lies elsewhere though, not just in the choice of copy command.
0
 

Author Comment

by:RetroRocker
ID: 22789052
OK thanks, it's useful at my novice level to know that I'm not apparently coming up on some size limitation issue ... which of course I shouldn't be.
It seems you may well be correct about another reason for the problem, all I'm doing is copying from a CDROM to a USB stick ... so I'm definitely not sure what it could be. Perhaps some kind of 'buffering' issue maybe.
I'll research 'rsync' and try it out (everything I currently do with Linux requires much research :) and then I'll post back.

RR
0
 
LVL 3

Expert Comment

by:raminhos
ID: 22791777
it might be problem of your useb stick..i have a cheap 8Gb usb pen, and sometimes i get errors reading/write..
0
 

Author Comment

by:RetroRocker
ID: 22794907
I tried 'rsync' and it also fails ! With this I started to think along the lines of 'raminhos' although this is a good quality 4GB USB stick ... it can still be broken. However, interestingly enough, I can copy the whole 692MB file from CDROM to my USB FAT16 partition in Windows XP SP3 without issue ... weird ? and a check on the partition shows no errors with the file system (in Windows anyhow).
Here is the output of my 'rsync' error message ...

rsync: read errors mapping "the file" : Input/output error(5)
WARNING: syslinux.cfg failed verification (try again)
(The error message is now repeated)
Sent 1452012754 received 64 bytes 5866718.46 bytes/sec
Total size is 725917696  speed up is 0.50
rsync error: some files could not be transferred (code 23) at main.c(892) [sender=2.6.8]

The USB FAT16 partition remains empty after this but incidentally the 'total size' quoted is correct.

OK, confused or what ! ... now maybe I'm working along the lines of a 'typo' in the magazine article I'm working from (essentially note the reference to 'syslinux.cfg' in the error message above).
The command I'm issuing to copy follows ...

cp (or rsync) /mnt/cdrom/livecd.sqfs /mnt/usbstick/syslinux.cfg

the implication here, I think, is that 'cp' is being instructed to rename the file as well as copy. Surely though a 'typo' because this is PCLinuxOS itself not a configuration file !!! and just perhaps under linux there is some size issue with files having .cfg extension ?

With  ?'typos', error messages and an operating system I'm completely new to ... I'm getting confused :(

I'm beginning to think there may not be a definitive conclusion here but any further advice would be appreciated.

thanks RR
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 
LVL 10

Expert Comment

by:PeturIngiEgilsson
ID: 22795156
Repartition your usb stick.
Maby you should use vfat instead of fat?
0
 

Author Comment

by:RetroRocker
ID: 22796181
Thanks but I have repartitioned the USB stick 3 times now using linux 'fdisk'. The file system was created using the following command ...

mkfs.vfat -F 16 -n usb /dev/sdj1    (in this particular case)

... and appears to be without error.

RR
0
 
LVL 10

Expert Comment

by:PeturIngiEgilsson
ID: 22796347
rsync: read errors mapping "the file" : Input/output error(5)

in Windows.. I/O 5 : Access is denied.
0
 

Author Comment

by:RetroRocker
ID: 22796699
Yes but unless I'm not understanding your point, are we just mixing Windows and Linux here ?

All partitioning / file system creation / file copying has been executed strictly under Linux using a LiveCD distro of PCLinuxOS 2007 with its subsequent failure to copy.

My comments about being able to access this USB FAT16 file system and copy to it from within Windows were purely to demonstrate that I do have this access in Windows but not under Linux and implying that my FAT16 file system creations under Linux at least appears completely successful.

cheers RR
0
 
LVL 10

Assisted Solution

by:PeturIngiEgilsson
PeturIngiEgilsson earned 125 total points
ID: 22820095
Are you able to copy any other (smaller) files to the stick?
Is it possible that the CD your copying from is bad?
0
 

Author Comment

by:RetroRocker
ID: 22829972
I have had a bit more time to investigate this further.
Yes it is possible to copy smaller files to the USB stick and the CD media used as the source is not bad.
Now the rest of what I've discovered ! ...
In actual fact I can now copy any file including the 692MB one I mention in previous posts ! Reading back you will see I mentioned being very new to Linux and as such I have been severely misled by a very badly written magazine article I was working with. The copy command I was using was ...

cp (or rsync) /mnt/cdrom/livecd.sqfs /mnt/usbstick/syslinux.cfg

... as quoted in the article (after fixing yet another 'typo') and this is the problem ! You can't do it ! If you omit the 'syslinux.cfg' then it executes correctly. I thought that this was merely an attempt by the author to rename the file during copying but I did have reservations about a 692MB config file !
The bottom line here is that now armed with a little more understanding and applying what I think the article was trying to do, I have now achieved a bootable copy of PCLinuxOS 2007 on a USB stick.
  There were a total of 3 significant 'typos' in this article and it was very poorly constructed ... no doubt a rush job to meet an editorial deadline with no testing. The article in question was written by a guy called Graham Morrison in the UK magazine PCPlus issue 262. Of course the advantage is that I've learned a lot from the failures !
Thanks for all who tried to help, I'm not sure you stood a good chance of doing so under the circumstances but it was appreciated. I'll simply split the points as evenly as possible because as such there was really no true solution involved here.

thanks RR
0

Featured Post

How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

Join & Write a Comment

rdate is a Linux command and the network time protocol for immediate date and time setup from another machine. The clocks are synchronized by entering rdate with the -s switch (command without switch just checks the time but does not set anything). …
SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now