c11v11
asked on
Does perl copy better than linux copy command?
Does anyone know if the Perl copy better than Linux copy command? I mean if the copy time would be better than Linux copy.
there is no copy command on linux....
Especially on Linux (not on old UNIX) |cp| command can take enormous number of arguments, so there is not much efficiency gained by using perl, or compiling C program to delete ton of files.
Especially on Linux (not on old UNIX) |cp| command can take enormous number of arguments, so there is not much efficiency gained by using perl, or compiling C program to delete ton of files.
ASKER
we have hundreds thousands files of small files ( average 16K) to copy from one NFS directory to another NFS directory on one same server. Actually it is for archive purpose. Does the rsync work for this situation ? Does rsync support multiple process to handle the file copy?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
To copy lots of files from nfs use actimeo=3600 mount option on Linux
In my experience, tar without gzip is faster than with it (but you must have room for the big tar file at both ends) - the gzip takes longer than the time difference to transfer compressed / uncompressed.
So I would say tar without gzip on the systems where the files actually reside and will reside is your best shot. Don't even think of running tar anywhere else.
If you don't go the tar route, check man 5 nfs to understand what gheist's suggested actimeo=3600 mount option does: pretty neat for the copy but you don't want to leave it there.
So I would say tar without gzip on the systems where the files actually reside and will reside is your best shot. Don't even think of running tar anywhere else.
If you don't go the tar route, check man 5 nfs to understand what gheist's suggested actimeo=3600 mount option does: pretty neat for the copy but you don't want to leave it there.
tar p keeps permissions (or cpio or pax) down to numeric uid
rsync would match usernames and try to get better permissions
no need for temporary files;)
(cd src ; tar cpf - .) | ssh dst '(cd dst ;tar xpf -)
rsync would match usernames and try to get better permissions
no need for temporary files;)
(cd src ; tar cpf - .) | ssh dst '(cd dst ;tar xpf -)
ASKER
I don't have a dst server. source directory and dest directory are both mount at NFS side.
remove 'ssh' from command line....
best if source and destination are different mounts and you can set source with actimeo=3600 (or more) and each can use differnet thread for processing in client side.
Much more optimal would be having source mounted from destination and copied over that shortcut. (ask admin of the NFS if they can help...)
best if source and destination are different mounts and you can set source with actimeo=3600 (or more) and each can use differnet thread for processing in client side.
Much more optimal would be having source mounted from destination and copied over that shortcut. (ask admin of the NFS if they can help...)
Have you tried rsync? Very fast...