how to copy large files across net?

Posted on 2009-04-24
Last Modified: 2013-12-01
I am trying to do a relatively simple task. I want to copy a  15,618,606,818 (15GB) file from one host to another. I've tried ftp, rsync and gftp. ftp and rsync die with "file too large" errors. gftp is probably dying for the same reason, but doesn't give that as a message. The target device has 250GB free.

nfs is not a desired solution. Nor is breaking the file up into smaller files.

How do I do this?
Question by:jmarkfoley
    LVL 5

    Expert Comment

    scp wil transfer the file for you.

    Command to use is:
    scp myfiletocopy username@remoteip:remotefile

    I've done 100GB files this way and is secure.
    LVL 7

    Expert Comment

    rsync -vrPtz -e ssh host:/remote_path/* /local_path/


    -e ssh rsync will use ssh client instead of rsh
    -z compress file transfer
    -t preserve time (other attributes as owner or permissions are also possible)
    -P resume incomplete file transfer
    -r recursive into subdirectories
    -v verbose

    if it stops running run it again Cheers!
    LVL 7

    Accepted Solution

    Read this wrong the first time, I would try upgrading rsync and you kernel to the latest version.   Any chance your rsyncing to fat, the limit of fat32 is 4gb.  What os and filesystem are you syncing to and from?
    LVL 1

    Author Comment

    I used scp. It works and I like it. I like the progress info. as it turned out, the problem was with none of the client programs. They probably all would work. The problem was my target filesystem. I mounted an external usb drive which was preformatted to Windows FAT. I thought that would be nice because I could access the files from my windows laptop but, as everyone *should* know, there is a 2GB file size limit. So, I reformated it to ext2 and no problem.

    Featured Post

    What Should I Do With This Threat Intelligence?

    Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

    Join & Write a Comment

    How to fix error ""Failed to validate the vCentre certificate. Either install or verify the certificate by using the vSphere Data Protection Configuration utility" when you are trying to connect to VDP instance from Vcenter.
    Workplace bullying has increased with the use of email and social media. Retain evidence of this with email archiving to protect your employees.
    This tutorial will walk an individual through the steps necessary to enable the VMware\Hyper-V licensed feature of Backup Exec 2012. In addition, how to add a VMware server and configure a backup job. The first step is to acquire the necessary licen…
    This tutorial will walk an individual through configuring a drive on a Windows Server 2008 to perform shadow copies in order to quickly recover deleted files and folders. Click on Start and then select Computer to view the available drives on the se…

    734 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    18 Experts available now in Live!

    Get 1:1 Help Now