Linux: Trouble with very large file

I created a tar archive like this:

tar czf foobar.tar.gz /my/dir/

The size of the file is 6 GB.

I need to send the file to a different computer using FTP but I get an error:
Error: File `foobar.tar.gz' does not exist.

So I tried to split it but I got an error:
split: cannot open `foobar.tar.gz' for reading: No such file or directory

I can see the file using "ls -lh"
-rw-r--r--   1 myuser mygroup 6.4G Dec 28 09:11 foobar.tar.gz

What should I do?  Is there a way to create multiple smaller tar files?  Or is there a way to send the file?

I use CentOS 4 (32 bit).
LVL 16
hankknightAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

arnoldCommented:
Is ftp the only way to get to the other system?
Is there an option to establish an ssh session and pump the data through
ssh user@remotesystem "tar cf - /my/dir/" | gzip -c >/where/there/is/enough/space/for/foobar.tar.gz
The - in the tar command indicates that the result should be sent to Standard Output.
What filesystem  is the file stored on? ext2/ext3?
hankknightAuthor Commented:
Unfortunately FTP is the only way I can connect to the remote system.

How can I find out what filesystem is used?
arnoldCommented:
On your system the command is:
mount

It will display the mounted devices with the filesystem type follows the type <filesystem>.

I do not believe that the 6GB is an issue.
Which ftp client are you using?

try
tar cf - | gzip -c | split -b <split_file_size>


where split_file_size if 500m you will end up with 12files.
xaa-xal

See if that gets the process moving along.  
hankknightAuthor Commented:
The data is stored on ext2.

Thanks, but I don't understand how to create a split tar.

The directory that I want to archive is: /my/dir/

This code does NOT work:
tar cf foobar.tar - | gzip -c | split -b 700m /my/dir/

Open in new window

arnoldCommented:
Oops,

The line should be:
tar cf -  /my/dir/| gzip -c | split -b 700m

tar cf - /my/dir #will tar the contents and will pump the data out to standard ouput to
gzip -c # gets the data stream on standard output, compresses the data and outputs the results on standard output to
split -b 700m #which gets the compressed data stream on standard input and outputs the xa* files based on the specified size (-b 700m)
one the proces is complete you will have 8 chunks xaa xab xac xad xae xaf xag xah.
To reconstruct you would cat  xaa xab xac xad xae xaf xag xah > foobar.tar.gz.
If you need to skip the single file and reverse the process that created these files
cat xaa xab xac xad xae xaf xag xah | gzip -cd | tar -xf - #note the restore location will be based on the current location where this command is executed plus  my/dir The first forward slash is stripped when the tar process runs.
-d for gzip tells to uncompress
-xf - for tar deals with getting data on standard input and extracting the data.

 

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.