Solved

Create Tar file in unix and split it to smaller segments

Posted on 2014-09-04
6
1,293 Views
Last Modified: 2014-09-04
I need to split a large tar file for backup purposes on a unix system. I need to tar the /u/appl directory and then split into chunks less the 2gb in size. The following command is what I'm using now, but I get an error "Split: cannot open input".

tar cvf - /u/appl | split -b 1500m /u/backup/appl.backup.tar.gz

I'm sure there's a syntax error, as I can run the tar command by itself with no issue. I've tried to resolve but haven't been able to.
0
Comment
Question by:sdholden28
  • 2
  • 2
  • 2
6 Comments
 
LVL 23

Assisted Solution

by:nemws1
nemws1 earned 250 total points
ID: 40303517
You need to provide an input file parameter to split.  In this case, you want STDIN, so you just need a hyphen:

 tar cvf - /u/appl | split -b 1500m - /u/backup/appl.backup.tar.gz

Open in new window

0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 250 total points
ID: 40303534
You must tell split which input to read. In your case it's stdin, expressed as "-"

tar czvf - /u/appl | split -d -b 1500m - /u/backup/appl.backup.tar.gz

"-d" tells split to use numeric suffixes. Use the "-z" flag of tar to compress.

Please be aware that you must concatenate the single input files before feeding them to tar for extracting!

cat /u/backup/appl.backup.tar.gz* | tar -xzvf -

tar is also able to create multi-volume archives.

If you know roundabout how big your input data will be you can do

tar -c -M -L 1536000 --file =/u/backup/appl.backup.tar1 --file =/u/backup/appl.backup.tar2 --file= /u/backup/appl.backup.tar3 --file =/u/backup/appl.backup.tar4

Note: You cannot use compression when the "-L" flag is in effect.

You can repeat the "--file" parameter as often as needed. "-L" specifies the chunk size in K.

You can also specify just one filename. tar will prompt you for the name of the next file once the amount of data specified by "-L" (in K) has been archived.. Answer the question "Prepare volume #n for 'archive' and hit return:" with "n new_file_name".

Finally, you can use a "new volume" script to get rid of the prompts. Here is how it works: http://www.gnu.org/software/tar/manual/tar.html#Using-Multiple-Tapes
0
 

Author Comment

by:sdholden28
ID: 40303985
does stdin use temp space on /dev/root? That looks to be the case as I'm running into a space issue in my test environment. Can I direct the temp storage to /u somehow?
0
Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

 
LVL 68

Expert Comment

by:woolmilkporc
ID: 40303987
stdin/stdout  do not use intermediate (temporary) disk space.
0
 
LVL 23

Assisted Solution

by:nemws1
nemws1 earned 250 total points
ID: 40304102
split *might* be using temp space (I kinda doubt it, though).

Maybe just try this (make sure both tar and split use the temporary TMP location).

mkdir /u/tmp
chmod 700 /u/tmp
TMP=/u/tmp tar cvf - /u/appl | TMP=/u/tmp split -b 1500m - /u/backup/appl.backup.tar.gz

Open in new window

0
 

Author Closing Comment

by:sdholden28
ID: 40304890
Thanks gentlemen, got it sorted. Could've solved my problem with the info from nemws1 alone, but I greatly appreciate the extra info from woolmilkporc and I used it to implement a better solution, so I split the points. Thanks again.
0

Featured Post

Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Entering a date in Microsoft Access can be tricky. A typo can cause month and day to be shuffled, entering the day only causes an error, as does entering, say, day 31 in June. This article shows how an inputmask supported by code can help the user a…
In this post we will learn how to connect and configure Android Device (Smartphone etc.) with Android Studio. After that we will run a simple Hello World Program.
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question