Want to win a PS4? Go Premium and enter to win our High-Tech Treats giveaway. Enter to Win

x
?
Solved

gzip cron job help

Posted on 2006-06-20
15
Medium Priority
?
658 Views
Last Modified: 2006-11-18
Hello,

I am a web developer and want to run a series of cron jobs but I don't know where to find the information on how to do each particular one...

1. I want to gzip a series of directories /root/userfiles/images /root/userfiles/images2 etc... but i want to encode some sense of the root structure so in the final gzip file it contains /root/userfiles/images/image1.jpg instead of image1.jpg do you know what I mean?

2. Secondly I want that gzipped file to be ftp'd or emailed or somehow transferred to me and deleted off the server...

We could probably break down question 1 into 2 steps of how do i gzip a series of directories using cron and how do i preserve directory structure while only including files in certain sub-directories

Thanks very much
:Ant
0
Comment
Question by:antum
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
15 Comments
 
LVL 6

Assisted Solution

by:JJSmith
JJSmith earned 200 total points
ID: 16946802


Do it all by hand - then script it - then put that script in a cron.

Notes:

gzip -N -r /root/userfiles/images
         # will compress all files in the directory saving the original filenames (use -N) on the gunzip to restore the original filename

Can't help on ftp or mail, obviously its possible but depends on setup and ftp and mail services at receving end.

Cheers
JJ
0
 
LVL 22

Accepted Solution

by:
pjedmond earned 800 total points
ID: 16946904
Probably easiest to create a bash script and tar the directories/files that you want:

--------8X----------
#!/bin/bash

tar czf ~/tarredfile.tar.gz /root/userfiles/images /root/userfiles/images2
mutt -a ~/tarredfile.tar.gz -s "Tarred files" me@mydomain.com <"."
rm -Rf ~/tarredfile.tar.gz
--------8X----------

Reason for using tar is that it maintains file structure and does not delete the original (Not a good idea in some situations). Also, you can stick them all in one tar file:)

OK - there are loads of other ways of sending the email or transferring it using scp (slightly complicated to do this securely, but it can be done - Look here:

http://www.cvrti.utah.edu/~dustman/no-more-pw-ssh/

Now for the cron bit:

crontab -e, and add your extra crontab (We'll call the script sendit.sh). Obviously test that it works without cron before adding a cron entry:

0 23 * * 1-5 /root/scripts/sendit.pl #Transfer of data

This entry runs the script every Mon-Fri at 2300.

http://www.itworld.com/Comp/2378/swol-0825-unix101/

for more examples.

HTH:)

0
 
LVL 22

Expert Comment

by:pjedmond
ID: 16946918
Ooops - should be sendit.sh in the crontab entry...but you spotted that 'deliberate mistake'...Didn't you?
0
NFR key for Veeam Agent for Linux

Veeam is happy to provide a free NFR license for one year.  It allows for the non‑production use and valid for five workstations and two servers. Veeam Agent for Linux is a simple backup tool for your Linux installations, both on‑premises and in the public cloud.

 
LVL 22

Expert Comment

by:pjedmond
ID: 16946936
Obviously replace the tar line with the gzip line in the script if you prefer. I just happen to use tar for this type of thing. Youcould use bzip2 as well - feel free to play;)

man tar
man bzip2

etc

HTH:)
0
 
LVL 35

Assisted Solution

by:Duncan Roe
Duncan Roe earned 600 total points
ID: 16946961
You want to use tar to keep the file structure. You can tar all the directories into a single file or have a tar file for each dir, as you prefer. tar has a -z (gzip) option and newer ones have a -j (bzip2) option (bzip2 usually compresses even better than gzip). Have a look at the man page for tar and post again if you have more questions.
As for transferring your files: mail has the advantage over ftp that you don't have to enter a password or code one in a script, you just need to be sure that no mail server on the way will refuse to pass on the attachment owing to size, content or whatever. What mail clients do you have on the web servers? - you will use one of them on the command line. Experiment, post again if you run out of ideas. I'd recommend installing mutt if it's not there already.
0
 
LVL 51

Assisted Solution

by:ahoffmann
ahoffmann earned 400 total points
ID: 16952285
I'd improve pjedmod's script as follows:

--------8X----------
#!/bin/sh
/bin/tar czf /tmp/tarredfile.tar.gz /root/userfiles/images /root/userfiles/images2 && \
/bin/mutt -a /tmp/tarredfile.tar.gz -s "Tarred files" me@mydomain.com <"." && \
/bin/rm -Rf /tmp/tarredfile.tar.gz
--------8X----------
0
 
LVL 22

Expert Comment

by:pjedmond
ID: 16952937
Yep - I'd agree with that:)

Always good to have someone check and improve things:) I'm going to have to work hard to improve the quality of my posts;)
0
 
LVL 35

Expert Comment

by:Duncan Roe
ID: 16955676
Is '<"."' an attempt to make a "here" document? Look what it does:
07:47:47$ cat <"."
cat: -: Is a directory
bash connected the current directory to standard input - what you asked it to do but not what you want

Take 3:-

--------8X----------
#!/bin/sh
/bin/tar czf /tmp/tarredfile.tar.gz /root/userfiles/images /root/userfiles/images2 && \
/bin/mutt -a /tmp/tarredfile.tar.gz -s "Tarred files" me@mydomain.com <<EOF
.
EOF
[ $? -eq 0 ] && /bin/rm -Rf /tmp/tarredfile.tar.gz
--------8X----------
0
 
LVL 22

Expert Comment

by:pjedmond
ID: 16955727
I see your point, but if you do it that way, surely you can:

--------8X----------
#!/bin/sh
/bin/tar czf /tmp/tarredfile.tar.gz /root/userfiles/images /root/userfiles/images2 && \
/bin/mutt -a /tmp/tarredfile.tar.gz -s "Tarred files" me@mydomain.com <"."
[ $? -eq 0 ] && /bin/rm -Rf /tmp/tarredfile.tar.gz
--------8X----------

0
 
LVL 22

Expert Comment

by:pjedmond
ID: 16955766
Thought about this a bit more.

The meaning of "." depends on the context.

As you quite rightly said:

cat <"."

does try and cat the current directory, because cat expects a filename, and hence "." is assumed to be a filename.

. filename

executes a filename.

Under these circumstances (the use of mutt), I'd always considered the <"." to be perfectly valid(as it works), as I'd assumed that the . concerned was the 'finalising' of the email, which it effectively does....but if you were to change this to <"hello there\n". From a 'purists' perspective, I like your apporach more. Always good to have someone 'red penning' my work ;)

(   (()
(`-' _\
 ''  ''
0
 
LVL 35

Expert Comment

by:Duncan Roe
ID: 16958550
Hi pjedmond - I think you must have been lucky with mutt - it would have been given a directory as its input and chose not to complain.
cat does not "expect a filename" - the man page documents that in the absence of an argument, it will read from standard input. The construct '<"."' tells the shell (bash, say) to connect "." (the current directory) to the standard input of the program you told it to run (cat or mutt). It seems that mutt chooses to treat the error as EOF (it could just be treating a -1 return from fgets as EOF, I haven't checked).
20:20:27$ cat <"hello there\n"
-bash: hello there\n: No such file or directory
Notice that error comes from *bash* - cat or mutt never gets started. "." is a valid existing filesystem entity, albeit a directory, so bash doesn't complain.
0
 
LVL 22

Expert Comment

by:pjedmond
ID: 16958675
cat does not "expect a filename" - Ok - I'll expand - If given a name it assumes it to be a filename.

I was wondering if this works due to the first 2 'files' in the current directory are:

.
..

Based on the fact that a '.' on its own terminates the email message when telnetting to an SMTP server, hence the email is completed by the first '.' on it's own. However, as you've suggested, it appears that the error produced *really* is interpreted as the EOT! EEeeeek scary - guaranteed to break unexpectedly during an upgrade! I've tweaked the code on a few of my customers to correct (even though I've used this hack for years! I also accept that it is a horrible hack, and I also admit having used it for years on the basis that it it just 'worked'.

How about going for the more conventional then:

/bin/mutt -a /tmp/tarredfile.tar.gz -s "Tarred files" me@mydomain.com < /home/me/messagefile

where messagefile is:

-------8X-------------------
Here are your tarred files
-------8X-------------------

Wow - best question I've participated in this month!

(   (()
(`-' _\
 ''  ''


0
 
LVL 22

Expert Comment

by:pjedmond
ID: 16958741
mutt -a ~/list.txt -s "Tarred files" me@cb.ws < /dev/null

also works:) ......

(   (()
(`-' _\
 ''  ''

0
 
LVL 35

Expert Comment

by:Duncan Roe
ID: 16958990
</dev/null is fine. Gives an immediate EOF (and *no* error ;)
0
 

Author Comment

by:antum
ID: 17015295
Thankyou all very much for your help!
0

Featured Post

Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Over the last ten+ years I have seen Linux configuration tools come and go. In the early days there was the tried-and-true, all-powerful linuxconf that many thought would remain the one and only Linux configuration tool until the end of times. Well,…
The purpose of this article is to demonstrate how we can use conditional statements using Python.
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
Suggested Courses

609 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question