how can I automate wget tasks on my linux server

I have a list of commands I need to run to sync my web servers.
I have a staging server that we stage our content changes on. when they are approved I then tar up a few directories and do a mysqldump in the web directry of staging.
I then delete the previously downloaded tar files and sql then I run wget from the live server grabbing the two tar files and the mysql database. I then un tar the files then restore the database.

what I need help with is taking the commands and putting them into a script that I could either add to a cron or fire off from the admin console of each server.
STAGING SERVER:
cd /var/www/html/site/htdocs
rm files.tar
rm theme.tar
rm site.sql.txt
tar cvzf files.tar sites/default/files/
tar cvzf theme.tar sites/all/themes/site/
mysqldump -u root --password=PASSWRD! site> site.sql.txt;
 
LIVE SERVER:
cd /var/www/html/site/htdocs
rm files.tar
rm theme.tar
rm site.sql.txt
wget http://207.195.x.x/site.sql.txt
wget http://207.195.x.x/theme.tar
wget http://207.195.x.x/files.tar
tar -xvf files.tar
tar -xvf theme.tar
mysql -u root --password=PASSWORD! site< site.sql.txt

Open in new window

LVL 3
jbrashear72Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

osintsevCommented:
I think it will be something similar to this:

STAGING SERVER:

Add this line to your crontab on staging server:
0 22 * * * /path_to_script/script.sh >/dev/null 2>&1

Save this as script.sh and set chmod 700.

#!/bin/sh
cd /var/www/html/site/htdocs
rm files.tar theme.tar site.sql.txt
tar cvzf files.tar sites/default/files/
tar cvzf theme.tar sites/all/themes/site/
mysqldump -u root --password=PASSWRD! site> site.sql.txt;

LIVE SERVER:

Add this line to your crontab on live server:
0 23 * * * /path_to_script/script.sh >/dev/null 2>&1

Save this as script.sh and and do not forget to change chmod 700.

#!/bin/sh
cd /var/www/html/site/htdocs
wget http://207.195.x.x/site.sql.txt
wget http://207.195.x.x/theme.tar
wget http://207.195.x.x/files.tar
tar -xvf files.tar
tar -xvf theme.tar
mysql -u root --password=PASSWORD! site< site.sql.txt
rm files.tar rm theme.tar rm site.sql.txt

But I really do not recommend to update mysql database on the live server in fully automatic mode. Saving database dump in the webserver directory without additional authorization is a very bad idea.
0
jbrashear72Author Commented:
I agree with you. I just don't know how  I can do this for the end user so that they dont need scripting/command line knowladge.

0
osintsevCommented:
Ok, a better solution... Staging server upload files to the live server by ssh.

STAGING SERVER:

Setup ssh keys for passwordless authorization on live server (one-time procedure). Look at this guide: http://www.linuxjournal.com/article/8257

Add this line to your crontab on staging server:
0 22 * * * /path_to_script/script.sh >/dev/null 2>&1

Save this as script.sh and set chmod 700.

#!/bin/sh
cd ~
tar cvzf files.tar /var/www/html/site/htdocs/sites/default/files/
tar cvzf theme.tar /var/www/html/site/htdocs/sites/all/themes/site/
mysqldump -u root --password=PASSWRD! site> site.sql.txt
scp files.tar theme.tar site.sql.txt user@live_server.example.com:~/
rm  files.tar theme.tar site.sql.txt

LIVE SERVER:

I still recommend that a user runs the script by hands that would catch possible errors. In addition, the script saving a working database to current.sql.txt file before uploading site.sql.txt in the working database.

Update script  (maximum closer to reality):

#!/bin/sh
cd ~
tar cvzf files.tar /var/www/html/site/htdocs/sites/default/files/
tar cvzf theme.tar /var/www/html/site/htdocs/sites/all/themes/site/
mysqldump -u root --password=PASSWRD! site> current.sql.txt
mysql -u root --password=PASSWORD! site< site.sql.txt
rm  files.tar theme.tar site.sql.txt
0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

jbrashear72Author Commented:
That is an exaclent solution! Thank you for your help.
I am working on the ssh setup right now.
0
jbrashear72Author Commented:
OK I have the SSH setup and the staging server script works when I remove:
cd ~
well that is only for the mysql dump. the tar's work.

when it SCP's the files over it puts them in the root folder. I need them to be in the htdocs root tu untar correct?
0
osintsevCommented:
You can untar archive content in the custom directory with specifying '-C <dir>' flag to tar utility. But better a little bit to change how the archive is created (use relative paths instead of absolute)

Archiving:

# cd /var/www/html/site/htdocs
# tar cvf files.tar ./sites/default/files/
# tar cvf theme.tar ./sites/all/themes/site/
... sending via scp and remove ...

Unpacking:

from any directory run this

# tar xvf /path_to/files.tar -C /var/www/html/site/htdocs/
# tar xvf /path_to/theme.tar -C /var/www/html/site/htdocs/

I think it should work.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
jbrashear72Author Commented:
I have it working..
Can I ask another question?
Is there a way to do a checksum on the file before I unpack it on the live server?
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.