jbrashear72
asked on
how can I automate wget tasks on my linux server
I have a list of commands I need to run to sync my web servers.
I have a staging server that we stage our content changes on. when they are approved I then tar up a few directories and do a mysqldump in the web directry of staging.
I then delete the previously downloaded tar files and sql then I run wget from the live server grabbing the two tar files and the mysql database. I then un tar the files then restore the database.
what I need help with is taking the commands and putting them into a script that I could either add to a cron or fire off from the admin console of each server.
I have a staging server that we stage our content changes on. when they are approved I then tar up a few directories and do a mysqldump in the web directry of staging.
I then delete the previously downloaded tar files and sql then I run wget from the live server grabbing the two tar files and the mysql database. I then un tar the files then restore the database.
what I need help with is taking the commands and putting them into a script that I could either add to a cron or fire off from the admin console of each server.
STAGING SERVER:
cd /var/www/html/site/htdocs
rm files.tar
rm theme.tar
rm site.sql.txt
tar cvzf files.tar sites/default/files/
tar cvzf theme.tar sites/all/themes/site/
mysqldump -u root --password=PASSWRD! site> site.sql.txt;
LIVE SERVER:
cd /var/www/html/site/htdocs
rm files.tar
rm theme.tar
rm site.sql.txt
wget http://207.195.x.x/site.sql.txt
wget http://207.195.x.x/theme.tar
wget http://207.195.x.x/files.tar
tar -xvf files.tar
tar -xvf theme.tar
mysql -u root --password=PASSWORD! site< site.sql.txt
ASKER
I agree with you. I just don't know how I can do this for the end user so that they dont need scripting/command line knowladge.
Ok, a better solution... Staging server upload files to the live server by ssh.
STAGING SERVER:
Setup ssh keys for passwordless authorization on live server (one-time procedure). Look at this guide: http://www.linuxjournal.com/article/8257
Add this line to your crontab on staging server:
0 22 * * * /path_to_script/script.sh >/dev/null 2>&1
Save this as script.sh and set chmod 700.
#!/bin/sh
cd ~
tar cvzf files.tar /var/www/html/site/htdocs/
tar cvzf theme.tar /var/www/html/site/htdocs/
mysqldump -u root --password=PASSWRD! site> site.sql.txt
scp files.tar theme.tar site.sql.txt user@live_server.example.c
rm files.tar theme.tar site.sql.txt
LIVE SERVER:
I still recommend that a user runs the script by hands that would catch possible errors. In addition, the script saving a working database to current.sql.txt file before uploading site.sql.txt in the working database.
Update script (maximum closer to reality):
#!/bin/sh
cd ~
tar cvzf files.tar /var/www/html/site/htdocs/
tar cvzf theme.tar /var/www/html/site/htdocs/
mysqldump -u root --password=PASSWRD! site> current.sql.txt
mysql -u root --password=PASSWORD! site< site.sql.txt
rm files.tar theme.tar site.sql.txt
STAGING SERVER:
Setup ssh keys for passwordless authorization on live server (one-time procedure). Look at this guide: http://www.linuxjournal.com/article/8257
Add this line to your crontab on staging server:
0 22 * * * /path_to_script/script.sh >/dev/null 2>&1
Save this as script.sh and set chmod 700.
#!/bin/sh
cd ~
tar cvzf files.tar /var/www/html/site/htdocs/
tar cvzf theme.tar /var/www/html/site/htdocs/
mysqldump -u root --password=PASSWRD! site> site.sql.txt
scp files.tar theme.tar site.sql.txt user@live_server.example.c
rm files.tar theme.tar site.sql.txt
LIVE SERVER:
I still recommend that a user runs the script by hands that would catch possible errors. In addition, the script saving a working database to current.sql.txt file before uploading site.sql.txt in the working database.
Update script (maximum closer to reality):
#!/bin/sh
cd ~
tar cvzf files.tar /var/www/html/site/htdocs/
tar cvzf theme.tar /var/www/html/site/htdocs/
mysqldump -u root --password=PASSWRD! site> current.sql.txt
mysql -u root --password=PASSWORD! site< site.sql.txt
rm files.tar theme.tar site.sql.txt
ASKER
That is an exaclent solution! Thank you for your help.
I am working on the ssh setup right now.
I am working on the ssh setup right now.
ASKER
OK I have the SSH setup and the staging server script works when I remove:
cd ~
well that is only for the mysql dump. the tar's work.
when it SCP's the files over it puts them in the root folder. I need them to be in the htdocs root tu untar correct?
cd ~
well that is only for the mysql dump. the tar's work.
when it SCP's the files over it puts them in the root folder. I need them to be in the htdocs root tu untar correct?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I have it working..
Can I ask another question?
Is there a way to do a checksum on the file before I unpack it on the live server?
Can I ask another question?
Is there a way to do a checksum on the file before I unpack it on the live server?
STAGING SERVER:
Add this line to your crontab on staging server:
0 22 * * * /path_to_script/script.sh >/dev/null 2>&1
Save this as script.sh and set chmod 700.
#!/bin/sh
cd /var/www/html/site/htdocs
rm files.tar theme.tar site.sql.txt
tar cvzf files.tar sites/default/files/
tar cvzf theme.tar sites/all/themes/site/
mysqldump -u root --password=PASSWRD! site> site.sql.txt;
LIVE SERVER:
Add this line to your crontab on live server:
0 23 * * * /path_to_script/script.sh >/dev/null 2>&1
Save this as script.sh and and do not forget to change chmod 700.
#!/bin/sh
cd /var/www/html/site/htdocs
wget http://207.195.x.x/site.sql.txt
wget http://207.195.x.x/theme.tar
wget http://207.195.x.x/files.tar
tar -xvf files.tar
tar -xvf theme.tar
mysql -u root --password=PASSWORD! site< site.sql.txt
rm files.tar rm theme.tar rm site.sql.txt
But I really do not recommend to update mysql database on the live server in fully automatic mode. Saving database dump in the webserver directory without additional authorization is a very bad idea.