need to create script to back up directories

Linux. I need to set up a script that I can run via cron in the middle of every night to copy / back up specific folders into another directory. I want all folders, files and permissions and attributes copied. The goal is to have a 24 hour old copy that I can restore in case a user screws up their files. Example:

/usr/home/thisuser/web_master would be copied / backed up to /usr/home/mainuser
/usr/home/thisotheruser/web_master would be copied / backed up to /usr/home/mainuser
/usr/home/andanotheruser/web_master would be copied / backed up to /usr/home/mainuser

Each night, the copy / back up would overwrite the one from 24 hours ago.

How do I set this script up, and what type of file extension should the script be?
LVL 1
BobbyAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

nociSoftware EngineerCommented:
most easy is using tar  to create a backup set.

lets put the list of usernames in /usr/local/etc/webmasters_to_backup
thisuser
thisotheruser
andanotheruser

Open in new window

lets call the script:   /usr/local/bin/backup_webmaster
#!/bin/bash

for u in $( cat /usr/local/etc/webmasters_to_backup ) 
do
    tar czf /usr/home/mainuser/${u}.tgz  -C /usr/home/${u}/web_master .
done

Open in new window

chmod 755 /usr/local/bin/backup_webmaster

And add a cron entry for root:
0 0 * * * /usr/local/etc/backup_webmaster

Open in new window


To restore execute:

tar xvpvf /usr/home/mainuser/<someuser>.tgz -C /usr/home/<someuser>/web_master the/file/to-restore
BobbyAuthor Commented:
Thanks. Why tar?
nociSoftware EngineerCommented:
Tar will preserve owners & protection mask in the most easy way.
If you also use ACL's you actually may need  star.

Just copying files won't get you the deletions that would be needed as well.
rsync could be another tool to use.  (tar is mostly installed anyway, rsync maybe not).
Also you didn't specify directories where data could be placed, to using an archive file cirumvents that.
(mingling of data of various users seems counter productieve if file overwrite each other.
Maximize Customer Retention with Superior Service

The IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy for valuable how-to assets including sample agreements, checklists, flowcharts, and more to help build customer satisfaction and retention.

BobbyAuthor Commented:
While I was waiting I did this in a .sh file and ran it, it worked (except it skipped copying symlinks,.... can that be changed?)... would this be good enough? And I assume it will just overwrite each folder each day? That's what I want.

rsync -r /usr/www/users/userone/eimg/ /usr/home/mainuser/backups/eimg/
rsync -r /usr/www/users/mainuser/beta.me.dev.mysite.com/ /usr/home/mainuser/backups/beta.me.dev.mysite.com/

Open in new window

nociSoftware EngineerCommented:
No you need rsync -a for that (archival mode).
or: rsync -rlptgoD
which is the same...
See man rsync for details on the flags.

https://linux.die.net/man/1/rsync
BobbyAuthor Commented:
Ok, can you please tell me how to write these two rsync commands so that they will copy everything in each folder, including symlinks, and then overwrite the previous copy at the destination. Thanks much.
rsync -r /usr/www/users/userone/eimg/ /usr/home/mainuser/backups/eimg/
rsync -r /usr/www/users/mainuser/beta.me.dev.mysite.com/ /usr/home/mainuser/backups/beta.me.dev.mysite.com/

Open in new window

BobbyAuthor Commented:
or is it just a simple matter of changing the -r to -a?
nociSoftware EngineerCommented:
rsync -a /usr/www/users/userone/eimg/ /usr/home/mainuser/backups/eimg/
rsync -a /usr/www/users/mainuser/beta.me.dev.mysite.com/ /usr/home/mainuser/backups/beta.me.dev.mysite.com/

Open in new window


or  to get somewhat more info
rsync -av /usr/www/users/userone/eimg/ /usr/home/mainuser/backups/eimg/
rsync -av /usr/www/users/mainuser/beta.me.dev.mysite.com/ /usr/home/mainuser/backups/beta.me.dev.mysite.com/

Open in new window

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
BobbyAuthor Commented:
The cron job executed the script last night, I had just the -a, not -av, and everything looks intact. What's the difference between -a and -av? Verbose? Where does that info go?
BobbyAuthor Commented:
If I add the v after the -a will I be sent an email with the info? The server does send me an email if the backup fails, I tested that by renaming one of the folders the script backs up.
David FavorLinux/LXD/WordPress/Hosting SavantCommented:
Based on what you've asked for, noci's suggestion is best.

Either rsync -a or -av (-v just means verbose [read the man page] which produces a list of file changes) for super fast cloning of files between directories.

If you do something like this...

rsync -a source-dir target-dir 2>&1 | your-mail-sending-voodoo-here

Open in new window


You'll get a list of problems, add -v to get an entire list of files changed.

Tip: If you're doing this to attempt reaching a consistent file state in your target-dir(s) to then do backups, use tar + zstd for least resource usage + maximum file compression.

Tip: Also, you can go one step further + do what I do for my backups. I do an rsync where my target-dir is on another machine. Then after all all machines (many) finish their rsync to the backup server, I create full backups (1x/week) or incremental backups (6x/week) on the backup machine. This way the tar + zstd has no effect on runtime performance of production sites.
BobbyAuthor Commented:
Thank you both very much.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.