Need an archival script .

nicky s
nicky s used Ask the Experts™
on
need an archival script that runs on a nighty basis

we maintain our logs for a given month in a directory

/a/b/c/2018-01/xyz-2018-01-01-1.log
/a/b/c/2018-01/xyz-2018-01-01-2.log
/a/b/c/2018-01/xyz-2018-01-01-3.log
/a/b/c/2018-01/xyz-2018-01-02-1.log
/a/b/c/2018-01/xyz-2018-01-01-2.log
.
.
.
/a/b/c/2018-02/xyz-2018-02-01-1.log
/a/b/c/2018-02/xyz-2018-02-01-2.log
.
.
.
/a/b/c/2018-02/xyz-2018-02-31-1.log
/a/b/c/2018-02/xyz-2018-02-31-2.log

on a nightly basis we have to copy logs to corresponding months folder with date and tar and gzip

if the directory does not exist for the month we have to create it when copy and tar the files,

ex : /a/b/c/archieve/`date +`'%Y-%m'`/`

we should be maintaining the archived files for the last 3 days and hould be removed older than 3 days
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Prabhin MPDevOps Engineer
Distinguished Expert 2018

Commented:
#!/bin/bash
set -e
nowdate=$(date +"%Y-%m-%d-%H")
dayago=`(date --date='02 days ago' '+%Y-%m-%d-%H')`
mondate=$(date +"%Y-%m")
backup=/backup/data
if [ -d "$backup/$mondate" ]; then
    echo "found month folder"
    mv /var/log/apache2/access.log $backup/$mondate/access.$nowdate.log
    gzip  $backup/$mondate/access.$nowdate.log
else
    echo "no month folder found creating $mondate"
    mkdir $backup/$mondate/
    cp /var/log/apache2/access.log $backup/$mondate/access.$nowdate.log
    gzip  $backup/$mondate/access.$nowdate.log
fi

Author

Commented:
there are so many logs generated for the day with -1,-2,-3 suffix
after moving to archive, we need to remove the logs older than 3 days from archieve folder
Prabhin MPDevOps Engineer
Distinguished Expert 2018

Commented:
#!/bin/bash
set -e
nowdate=$(date +"%Y-%m-%d")
dayago=`(date --date='03 days ago' '+%Y-%m-%d')`
mondate=$(date +"%Y-%m")
backup=/backup/data
if [ -d "$backup/$mondate" ]; then
    echo "found month folder"
    mv /var/log/apache2/access-1.log $backup/$mondate/access-1.$nowdate.log
    mv /var/log/apache2/access-2.log $backup/$mondate/access-2.$nowdate.log
    gzip  $backup/$mondate/access-1.$nowdate.log
    gzip  $backup/$mondate/access-2.$nowdate.log
    rm $backup/$mondate/access-1.$dayago.log
    rm $backup/$mondate/access-2.$dayago.log
else
    echo "no month folder found creating $mondate"
    mkdir $backup/$mondate/
    mv /var/log/apache2/access-1.log $backup/$mondate/access-1.$nowdate.log
    mv /var/log/apache2/access-2.log $backup/$mondate/access-2.$nowdate.log
    gzip  $backup/$mondate/access-1.$nowdate.log
    rm $backup/$mondate/access-1.$dayago.log
    rm $backup/$mondate/access-2.$dayago.log

fi

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial