Apache Log Rotation

Hi,

I am facing lots of problem due to heavy logs generated by my Webserver.

I am using the following shell script to rotate the logs...but it's not working properly.
==========================
#!/bin/bash
HTTPD_LOGS_DIR=/scratch/content/logs/apache       # path to httpd log dir
YYMMDD=`/bin/date +%Y%m%d`

# This section will move Access Logs #

cd $HTTPD_LOGS_DIR/access
pwd
for i in `find . -name "*.$YYMMDD" -atime +0`
do
 gzip $i
 mv *.gz $HTTPD_LOGS_DIR/logs_backup/.
done
==================================


Can any body suggest me a good Shell script to rotate the Apache logs...


Waiting for a quick Response....

Thanks
Deepak
deepakjena_2003Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

hernst42Commented:
Hi deepakjena_2003,
You can use the rotatelogs(2) command that comnes with apache(2) to write to the logfile. After logrotet has begun a new file you can zip the old one. If you change a log-file manual you need to reload the webserver so he closes the old file-pointer and starts with a new one. That might be your problem with your script.

Cheers!
0
deepakjena_2003Author Commented:
Hi,

i have around 20 domains configured in a single box.

So separate logs are generate for separate Domains.

Log files are generated in different fashion as i have mentioned in the scripts...

So based on this i need a script to automate the proces....

Thanks
Deepak
0
hernst42Commented:
If you reload the webserver after you have archived the files your script should work. Some infomation might be lost during the time you started archiving and the reload of the webserver
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

deepakjena_2003Author Commented:
Problems i am facing from my existing scripts.....

1)  Log files are created automatically once i delete the log files after i archive these logs.(without   restarting apache)
2)  sometimes getting incorrect timestamp for the log files.
3)  i tried "> logfile" option also without deleting the log files, but i didn't workout.

Etc......

Thanks
Deepak
0
deepakjena_2003Author Commented:
1)  Log files are not created automatically once i delete the log files after i archive these logs.(without   restarting apache)
0
cyb3rj0hnCommented:
Why are you writing a script to do this, when you can use Linux's built in utility called logrotate. All you need to do is add the files you would like to be rotated to /etc/logrotate.conf. Please refer to the following link for all the details:

http://www.linuxforum.com/man/logrotate.8.php

A simple config that I use all the time to add to /etc/logrotate.conf would be:

"/var/log/httpd/somedomain-access.log" {
        rotate 5
        weekly
        compress
        postrotate
                     /sbin/killall -HUP  httpd
         endscript
}

This will keep 5 weeks worth of logs in a compressed format on the server. Please note that the reason for the kill -HUP is so that Apache can establish the new log file on the restart of Apache. If Apache is not restarted a new log file will not be created. After adding the above to /etc/logrotate.conf you can verify/test the config by issuing the following command:

logrotate -f /etc/logrotate.conf

Hope this helps. Cheers,

John


0
ChrisDrakeCommented:
The last guy (cyb3rj0hn) has the correct answer:  you should be using logrotate
0
deepakjena_2003Author Commented:
Hi John,

here i am putting the access log line from my httpd.conf file.....
------------------------------------------------------------------------
CustomLog "| /apps/apache/2.0.48/bin/rotatelogs /data/access/abc.com_access.%Y%m%d 86400" common
------------------------------------------------------------------------

above line create access log for abc.com in the format "abc.com_access.20041217"

so daily one log file is created for this domain.

Requirement :
i want to compress and move all logs to different folder(ex: /data/log_backup ) except current date logs and also delete all
logs from "/data/log_backup" folder which are 7 days old.


I guess this will not work out for me, so i have to remove "rotatelogs" implementation from httpd.conf file and follow your technique.

i have some doubts....

1) is "logrotate" calles to "/etc/logrotate.conf" file ?
2) shall i write a script file which will implement "logrotate" and call that script file in crontab ?

Still in Doubt...

Please give me the steps to implement "logrotate" and "logrotate.conf" for this...

Thanks
Deepak



0
hernst42Commented:
I would use logrotate (yes it's logrotate.conf) to compress and rename the logfiles. Then have a second script within cron which moves the logfiles to the backup directories and then deletes old files.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
cyb3rj0hnCommented:
I agree with hernst42.
0
deepakjena_2003Author Commented:
Thanks for all your help....
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Linux

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.