Solved

Why Is Logrotate Not Working?

Posted on 2009-05-19
18
1,456 Views
Last Modified: 2013-12-16
I am running CentOS 5 and have set up a custom logrotate script however its not working properly.  It seems to do the rotating of the logs - however it does not delete the OLD ones "ie: rotate 12".  Instead it keeps all of them and keeps compressing them.

I have setup the following in /etc/logrotate.d/asterisk
/home/asterisk/asterisk-bin/log/asterisk/* /home/asterisk/asterisk-bin/log/asterisk/cdr-csv /home/asterisk/asterisk-bin/log/asterisk/cdr-custom {
missingok
sharedscripts
monthly
rotate 12
postrotate
asterisk -rx "logger rotate" > /dev/null 2> /dev/null
endscript
}

With the above code I expect to only have 12 versions of any given log in the paths specified.  However I get logs like this.
-rw-r--r-- 1 asterisk asterisk        0 Dec  1 04:02 queue_log.8.1.5
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.2.1
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.2.1.1
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.2.1.1.1
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 queue_log.8.2.1.1.1.1
-rw-r--r-- 1 asterisk asterisk        0 Mar  1 04:02 queue_log.8.2.1.1.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.2.1.2
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 queue_log.8.2.1.2.1
-rw-r--r-- 1 asterisk asterisk        0 Feb  1 04:02 queue_log.8.2.1.3
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.2.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.2.2.1
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 queue_log.8.2.2.1.1
-rw-r--r-- 1 asterisk asterisk        0 Mar  1 04:02 queue_log.8.2.2.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.2.3
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 queue_log.8.2.3.1
-rw-r--r-- 1 asterisk asterisk        0 Jan  1 04:02 queue_log.8.2.4
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.3
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.3.1
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.3.1.1
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 queue_log.8.3.1.1.1
-rw-r--r-- 1 asterisk asterisk        0 Mar  1 04:02 queue_log.8.3.1.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 queue_log.8.3.2
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 queue_log.8.3.2.1
-rw-r--r-- 1 asterisk asterisk        0 Feb  1 04:02 queue_log.8.3.3
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.1.1
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.1.1.1
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.1.1.1.1
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 messages.4.1.2.1.1.1.1.1
-rw-r--r-- 1 asterisk asterisk        0 Mar  1 04:02 messages.4.1.2.1.1.1.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.1.1.2
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 messages.4.1.2.1.1.2.1
-rw-r--r-- 1 asterisk asterisk        0 Feb  1 04:02 messages.4.1.2.1.1.3
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.1.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.1.2.1
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 messages.4.1.2.1.2.1.1
-rw-r--r-- 1 asterisk asterisk        0 Mar  1 04:02 messages.4.1.2.1.2.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.1.3
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 messages.4.1.2.1.3.1
-rw-r--r-- 1 asterisk asterisk        0 Jan  1 04:02 messages.4.1.2.1.4
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.2.1
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.2.1.1
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 messages.4.1.2.2.1.1.1
-rw-r--r-- 1 asterisk asterisk        0 Mar  1 04:02 messages.4.1.2.2.1.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.2.2
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 messages.4.1.2.2.2.1
-rw-r--r-- 1 asterisk asterisk        0 Feb  1 04:02 messages.4.1.2.2.3
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.3
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.3.1
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 messages.4.1.2.3.1.1
-rw-r--r-- 1 asterisk asterisk        0 Mar  1 04:02 messages.4.1.2.3.2
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.2.4
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 messages.4.1.2.4.1
-rw-r--r-- 1 asterisk asterisk        0 Dec  1 04:02 messages.4.1.2.5
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.3
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.3.1
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.3.1.1
-rw-r--r-- 1 asterisk asterisk        0 May  1 04:02 messages.4.1.3.1.1.1
-rw-r--r-- 1 asterisk asterisk        0 Apr  1 04:02 messages.4.1.3.1.1.1.1
-rw-r--r-- 1 asterisk asterisk        0 Mar  1 04:02 messages.4.1.3.1.1.2

But there are literally hundreds and hundreds and hundreds of these things.  Why is my log rotate not working properly?  I only ever want to have 12 copies of each log, not hundreds.

Can someone please help?
0
Comment
Question by:purestealth
18 Comments
 
LVL 84

Expert Comment

by:ozo
ID: 24425756
can you post your  custom logrotate script?
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 24425927
The "postrotate" script is executed after each rotation of a logfile done by 'logrotate'
With "logger rotate" you force asterisk to  rotate and reopen all the logfiles on its own one more time, because "logger rotate" invokes asterisks own logfile rotation method.
If you insist in using 'logrotate', change the postrotate command to "logger reload", which only reopens the logs and does not rotate them.

wmp

0
 
LVL 16

Expert Comment

by:ai_ja_nai
ID: 24427322
use the maxage directive, in which you specify how many days to wait before deleting
0
 

Author Comment

by:purestealth
ID: 24433522
from woolmilkporc's suggestion I have changed "logger rotate" to "logger reload", and I have also added the maxage directive as suggested by ai_ja_nai.

I will now let the logger do its thing and see what happens.  I hope I have understood both of you properly.  Thanks for the tips, I will see how it goes now.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 24433642
Well,
might be that I have not been clear enough -
what you did is rotating the logs twice, once by means of logrotate, and once by means of "logger rotate".
And moreover, the rotation by "logger rotate" was done every time logrotate processed a single log (because of the postrotate directive), and that for each of asterisk's logfiles. Thus those many files, as the above leads to ([number of logfiles] * [number of logfiles]) new files at each logrotate run.
I fear 'maxage' will not change anything of that behaviour, but "logger reload" will.
And, don't forget to remove those useless empty logs, to better see the effect of your changes.

Good luck!

wmp

0
 

Author Comment

by:purestealth
ID: 24434633
thanks woolmikporc, I better understand you now.  So asterisk has its own built in logrotate, and your explanation makes sense.

My only remaining questions would be:

1) how do I configure how many copies of each log file are kept before being deleted?  

2) Will the logs still be rotated now that we changed it to "logger reload" instead of "logger rotate"?  

3) Is it safe to delete all the log files in the Asterisk log directory (by safe I mean, it wont impact Asterisk and it will start to make new logs again)?  

4) Lastly will the old log files be deleted or will they continue to grow like they are currently.

Thanks again for your assistance with this.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 24435045
1) with logrotate this is the 'rotate' directive. In your config 'rotate 12' would mean keeping 12 versions.
2)  logs will be rotated by 'logrotate'. I assume you have a cronjob for logrotate, so it will keep working. "logger reload" forces asterisk to re-read its logger config and to reopen its logs. This makes sense after logrotate processing, to get a clean logfile status. The place where you put it in ('postrotate') is just the right one for that.
3) if you delete the logfiles by hand, you should also run 'asterisk -rx "logger rotate"' by hand. As I wrote under 3) above, this will make everything fresh and clean.
4) this is just the purpose of logrotate - new empty files will be created to be used by asterisk each time logrotate processes a log, 12 versions will be kept (they get an appropriate suffix), excess ones will be deleted. You use the 'monthly' directive, which means that one month's worth of log data will be contained in a single log before it will be rotated.
4a) there are other possibilities than to just delete the excess logs. You can e.g. tell logrotate to move them to a configurable folder, or even to mail them to you.
Read more about all this in logrotate's manpage, it's very informative -
http://linux.die.net/man/8/logrotate
wmp
 
 
0
 

Author Comment

by:purestealth
ID: 24442649
This is great, thanks so much for your detailed information woolmilkporc

Here is my revised Logger Script, can you please check everything looks good now?

/home/asterisk/asterisk-bin/log/asterisk/* /home/asterisk/asterisk-bin/log/asterisk/cdr-csv /home/asterisk/asterisk-bin/log/asterisk/cdr-custom {
missingok
sharedscripts
monthly
rotate 12
postrotate
asterisk -rx "logger reload" > /dev/null 2> /dev/null
maxage 12
endscript
}

As I understand it, all the log files will collect a months worth of log data then be rotated, and the system will keep 12 copies thus ensuring I have a full year's worth of logs before the "oldest" log is deleted.

I wiped out all the old log files which oddly enough were all ZERO bytes (they had no data in them) - and then from the Asterisk CLI I did a "logger reload" and I can see the new log files which have actual data in them now.

If my understanding is correct, and my script pasted above looks good, then we are good to go :) - All though I guess it would take 1 month before I see if its working properly or not.....
0
How to improve team productivity

Quip adds documents, spreadsheets, and tasklists to your Slack experience
- Elevate ideas to Quip docs
- Share Quip docs in Slack
- Get notified of changes to your docs
- Available on iOS/Android/Desktop/Web
- Online/Offline

 
LVL 68

Expert Comment

by:woolmilkporc
ID: 24443474
1) You should be very careful with the wildcard in '/home/asterisk/asterisk-bin/log/asterisk/*' Do you really need it? It could lead to logrotate processing already rotated files! At least use something like '*_log'
Note that the asterisk should not be at the end, to avoid processing of alredy suffixed old logs.
I would recommend something like:

/home/asterisk/asterisk-bin/log/asterisk/*_log /home/asterisk/asterisk-bin/log/asterisk/*messages /home/asterisk/asterisk-bin/log/asterisk/cdr-csv /home/asterisk/asterisk-bin/log/asterisk/cdr-custom { ...

The idea is to catch all filenames of to-be-rotated logs without a wildcard at the end of a filename!

An alternative was to use the 'olddir [/dir/for/rotated/logs]' directive, so that rotated logs go away from the directory examined by logrotate.
2) 'maxage' is at the wrong place. The entries between 'postrotate' and 'endscript' form an instruction block to be executed by a shell, not by logrotate. Thus a directive like 'maxage' cannot be placed there. If you want to keep it (see below), please place it somewhere above the 'postrotate/endscript' block.

3) 'maxage' is, as I see it, somewhat redundant to 'rotate n'. It is obviously some kind of 'whatever comes first' option. Moreover, 'maxage' is expressed in days, so you should at least use 'maxage 365' or the like.

From the manpage:

 maxage count
              Remove  rotated  logs  older  than <count> days. The age is only              checked if the logfile is to be rotated.

rotate count
              Log files are rotated <count> times before being removed or mailed
              to the address specified in a mail directive. If count is 0, old
              versions are removed rather than rotated.

I never used 'maxage'. Maybe I'll try it some day ...

4) Basically you're right, you'll have to wait a bit. But not a full month, since monthly rotation is done at some point at or after the first of a month, depending on how often you run 'logrotate'.

I always thought logrotate was simple and easy, but the more I write about it ...

wmp

0
 

Author Comment

by:purestealth
ID: 24451609
Hello again WMP

You posted some good information.

1) This makes total sense and I agree with your statements, I have updated my script accordingly, please see below.

2) Ah, good catch I was not aware of that, thanks for noticing that and correcting the mistake.

3) Yes, I see what you mean, I don't see a point to keep it if its not necessary

4) Gotcha, and I checked on it again a few minutes ago, the logs seem to look good they are growing and are not zero bytes.

Here is the revised script.


/home/asterisk/asterisk-bin/log/asterisk/*_log /home/asterisk/asterisk-bin/log/asterisk/*messages /home/asterisk/asterisk-bin/log/asterisk/cdr-csv /home/asterisk/asterisk-bin/log/asterisk/cdr-custom {
missingok
sharedscripts
monthly
rotate 12
postrotate
asterisk -rx "logger reload" > /dev/null 2> /dev/null
endscript
}

Does that look good to you now?  Thanks again for all your help with this, I am looking forward to seeing what will happen on the first rotation of the logs.
0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 24451799
Hi purestealth,
looks perfect now!
The only thing you should double check is whether you will really catch all concerned logs, I mean, are there files not ending in ...._log or .....messages? (Apart from the two cdr-... files, that's OK)
Besides that, as I said, perfect!
Have much pleasure at the 1st of June,
Cheers
wmp
 

0
 

Author Comment

by:purestealth
ID: 24452011
Thanks again WMP, yup I can confirm the only logs we need end in '_log' or 'messages' so we are good to go.

I am going to check on this at the first rotation and post back the results.

Thanks so much for your effort on this, I will definitely let you know if it works or if there are any glitches.  

Until then, all the best :)
0
 

Author Comment

by:purestealth
ID: 24890250
This question was defiantly not abandoned, I have been letting the logger run to watch and see what it does, since it does not rotate daily it takes some time to see if its working properly.

WMP: here is the results of my log directory, how does it look to you?  If all looks good I can accept your comment as the solution, thanks again for your help and patients while I tested this out.

ls -la /home/asterisk/asterisk-bin/log/asterisk
drwxrwxr-x 4 asterisk asterisk  442368 Jul  1 04:02 .
drwxrwxr-x 3 asterisk asterisk    4096 Jun  2  2008 ..
drwxrwxr-x 2 asterisk asterisk    4096 Jun 17 19:13 cdr-csv
drwxrwxr-x 2 asterisk asterisk    4096 Jun  2  2008 cdr-custom
-rw-r--r-- 1 asterisk asterisk       0 Jul  1 04:02 event_log
-rw-r--r-- 1 asterisk asterisk       0 Jun  1 04:02 event_log.1
-rw-r--r-- 1 asterisk asterisk       0 May 21 11:28 event_log.2
-rw-r--r-- 1 asterisk asterisk 3102087 Jul 19 13:22 messages
-rw-r--r-- 1 asterisk asterisk 4467466 Jul  1 03:55 messages.1
-rw-r--r-- 1 asterisk asterisk 2411044 Jun  1 03:57 messages.2
-rw-r--r-- 1 asterisk asterisk  882495 Jul 19 13:19 queue_log
-rw-r--r-- 1 asterisk asterisk 1402038 Jun 30 22:53 queue_log.1
-rw-r--r-- 1 asterisk asterisk  368255 May 31 19:12 queue_log.2
0
 
LVL 68

Accepted Solution

by:
woolmilkporc earned 500 total points
ID: 24890514
OK,

the event.logs don't contain any entry.

If you feel that this is correct, everything looks fine.

No more .......4.1.2.1.1.1.1.1.... files, which were obviously the results of a misconfiguration.

Congrats!

wmp


0
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 24892975
One more thing ...
to get rid of the empty rotated event.logs, try and add the 'notifempty' option, on a separate line just following 'missingok'.
Thx for the points!
Cheers
wmp
 
0
 

Author Comment

by:purestealth
ID: 24897969
Oh, excellent thanks for the tip and for your efforts on this, much appreciated.
0

Featured Post

Backup Your Microsoft Windows Server®

Backup all your Microsoft Windows Server – on-premises, in remote locations, in private and hybrid clouds. Your entire Windows Server will be backed up in one easy step with patented, block-level disk imaging. We achieve RTOs (recovery time objectives) as low as 15 seconds.

Join & Write a Comment

This document is written for Red Hat Enterprise Linux AS release 4 and ORACLE 10g.  Earlier releases can be installed using this document as well however there are some additional steps for packages to be installed see Metalink. Disclaimer: I hav…
Little introduction about CP: CP is a command on linux that use to copy files and folder from one location to another location. Example usage of CP as follow: cp /myfoder /pathto/destination/folder/ cp abc.tar.gz /pathto/destination/folder/ab…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now