Link to home
Start Free TrialLog in
Avatar of BharathKumarRaju DasaraRaju
BharathKumarRaju DasaraRajuFlag for India

asked on

how to include conditional log rotate in liunx.

Hi Team,

I am little bit confused with log rotate, Could anyone help me in doing log rotate as below.

1. I would like to write a condition inside linux log rotate that first it need to check the log file size , if it is more than 1GB then it should gzip the log file and should purge it.
2. If the above condition didn't match for the specified log files i would like to retain the 3 days of log files and remove more than 3days of log files.

I didn't know how to put condition to handle this?

Regards,
Bharath.
Avatar of BharathKumarRaju DasaraRaju
BharathKumarRaju DasaraRaju
Flag of India image

ASKER

Some i tried as below ... but how can i use condition first as size and then time in days ?

/paas/logmsgs /paas/basicauth.log /paas/server.out
{
        size 1G
        notifempty
        missingok
        rotate 3
        #copytruncate
        compress
        create 755 appuser appuser 

}

Open in new window

I have tried some complex log rotate as below. Could you suggest me how to put if condition first to check based on size and it is not matched i have to do based on days.

/paas/logs/*.log /paas/default/logs/*.log{
        daily
        missingok
        rotate 5
        copytruncate
        compress
        maxage 5
        dateext
        #dateyesterday
        prerotate
        find /paas/logs/*.log.* -type f -mtime +3 -exec rm -f  {} \;
        find /paas/logs/*.log  -type f  -mtime +3 -exec rm -f  {} \;
        find /paas/default/logs/*.log.* -type f -mtime +3 -exec rm -f  {} \;
        find /paas/default/logs/*.log  -type f  -mtime +3 -exec rm -f  {} \;
        endscript
		}

Open in new window

Avatar of arnold
Rotate sets how many historical log copies you want, setting it to 1 will only maintain one version it will make no sense to consume resources to compress.
Your find commands are look for regular files, but since you compress, the -f will not match...

Lets try from the beginning
Waiting for a log file to grow to 1GB is often unnecessary. It might be better to maintain access to the same set of data by rotating the logs when size exceeds 200MB while maintaing 5 prior.

If need no prior copy of life, why not disable logging unless needed for purposes of debugging issues.
You want to compress big file but keep 3 maximum? That cannot do with logrotate.
@Team,

First thanks a lot for your replies.

What i am trying to do is. I would like to rotate log files basically. My Target is i need to rotate logs based in size lets say 50MB not hourly,daily,weekly or monthly in deed every minute. Becuase i am not sure when the log file grows hugely...my basic target logrotate has to see the size lets say 50MB rotate it...if it's not 50MB for 3 days or 10days don't do any rotate....i need to put condition like only rotate based on size 50MB to check every minute...is there anyway in logrotate?

your help would be greatly apprecited?

i tried like

/paas/*.log {
maxsize 50M
roatate 3
compress
}

but above didn't worked... i knew i have to do some scripting here like in sharedscripts firstaction endscript lastaction endscript am i right?
any suggestions please?
At certain point of time my log file growing very hugely . i am completely clueless how to handle it ? since i put log rotate daily i need to wait for till next day to do rotation :( :(
I am testing like below...my logfiles in below path are

root@hklvatedm01[test] # pwd
/softwareag/brokerdata/inst1/test

root@hklvatedm01[test] # ls -rtlh
total 31M
-rw-r----- 1 webmadmn webmadmn 170K Mar 19 23:00 stats20170319.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 19 23:55 stats20170316.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 19 23:55 stats20170317.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 19 23:55 stats20170318.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 19 23:55 Export-03-17-2017-06-25-09.log
-rw-r--r-- 1 webmadmn webmadmn    0 Mar 19 23:55 install.log
-rw-r--r-- 1 webmadmn webmadmn    0 Mar 19 23:55 schema.log
-rw-r--r-- 1 webmadmn webmadmn    0 Mar 19 23:55 soap.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 20 00:00 caf.log
-rw-r--r-- 1 webmadmn webmadmn  19K Mar 20 22:37 audit.log
-rw-r----- 1 webmadmn webmadmn 340K Mar 20 22:37 _problems_.log
-rw-r----- 1 webmadmn webmadmn 359K Mar 20 22:37 errors.log
-rw-r----- 1 webmadmn webmadmn  29M Mar 20 22:46 _full_.log
-rw-r----- 1 webmadmn webmadmn 169K Mar 20 22:51 stats.log
-rw-r----- 1 webmadmn webmadmn 591K Mar 20 22:51 server.log

Above mentoned logs are lessthan 200k...my logrotate file is as below..

root@hklvatedm01[test] # cat /home/devops/testlog.sh
/softwareag/brokerdata/inst1/test/*.log
 {
    minsize 200k
    rotate 3
    compress
    copytruncate
    dateext
    notifempty
    create 644 webmadmn webmadmn
}

Open in new window


and after running the above logrotate even my files less than 200k got compressed :( :( like below...

root@hklvatedm01[test] # /usr/sbin/logrotate -f /home/devops/testlog.sh
root@hklvatedm01[test] # ls -rtlh
total 572K
-rw-r----- 1 webmadmn webmadmn    0 Mar 19 23:55 stats20170316.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 19 23:55 stats20170317.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 19 23:55 stats20170318.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 19 23:55 Export-03-17-2017-06-25-09.log
-rw-r--r-- 1 webmadmn webmadmn    0 Mar 19 23:55 install.log
-rw-r--r-- 1 webmadmn webmadmn    0 Mar 19 23:55 schema.log
-rw-r--r-- 1 webmadmn webmadmn    0 Mar 19 23:55 soap.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 20 00:00 caf.log
-rw-r--r-- 1 webmadmn webmadmn    0 Mar 20 22:57 audit.log
-rw-r--r-- 1 webmadmn webmadmn 2.4K Mar 20 22:57 audit.log-20170320.gz
-rw-r----- 1 webmadmn webmadmn    0 Mar 20 22:57 errors.log
-rw-r----- 1 webmadmn webmadmn  11K Mar 20 22:57 errors.log-20170320.gz
-rw-r----- 1 webmadmn webmadmn    0 Mar 20 22:57 _full_.log
-rw-r----- 1 webmadmn webmadmn 460K Mar 20 22:57 _full_.log-20170320.gz
-rw-r----- 1 webmadmn webmadmn    0 Mar 20 22:57 _problems_.log
-rw-r----- 1 webmadmn webmadmn 9.9K Mar 20 22:57 _problems_.log-20170320.gz
-rw-r----- 1 webmadmn webmadmn    0 Mar 20 22:57 server.log
-rw-r----- 1 webmadmn webmadmn    0 Mar 20 22:57 stats20170319.log
-rw-r----- 1 webmadmn webmadmn  30K Mar 20 22:57 server.log-20170320.gz
-rw-r----- 1 webmadmn webmadmn    0 Mar 20 22:57 stats.log
-rw-r----- 1 webmadmn webmadmn  23K Mar 20 22:57 stats20170319.log-20170320.gz
-rw-r----- 1 webmadmn webmadmn  26K Mar 20 22:57 stats.log-20170320.gz
root@hklvatedm01[test] #
Maybe look in the log file, it is not like potatoes that you plant and they grow uncontrollably. You need enough space to hold uncompressed log of rotation period.
yeah sure is there anyway we can handle it in logrotate?
Look at the options you use with logrotate, if not mistaken your run actually with forcing through the use -f
Possible conflicting directives
1) you say do not rotate if smaller than 200k via the minsize
2) but then near the end you set the option do not rotate if empty by way of notifempty
I think minsize eliminates the need for notifempty..

Look at logrotate options
Usually when one wants automatic logrotation one would place logrotatetion_application.conf files in /etc/logrotate.d/ directory
Then running logrotate -c /etc/logrotate.conf would be a way of testing your logrotation configuration.
@arnold  thanks a lot for my case maxsize would be apt.
SOLUTION
Avatar of arnold
arnold
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
is there anyway i can specify the timeext like dateext in logrotate so that my maxsize works as expected.
I need to use time and date as an extension of rotated logs. Right now i'm using dateext. but problem with dateext is that I can't achieve rotation more than once on same date/day. and I have to do rotation on hourly basis, is there anyway to this :(
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Thanks a lot. I found a solution with your help. at last i ahve used as below and it worked.

/paas/*.log
{
    maxsize 200k
    rotate 3
    daily
    compress
    copytruncate
    dateext
    dateformat -%Y%m%d-%s
    missingok
    notifempty
    create 644 webmadmn webmadmn
}