[Webinar] Streamline your web hosting managementRegister Today

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 865
  • Last Modified:

Delete lines older than a specific date from a Unix file.

The alert log file gets appended all the time. I want to do a automatic maintainence of the alert log file by going through the log and deleting lines OLDER than 30 or 60 or 90 days and keeping the same name. I do not want to pipe it to a new name.
0
KamalAgnihotri
Asked:
KamalAgnihotri
  • 3
  • 3
  • 3
  • +2
2 Solutions
 
slightwv (䄆 Netminder) Commented:
You probably 'can'  come up with a script to do this but why not just move it or delete it ever 30, 60 or 90 days?

Oracle will create a new one automatically if it doesn't exist.
0
 
johnsoneSenior Oracle DBACommented:
We used to do it with a cron job.  Every week a cron job would run and rename the current alert log with a date stamp.  Then we would go through and remove the older ones manually, but you could make that part of the cron job as well if you wanted.  We just wanted to be sure that if there was an issue we kept the logs until the issue was resolved and you couldn't really script around that.
0
 
KamalAgnihotriAuthor Commented:
You are correct. But when you have over 100 databases, you want to automate as much as possible. This is also a generic question. When a log file is continously appended and keeps growing and ultimately causes the diskspace hit the % full threshold.

What is the script/command that can be put in a cron job which can be run daily and will delete the lines older than 30 days. The good thing is that every line in the log file begins with a time stamp like this:

6/25/13 07:04:11 PM EDT [INFO] [EPAgent] Using Introscope installation at: /oracle/epagent9/.
6/25/13 07:04:11 PM EDT [INFO] [EPAgent] CA Wily Introscope(R) Version 9.1
6/25/13 07:04:11 PM EDT [INFO] [EPAgent] Copyright (c) 2012 CA. All Rights Reserved.
6/25/13 07:04:11 PM EDT [INFO] [EPAgent] Introscope(R) is a registered trademark of CA.
6/25/13 07:04:11 PM EDT [INFO] [EPAgent] Starting Introscope EPAgent...
0
The new generation of project management tools

With monday.com’s project management tool, you can see what everyone on your team is working in a single glance. Its intuitive dashboards are customizable, so you can create systems that work for you.

 
slightwv (䄆 Netminder) Commented:
Why are you not wanting to just move it or delete it?  It is so much easier than writing a script to edit it.
0
 
KamalAgnihotriAuthor Commented:
slightwv,

There are several ways of doing a task. What you said is one way, which is correct. I want to find anther way. Any suggestion on "another way" would be greatly appriciated.
0
 
slightwv (䄆 Netminder) Commented:
Sorry but I don't have a script around to do it the hard way.
0
 
johnsoneSenior Oracle DBACommented:
I don't have one that will delete only specific lines as well.  Also, be aware that messages in the alert log can span multiple lines and not every line has a date stamp.  That makes the task you are trying to do much more difficult.
0
 
simon3270Commented:
Editing active files is fraught with problems, particularly if new data arrives while the file is being changed.  Almost all "in-place" editors actually create a new file and rename the new file to have the same name as the old one.  If a process was writing to the old file and keeps the file open during your edit, it will continue to write to that old file, even if it has apparently been deleted. As soon as the process closes the file, all changes since the edit are lost.

You would be better off using something like the "logrotate" program available on many Linux/UNIX systems.  This automates the rotation of log files, optionally compressing old logs, and deleting old logs so that only a specified number are kept.  When it moves a file to a new name (to mark it as "old"), it can optionally run a command which tells the processes writing that log file to close their current file handle and reopen the log file with the original name (often by running "kill -1" on the process).
0
 
Mark GeerlingsDatabase AdministratorCommented:
I think the best two ways to solve this problem (at least for Oracle "alert.log" files is either:
1. use a simple shell script to rename the alert.log file each night, just before midnight, to a file name that includes the month and day.
2. use the UNIX log rotate mechanism to rename the alert.log file

In either case, the database will create a new "alert.log" file with the standard name the next time it tries to write to the file.

The same problem exists with the listener.log file for Oracle's TNS listener, but that one is more-complex to rename because of the problem that simon3270 mentioned.  I usually script a three-step process for these: "lsnrctl [name] stop", rename the log file to include the current month and day, then "lsnrctl [name] start" to create a new log file with the standard name.
0
 
simon3270Commented:
For the listener you can also use the copytruncate option for logrotate.  This blog entry shows how.  The actual control file used in this example is:
$ cat /etc/logrotate.d/oracle-listener
/u01/app/oracle/product/10.2.0/db_1/network/log/listener.log {
weekly
copytruncate
rotate 4
compress
}

Open in new window

"weekly" is how often the rotation is done (you could have other periods, such as "daily", or specify a maximum size).
"copytruncate" copies the old log file to a new one, then empties the original one - when the listener next writes a log entry, it writes to the start of the file.
"rotate 4" keeps 4 old files - if rotation creates a new "old" file while there are already 4 "old2 files,  the oldest is deleted.
"compress" gzips the "old" files to save space.  an alternative is "delaycompress" which leaves the most recent "old" file alone (to make searches easier, for example), but compresses any older "old" files.
0
 
johnsoneSenior Oracle DBACommented:
Actually, with the listener, you don't want to do a stop, rename, start.  This affects incoming connections as the listener is down.  It is for a short period of time, but it is down.

I use this method:

lsncrtl [name] set log_status off
rename listener.log
lsncrtl [name] set log_status on

You may miss a couple of messages, but you will not affect incoming connections.
0
 
Mark GeerlingsDatabase AdministratorCommented:
Thank you, simon3270 and johnsone.  That is two options that I wasn't aware of before for the listener.log file.  I plan to test both of those in non-PROD systems here today.
0
 
KamalAgnihotriAuthor Commented:
I like the Logrotate idea and I am going to try that. On Sun Solaris logrorate is logadm. I will keep this question open till I have the script created and tested.
0

Featured Post

Take Control of Web Hosting For Your Clients

As a web developer or IT admin, successfully managing multiple client accounts can be challenging. In this webinar we will look at the tools provided by Media Temple and Plesk to make managing your clients’ hosting easier.

  • 3
  • 3
  • 3
  • +2
Tackle projects and never again get stuck behind a technical roadblock.
Join Now