Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17


Where to look for System Error Messages

Posted on 2002-03-19
Medium Priority
Last Modified: 2010-04-21
I've worked with UNIX as an end user of applications, but haven't really done much administration.  I've inherited a system and want to know where I should look (files/directories) to see if there are any error, informational or warning messages being recorded.  Can someone point me in the direction of all the places I should be looking?

What prompted this was trying to figure out why all three of my UNIX web servers running apache were not writing information to their access_log file.  I "think" I may have figured that one out.. I believe there is a 2 gig file size limitation in UNIX and all three files on each server are the same size: 2147483647

If this is the case would there be some UNIX log file that would throw an error stating the the max file size had been reached?  I just want to be able to look at messages from the OS level to see if the system is healthy.


Question by:lphillips120898
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2

Author Comment

ID: 6879796
One more bit of information:

[root@penguin12 logs]# uname -a
Linux 2.2.16-22 #2 SMP Wed Jun 20 15:46:15 PDT 2001 i686 unknown
[root@penguin12 logs]# uname
[root@penguin12 logs]#

How can I tell which version of Linux?
LVL 40

Accepted Solution

jlevie earned 300 total points
ID: 6879850
You've found the exact cause of the failure to log messages. The file size shown above is the max file size that Linux can have, i.e., a 2GB file. I think there might be something in /var/log/messages or in your Apache error_log about the failure.

You probably need to institute a log rolling function either via a cron script or via logrotate to periodically roll your logs. Depending on the level of web activity that might be once a day, once a week, or even less often.
LVL 51

Expert Comment

ID: 6879895
as jlevie said: max file size exceeded for your linux 2.2.16 (see your uname -a output)

AFAIK, writing to files might still working, while most UNIX-tools complain about the file size. So there might not be a message in the logfiles (except if the file system is full too).
You either need a kernel with support for huge files, or some kind of log rotate.

Keep in mind that it will be hard to remove these files now, 'cause all standard UNIX-tools fail.
I only know of debugfs to do that. Take care !!

Author Comment

ID: 6880268
I was able to get some information from the /var/log/messages file - thanks.

Now base on the comments I got from both of you I am going to post a message on how to set up a cron job (haven't done in years).... so be on the lookout for it!


LVL 40

Expert Comment

ID: 6883497
How about a bonus for this question...

You can roll your apache logs by creating a script something like:

# Roll the web logs for the sites indicated
SITES="/opt/Apache /home/virt-site"

for s in $SITES; do
  /bin/mv $s/logs/access_log.5 $s/logs/access_log.6
  /bin/mv $s/logs/access_log.4 $s/logs/access_log.5
  /bin/mv $s/logs/access_log.3 $s/logs/access_log.4
  /bin/mv $s/logs/access_log.2 $s/logs/access_log.3
  /bin/mv $s/logs/access_log.1 $s/logs/access_log.2
  /bin/mv $s/logs/access_log.0 $s/logs/access_log.1
  /bin/mv $s/logs/access_log $s/logs/access_log.0

  /bin/mv $s/logs/error_log.5 $s/logs/error_log.6
  /bin/mv $s/logs/error_log.4 $s/logs/error_log.5
  /bin/mv $s/logs/error_log.3 $s/logs/error_log.4
  /bin/mv $s/logs/error_log.2 $s/logs/error_log.3
  /bin/mv $s/logs/error_log.1 $s/logs/error_log.2
  /bin/mv $s/logs/error_log.0 $s/logs/error_log.1
  /bin/mv $s/logs/error_log $s/logs/error_log.0

echo "Signalling apache"
kill -USR1 `cat $HTTPID`
sleep 300

for s in $SITES; do
  /bin/rm $s/logs/*.old

and execute the script from cron by including a line like:

59 23 * * * /path-to/script-like-above

The cron job ('crontab -e' will let you edit the crontab) above will run the script in the last minute of the last hour of every day. Other schedules are possible, see 'man crontab' for the meanings of the other fields if you want a different schedule.

The script can rotate the standard Apache logs (mine happen to be in /opt/Apache/logs) and any other logs that you might have set up for virtual sites (like /home/virt-site). The later feature is handy when you are hosting a number of virtual servers and you want the logs for each virtual site accessible to the maintainer of that site. When I set up a virtual site I create a site directory in the site maintainer's home dir that contains their cgi-bin, htdocs. and logs sub-dirs.

Another way to rotate Apache's logs is to change the httpd.conf file to pipe the logs to Apache's rotatelogs util. Having something like:

ErrorLog "| rotatelogs /opt/Apache/logs/error_log 86400"

in your httpd.conf file will cause the log to be rotated once a day. See the man page for rotatelogs for details.


Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A metadevice consists of one or more devices (slices). It can be expanded by adding slices. Then, it can be grown to fill a larger space while the file system is in use. However, not all UNIX file systems (UFS) can be expanded this way. The conca…
Java performance on Solaris - Managing CPUs There are various resource controls in operating system which directly/indirectly influence the performance of application. one of the most important resource controls is "CPU".   In a multithreaded…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Suggested Courses

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question