Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17


Apache - How do I Write a Custom Error Log

Posted on 2009-04-03
Medium Priority
Last Modified: 2013-12-16

I am using Centos5.2/ Apache 2 and I would like a custom error log that records all unsuccessful htaccess login attempts. Nothing else. Just  unsuccessful htaccess login attempts. I have about 10 virtual hosts running and they are situated in:


Can anybody help me write a custom log.

Kind Regards,

Adrian Smith
Question by:lwfuk
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
LVL 29

Expert Comment

by:Michael Worsham
ID: 24061453
When it comes to Apache and logs, always remember that you can only use the log formatting on the access_log.

The error_log cannot use log formatting.


Author Comment

ID: 24061962
Sorry Mwecomputers, I don't understand what that means. Are you saying that it is not possible to filter out apache htaccess failures and put them in a special file? As for the format - I just need to the ipaddress.

I want to use the special log to record hacker attempts so that I can block them with fail2ban. At the moment my log files record everything as [error] including a missing favicon or a php failure. If I ban people based on the main error log I'll ban everybody who visits my site(s).

LVL 27

Expert Comment

ID: 24062174
What is a "htaccess failure"? A .htaccess file is a per-directory configuration file where you put directives provided by different modules instead of putting the same directives into a <directory /some/path> container in your httpd.conf.
What is a Denial of Service (DoS)?

A DoS is a malicious attempt to prevent the normal operation of a computer system. You may frequently see the terms 'DDoS' (Distributed Denial of Service) and 'DoS' used interchangeably, but there are some subtle differences.


Author Comment

ID: 24062462
Dear caterham_www

Setup a .htaccess protected directory.

Try to login as bill gates (assuming he isn't a valid user)

Look at your error_log file.

You will see an error. - Something like "user bill gates does not exists"

I want to log those errors in a custom file.

LVL 29

Expert Comment

by:Michael Worsham
ID: 24062601
If the failures are listed in in your error_log, then you are going to have to use a custom script to read the error_log and parse out the information. Apache's log formatting is strictly for the access_log and does not apply to the error_log.

I did a quick research and found this Perl script for parsing the error_log and extracting failures:
LVL 27

Expert Comment

ID: 24062755
Ah, you're talking about HTTP Auth provided by mod_auth_basic. There is no "htaccess protection", HTTP auth is provided by mod_auth et al.

I can think of four approaches:

- Using piped logging to a program and the program analyzes the input and writes it into different files
- analyzing the error_log itself as already suggested
- Using mod_perl
- using a self written c module which provides an additional log; patching the module mod_auth_basic to use the additional log instead of the error_log.

Author Comment

ID: 24066435
Many thanks for comments.

I think a practical demo would help.

Here is a line from my error log. I tried to login as BillGates.

[Wed Apr 01 14:45:56 2009] [error] [client] user BillGates not found: /admin

Compare this to this line.

[Wed Apr 01 19:11:34 2009] [error] [client] File does not exist: /home/www/web8/web/favicon.ico

Apache is reporting both of these errors in the error_log. It must know that each is different because each has a different error message.

Therefore, there must be a way of:

a)Instructing apache to pipe all of the "user [X] not found" into a text file.

(or alternatively)

b)Getting apache to re write the [error] tag as [apache-auth]

That's what I would like to achieve.

I don't want to use a 2 stage approach (ie parsing log files) although thank you for the suggestions. It would be a waste of system resources.

A C demon might be a little lighter on resources - but again it would need extra resources and I am convinced that the must be a more elegant way

Can anydody help?
LVL 29

Accepted Solution

Michael Worsham earned 2000 total points
ID: 24072921
By default, Apache customization for logs only applies to the CustomLog (aka access_log) file. That was the way it was written, so I highly doubt you are going to get the Apache development team to change it to apply to the ErrorLog as well.

About your only options would be...

1) Modify the Apache ErrorLog function source code directly possibly adding in the subroutine functionality that CustomLog does and recompile.

2) Look at using a 3rd party tool, whether it be a parser, daemon or some other program to do modify the output and rewrite it the way you would like to have it displayed.

Author Closing Comment

ID: 31566154
Dear All

Many thanks for your help on this issue.

Other experts on different forums have concluded the same and so in summary it seems that there is no simple solution.

Fortunately, I have found another solution to my issue which was to do with fail2ban. My original problem was that I had installed fail2ban but it wasnt detecting apache auth errors. I subsequently found that I can modify the fail2ban regex filters housed in the filters.d directory.
Thanks Again,
Adrian Smith

Featured Post

Will your db performance match your db growth?

In Percona’s white paper “Performance at Scale: Keeping Your Database on Its Toes,” we take a high-level approach to what you need to think about when planning for database scalability.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you've heard about htaccess and it sounds like it does what you want, but you're not sure how it works... well, you're in the right place. Read on. Some Basics #1. It's a file and its filename is .htaccess (yes, with a dot in the front). #…
You ever wonder how to backup Linux system files just like Windows System Restore?  Well you can use Timeshift in Linux to perform those similar action.  This tutorial will show you how to backup your system files and keep regular intervals. Note…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
Suggested Courses

670 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question