Apache - How do I Write a Custom Error Log

Posted on 2009-04-03
Last Modified: 2013-12-16

I am using Centos5.2/ Apache 2 and I would like a custom error log that records all unsuccessful htaccess login attempts. Nothing else. Just  unsuccessful htaccess login attempts. I have about 10 virtual hosts running and they are situated in:


Can anybody help me write a custom log.

Kind Regards,

Adrian Smith
Question by:lwfuk
  • 4
  • 3
  • 2
LVL 29

Expert Comment

by:Michael Worsham
ID: 24061453
When it comes to Apache and logs, always remember that you can only use the log formatting on the access_log.

The error_log cannot use log formatting.


Author Comment

ID: 24061962
Sorry Mwecomputers, I don't understand what that means. Are you saying that it is not possible to filter out apache htaccess failures and put them in a special file? As for the format - I just need to the ipaddress.

I want to use the special log to record hacker attempts so that I can block them with fail2ban. At the moment my log files record everything as [error] including a missing favicon or a php failure. If I ban people based on the main error log I'll ban everybody who visits my site(s).

LVL 27

Expert Comment

ID: 24062174
What is a "htaccess failure"? A .htaccess file is a per-directory configuration file where you put directives provided by different modules instead of putting the same directives into a <directory /some/path> container in your httpd.conf.
Master Your Team's Linux and Cloud Stack!

The average business loses $13.5M per year to ineffective training (per 1,000 employees). Keep ahead of the competition and combine in-person quality with online cost and flexibility by training with Linux Academy.


Author Comment

ID: 24062462
Dear caterham_www

Setup a .htaccess protected directory.

Try to login as bill gates (assuming he isn't a valid user)

Look at your error_log file.

You will see an error. - Something like "user bill gates does not exists"

I want to log those errors in a custom file.

LVL 29

Expert Comment

by:Michael Worsham
ID: 24062601
If the failures are listed in in your error_log, then you are going to have to use a custom script to read the error_log and parse out the information. Apache's log formatting is strictly for the access_log and does not apply to the error_log.

I did a quick research and found this Perl script for parsing the error_log and extracting failures:
LVL 27

Expert Comment

ID: 24062755
Ah, you're talking about HTTP Auth provided by mod_auth_basic. There is no "htaccess protection", HTTP auth is provided by mod_auth et al.

I can think of four approaches:

- Using piped logging to a program and the program analyzes the input and writes it into different files
- analyzing the error_log itself as already suggested
- Using mod_perl
- using a self written c module which provides an additional log; patching the module mod_auth_basic to use the additional log instead of the error_log.

Author Comment

ID: 24066435
Many thanks for comments.

I think a practical demo would help.

Here is a line from my error log. I tried to login as BillGates.

[Wed Apr 01 14:45:56 2009] [error] [client] user BillGates not found: /admin

Compare this to this line.

[Wed Apr 01 19:11:34 2009] [error] [client] File does not exist: /home/www/web8/web/favicon.ico

Apache is reporting both of these errors in the error_log. It must know that each is different because each has a different error message.

Therefore, there must be a way of:

a)Instructing apache to pipe all of the "user [X] not found" into a text file.

(or alternatively)

b)Getting apache to re write the [error] tag as [apache-auth]

That's what I would like to achieve.

I don't want to use a 2 stage approach (ie parsing log files) although thank you for the suggestions. It would be a waste of system resources.

A C demon might be a little lighter on resources - but again it would need extra resources and I am convinced that the must be a more elegant way

Can anydody help?
LVL 29

Accepted Solution

Michael Worsham earned 500 total points
ID: 24072921
By default, Apache customization for logs only applies to the CustomLog (aka access_log) file. That was the way it was written, so I highly doubt you are going to get the Apache development team to change it to apply to the ErrorLog as well.

About your only options would be...

1) Modify the Apache ErrorLog function source code directly possibly adding in the subroutine functionality that CustomLog does and recompile.

2) Look at using a 3rd party tool, whether it be a parser, daemon or some other program to do modify the output and rewrite it the way you would like to have it displayed.

Author Closing Comment

ID: 31566154
Dear All

Many thanks for your help on this issue.

Other experts on different forums have concluded the same and so in summary it seems that there is no simple solution.

Fortunately, I have found another solution to my issue which was to do with fail2ban. My original problem was that I had installed fail2ban but it wasnt detecting apache auth errors. I subsequently found that I can modify the fail2ban regex filters housed in the filters.d directory.
Thanks Again,
Adrian Smith

Featured Post

U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Postfix issues with spam/auth attempts under NAT 9 112
AWS - HAProxy- KeepAlived 5 54
Run DOS2UNIX and then execute the command 21 88
CentOS 7 Linux for HP DL380 G4 32Bits 7 49
If you've heard about htaccess and it sounds like it does what you want, but you're not sure how it works... well, you're in the right place. Read on. Some Basics #1. It's a file and its filename is .htaccess (yes, with a dot in the front). #…
The purpose of this article is to demonstrate how we can use conditional statements using Python.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

828 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question