How to use Fail2Ban to catch Apache 500 errors, then email a notification list

pkromer
pkromer used Ask the Experts™
on
I want to install that on my Ubuntu web server ( the web host already has a copy on there but cant config it to send me an email for this, so i have to have my own copy)... any tips on how to set it up so that I get an email whenever a specific domain on that box has a 500 error?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Fractional CTO
Distinguished Expert 2018
Commented:
There 2x parts to this.

1) Setup your MTA where messages are sent + actually deliverable, which likely means you'll use a Mail Relay Service like MailGun.

Otherwise, you'll have to setup your MTA + SPF + DKIM, then warm up your IP.

First verify you can send mail out of your MTA + have the email submission accepted by a Gmail address.

2) You'll have to play around with your Fail2Ban version to get this working. You'll do a few things...

a) Create an /etc/fail2ban/filter.d/apache-500-report.conf file containing something like this...

[Definition]

failregex = ^\S+:\d+ <HOST> .*(GET|HEAD|POST) .* HTTP.+\s+500\s+

ignoreregex =

Open in new window


b) Create an /etc/fail2ban/jail.conf stanza something like this...

[apache-500-report]
enabled  = true
port     = http,https
action   = action_mw[name=apache-500-report, port="http,https", protocol=tcp]
logpath  = %(apache_access_log)s tail
maxretry = 1

Open in new window


Note: The above requires a sendmail wrapper be installed + working.

Note: I only use unattended (never notification) recipes, so you might have to play around with the above a bit to get a notification only recipe working.

Author

Commented:
aye carumba... had no idea this would be so convoluted. Why in hell doesnt Apache already have this simple ability? Rhetorical question.
nociSoftware Engineer
Distinguished Expert 2018

Commented:
enabled needs to be true in the jail to enable it.
(Not rhetorical answer) fail2ban grew from a lot of specific  purpose build scanners as a "simple" solution. More complex ones are logminers like splunk.
How to Generate Services Revenue the Easiest Way

This Tuesday! Learn key insights about modern cyber protection services & gain practical strategies to skyrocket business:

- What it takes to build a cloud service portfolio
- How to determine which services will help your unique business grow
- Various use-cases and examples

David FavorFractional CTO
Distinguished Expert 2018

Commented:
@noci, Thanks for the catch about adding enabled = true. Recipe modified above.

@pkromer, Guess this might seem convoluted at first glance. Once you get into Fail2Ban, you'll likely use this facility for many applications.

Author

Commented:
Thanks. I did get it installed but am unsure of next step... my web host says I cant copy their etc/fail2ban config file, so Im not sure where to set my own config file up, or what command to run from where. I know its installed where I want it though, because I run fail2ban-client -h and I see all the help stuff. Im using https://www.fail2ban.org/wiki/index.php/Fail2Ban as docs.

Author

Commented:
crap, my web host says:
-------------------------------------------------------
Hello,

We cannot provide support for the configuration of fail2ban.  Please read the full documentation as modifying the default configuration directory is covered in their wiki.  If you are looking for a copy of the default configuration files, I would suggest looking in their github page.

  https://github.com/fail2ban/fail2ban
-------------------------------------------------------

That was in response to what i sent them below:

-------------------------------------------------------
Ok, Im now using the official fail2ban wiki for docs... it says:

Configuration
You can configure Fail2Ban using the [file:///etc/fail2ban configuration files].  It is possible to configure the server using commands sent to it by fail2ban-client.  The available commands are described in the [man:/fail2ban-client manual page].  Please refer to it or to the website.

But you say I cant use your etc/fail2ban file. Do you know what I use as an alternative?
-------------------------------------------------------

I don't see anywhere in the docs how to change any config location.
Distinguished Expert 2017

Commented:
Much also depends on how your logs in Apache are currently handled.
Others addressed the post processing when the

Another option is either passing the log throu rsyslog within which events can be filtered/evaluated and actions taken.

Though often a 500 error is an internal web server issue and IMHO it will not be ir should be a target of who shoukd be banned.

Commonly banning would be used to limit a source if causing issues inordinate number of requests.

Are you trying to get notification when your setup experiences server (Apache/processing) related error.

Author

Commented:
trying to get notification when our Symfony web code fails.
Distinguished Expert 2017

Commented:
Look at either passing log entries through a program that evaluates the entry as it cones.

The other straight and simple option is to use a custom 5xx.php page as an example which when triggered will generate an email to you with info Apache has for the reason.
The cause will be part of the environment variables.
David FavorFractional CTO
Distinguished Expert 2018

Commented:
@pkromer, Fail2Ban is fairly easy to work with. If your hosting company can't help, just do this work yourself.

Tip: Building up your Fail2Ban skill can be useful for all manner of projects.
nociSoftware Engineer
Distinguished Expert 2018

Commented:
The server can be run with -c somefig-file

(and a few more options for command & control)    check man fail2ban-server

Author

Commented:
@David Favor,

Your first step said:

Create an /etc/fail2ban/filter.d/apache-500-report.conf file containing something like this...

[Definition]

failregex = ^\S+:\d+ <HOST> .*(GET|HEAD|POST) .* HTTP.+\s+500\s+

ignoreregex =

So where would I place that file after i create it?
nociSoftware Engineer
Distinguished Expert 2018

Commented:
/etc/fail2ban/filter.d    for filters & jail.d for jails.
with a name ending in .local

See man jail.conf  for more info.
(fixed typo)

Author

Commented:
So enter "man fail2ban.conf" on the command line?
nociSoftware Engineer
Distinguished Expert 2018

Commented:
Yes as a command on the command line.

Author

Commented:
ran the command, got error:

No manual entry for jail.conf
David FavorFractional CTO
Distinguished Expert 2018

Commented:
Where you place these files will vary depending on Distro you use.

Normally there will be an /etc/fail2ban/filter.d where the filter file will go.

Then some distros use /etc/fail2ban/jail.d, while most just use /etc/fail2ban/jail.conf to collect all jail entries, as jail entries tend to be simple.
David FavorFractional CTO
Distinguished Expert 2018

Commented:
Tip: Wow this is going to sound bad + here goes...

Most hosting company personnel are nearly 100% incompetent.

Best if you just relax + take on the attitude best to rely on yourself, rather than hosting company Dilbert-esque characters.

This suggests you pick technologies + learn them fully.

Fail2Ban is a great starting point as Fail2Ban is powerful + fairly easy to understand.

Over time always ask for help... and be sure to ask from the standpoint of expanding your skillset with the idea in mind you will eventually become your own best admin staff.
nociSoftware Engineer
Distinguished Expert 2018

Commented:
If the doc kits are not installed on your system .. ok try this link:
https://www.systutorials.com/docs/linux/man/5-jail.conf/
David FavorFractional CTO
Distinguished Expert 2018

Commented:
Try this to get your docs working...

apt-get install man-db

Open in new window


Or use http://manpages.ubuntu.com/manpages/bionic/man1/fail2ban.1.html which contains links at bottom of man page to all Fail2Ban related man pages.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial