I've been trying to battle a bunch of spammers and brute force attacks on my VPS server with HostGator. I feel like a person who brought home a wifi router, plugged it in, and thought "it works, great!" Yet they don't change the default IP address, default password, wifi security, etc. That's I feel with this VPS. I'm not a server admin, and HostGator isn't helping me figure out how to secure this thing (even with offers to pay for service.)
So, with a little bit of digging, I found this ModSecurity stuff inside the WHM called ModSecurity. I had to enable it, and then I had to "add a vendor" which then gives me access to a bunch of different security settings I can turn on/off. When everything enabled, though, my sites on this server weren't functioning correctly, so I turned off everything, and then went through 1 service at a time, turning it on, and testing to see if any of the sites had issues with that particular rule.
At this point, I have everything except for the following enabled, because I had issues...
rules/REQUEST-10-IP-REPUTATION.conf
I’m pretty sure this one was enabled by default and was fine, but now that I everything disabled and I’m re-enabling one-by-one, I get this error when I try to enable this:
The system could not validate the new Apache configuration because httpd exited with a nonzero value. Apache produced the following error: Syntax error on line 31 of /usr/local/apache/conf/modsec_vendor_configs/OWASP/rules/REQUEST-10-IP-REPUTATION.conf:
ModSecurity: Found another rule with the same id
rules/REQUEST-20-PROTOCOL-ENFORCEMENT.conf
With this enabled, some features on my WordPress sites break. For example, the site stats feature that the Jetpack plugin provides (if you're familiar with that, by chance) breaks.
I have everything else still turned on right now, but what I'm finding is that people hitting my legitimate web services are getting failures. I can't figure out which individual rule is doing it, but when I disable the entire ModSecurity firewall it works again.
So, finally, my question is whether or not there's a way I can fine-tune these rules and allow legitimate traffic..?? I've been reading through this guide, but I just don't see anything other than simply turning on or off a rule. That seems counter-intuitive, though, and again, I can't seem to find which individual rule is causing my problem with people hitting my legitimate web services.
Any information on all of this would be greatly appreciated. Thanks!