asked on

Apache overloaded with unwelcome requests: ?DoS attack

Our Apache 2.0.47 server is receiving a huge number of requests (average of a couple per second all day) for pages unrelated to the hosted websites. This was filling up the access_log and error_log with entries like:

63.13.131.217 - - [10/Sep/2004:08:46:52 -0700] "GET http://l35.login.scd.yahoo.com/config/login?.redir_from=PROFILES?&.tries=1&.src=jpg&.last=&promo=&.intl=us&.bypass=&.partner=&.chkP=Y&.done=http://jpager.yahoo.com/jpager/pager2.shtml&login=a-rt&passwd=316 HTTP/1.0" 200 18166 "-" "-"

I downloaded the latest patches (under Linux Mandrake 9.2) and the messages keep going but the server is now replying with a 403 (forbidden) message:

61.241.238.124 - - [10/Sep/2004:10:30:45 -0700] "GET http://dedicatedhits.com/search.php?username=aigo&keywords=Domain+Registration HTTP/1.1" 403 373

The current error_log messages:

[Fri Sep 10 10:28:14 2004] [error] [client 80.182.37.222] client denied by server configuration: proxy:http://edit.europe.yahoo.com/config/login?.redir_from=PROFILES?&.tries=1&.src=jpg&.last=&promo=&.intl=us&.bypass=&.partner=&.chkP=Y&.done=http://jpager.yahoo.com/jpager/pager2.shtml&login=s_newman&passwd=mike

Questions:
1) What is going on here? (Hacked...?)
2) With the 403 messages, the next entry is the return message size, usually about 0.5KB. Can I safely assume that the content is simply the "forbidden" message text - not some other stuff?
3) Can anything else be done to prevent the unwanted traffic?
4) Is more serious work needed - reinstalling?

The error_log has this "client denied by server configuration: proxy:" string. Is this related to the Apache settings? Should they be changed? Is the following the place to look/change:

<IfDefine PERLPROXIED>
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteRule ^proxy:.* - [F]
RewriteRule ^(.*\/perl\/.*)$ http://%{HTTP_HOST}:8200$1 [P]
RewriteRule ^(.*\/cgi-perl\/.*)$ http://%{HTTP_HOST}:8200$1 [P]
</IfModule>
</IfDefine>

I have tried adding lines to the hosts.deny file, but the number of IP addresses is too large.
The firewall only allows Web and SSH through. The server is set to listen on 80 and 443.

CajunBill

As your title suggests, it sounds like a DOS attack, not a hack of your system. In fact, it sounds like DDOS.

Here is a conceptual solution - let me know what you think.

The basic idea would be to not reply at all to the attacks. Replying, even with 403, just shows them they reached you.
If you stop replying, the attacks may slow down.
At the least, it will have the attackers waiting for the replies instead of able to process and re-attack.
It would also remove a little load off your server.

You could try putting another server in front of the real one to just filter out the attacks, again, the idea is to remove load from your real server.

HTH - Cajun

apgw

ASKER

Thanks for the comments. I am not sure how I would put in another server as a filter: would this be on the same machine? Also, upstream of the server is the router - maybe that is the place for a block of some kind, so that the packets don't reach the server at all. Currently it has the following firewall rules in place:

Description Source Destination Protocol
"Allow Virtual Server FTP" WAN,* LAN,192.168.0.244 TCP,21
"Allow Virtual Server HTTP" WAN,* LAN,192.168.0.244 TCP,80
"Allow Virtual Server HTTPS" WAN,* LAN,192.168.0.244 TCP,443
"Allow SSH" WAN,* LAN,192.168.0.244 TCP,22
"Allow to ping WAN port" WAN,* LAN,192.168.0.244 ICMP,8
"Allow msmsgs (192.168.0.194: 14830) 34 *.* LAN,192.168.0.194 UDP, 14830 - 34041
"Allow msmsgs (192.168.0.194: 7483) 209 *.* LAN,192.168.0.194 TCP, 7483 - 20957
"Deny Default" *.* LAN,* IP (0),*
"Allow Default" LAN,* *.* IP (0),*

The last 2 lines are not editable, but one can add new allow/deny lines.
Upstream of that is the Cisco switch. Possibilities here?

CajunBill

It depends on the firewall. Really, you need something more sophisticated than a simple firewall, because you want to block based on the message content, not the IP addresses (there were too many, you said).

As for putting a server upstream, I had in mind an actual physical server, so as to take the load off the real server. It is your call as to whether the load is so damaging to your business as to make it worthwhile to put in another server.

I will look more into what you can do. But being the object of a Distributed DOS attack is special - how did you achieve that? You must have a popular website for someone to go to that trouble.

apgw

ASKER

I am thinking that the site was found by some automated system because of the earlier hole in the security (it was returning 200 messages [OK]). Since installing the patches (it is now returning 400 messages) the traffic is slowing down slowly - instead of a few per second it is more like one every 2-3 seconds, as if our automated 'hacker' was counting refusals and dropping the requests after a certain period. So the impact on website page retrieval is now minimal.

I am surprised that no one else gave any suggestions on this question! Any other ideas?

ASKER CERTIFIED SOLUTION

CajunBill

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial