?
Solved

How to prevent Slowloris attack

Posted on 2010-01-09
15
Medium Priority
?
2,688 Views
Last Modified: 2012-05-08
Please help me prevent slowloris atacks on my server.
I have dedicated server with some websites running on it.
I have been under slowloris attack every day for about a month now.

- I have installed mod_antiloris .... it doesn't help
- I have installed mod_qos .... it gets me glibc errors and makes httpd halt or die
- I see i could install haproxy but i have no knowledge of setting up proxies (IP forwarding, URL rewriting, ...)
- I could use nginx or any other webserver .... impossible because it would take to much time for setting things up and learning all new configurations and differences

I have Centos 5.4, Apache/2.2.3, PHP 5.3.1 with all the recent updates.

Thank you!
0
Comment
Question by:djcybex
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 5
  • 3
  • +1
15 Comments
 
LVL 25

Expert Comment

by:madunix
ID: 26273090
at the moment no _100%_ defense against slowloris,  i think using iptables could protect you from  multiple connections coming from one specific IP address

madunix
0
 
LVL 1

Author Comment

by:djcybex
ID: 26273133
mod_antiloris already does that. But i have distributed attack from many IPs, so it doesn't help me.
0
 
LVL 25

Expert Comment

by:madunix
ID: 26273407
0
Free learning courses: Active Directory Deep Dive

Get a firm grasp on your IT environment when you learn Active Directory best practices with Veeam! Watch all, or choose any amount, of this three-part webinar series to improve your skills. From the basics to virtualization and backup, we got you covered.

 
LVL 1

Author Comment

by:djcybex
ID: 26273428
As i wrote in my original question i have tried the mod_qos but it gets me glibc errors and is halting or stopping apache.
I have posted a problem to mod_qos author's forum, but noone has answered yet.
http://sourceforge.net/projects/mod-qos/forums/forum/697421/topic/3508707/index/page/1
0
 
LVL 1

Author Comment

by:djcybex
ID: 26273437
And to add to mod_qos... my friend has two more servers running latest Red Hat and gets same glibc errors. So the problem is not only at my server.
0
 
LVL 25

Expert Comment

by:madunix
ID: 26273454
sorry i overlooked it, i ll check the above link


madunix
0
 
LVL 3

Expert Comment

by:tbrent77
ID: 26277948
Here is a link you might want to look at and see if you can impliment some configuration changes to your server: http://hackaday.com/2009/06/17/slowloris-http-denial-of-service/  Limiting requests per IP seems to be the answer for now. You could turn it off later.  Or move to a windows server.
0
 
LVL 25

Expert Comment

by:madunix
ID: 26278357
0
 
LVL 1

Author Comment

by:djcybex
ID: 26278424
fail2ban only check logs for brute force password attempts and bans IP if there are to many attempts.
It has nothing to do with apache and slowloris. Or did i miss something?
0
 
LVL 25

Expert Comment

by:madunix
ID: 26278466
0
 
LVL 1

Author Comment

by:djcybex
ID: 26278499
But that is only for authentications... has nothing to do with slowloris attack.
0
 
LVL 25

Assisted Solution

by:madunix
madunix earned 400 total points
ID: 26278963
Let me start again ..what does the website do? are the DOS attacks known attacks or unknown or new DOS attacks? what are you seeing, what did you diagnose until now?..etc.

The point is: if you got some kind of setup which is able to detect it's an attack, it could disable the ip addresses for let's say a day...another things which is done, is restricting the amount of traffic from a single ip address in general, but that requires a careful balance between what's feasible and what's not (thus an attack). an that's hard really hard and if some customers have proxies, they could somewhat show the same pattern as an dos-attack.

As I said its hard.... I think it's wise to look at dedicated appliances such as cisco fortinet and juniper which are made to detect and defend from exactly this...they may seem expensive at first, but pending time, and missing turnover because of an attack is also expensive...because: if you firewall your machines from outside, traffic will only enter from a few ports. these ports are the ones you want open, and to respond. so there is no benefit  you can achieve almost anything on the network layer the firewall can do. it's more effective to make a single rule on the firewall for all, instead of doing per machine.

Again: this can be done with iptables (I've got that on my screen right now), but is better to be implemented on the firewall, so you can activate it for all hosts with little effort, here's something on server fault: I experienced such attack ...  I put my apache behind http://varnish.projects.linpro.no/, which not only protected from slowloris, but also accelerated web requests quite a bit. also, iptables helped me iptables -I INPUT -p tcp --dport 80 -m connlimit --connlimit-above 20 --connlimit-mask 40 -j DROP     .... It just limits one host to 20 connections to 80 port, which should not affect non-malicious user, but would render slowloris unusable from one host.

Please mind: what you need to do, is build a global connection profile,get the, say, top two or three top connections, and the DOS attack profile...based upon that profile you should be able to make the iptables rule...if that works, see if you can do the same on your firewall.because if you defend one site, an attacker could go to another public site....

Well, slowloris is not made to distributed, so you could defend to some extend with a firewall rule...but have a look at cisco's DOS solutions. problably overly expensive, but doesn't require to study very new DOS attack but because the slowloris attack does a regular http call, it's hard to defend. it only does it much slower.....

Is this of some use to you?

madunix
0
 
LVL 5

Accepted Solution

by:
bplant earned 1600 total points
ID: 26328881
Hi djcybex,

 I've done some research and testing of the slowloris issue in the past and while it's impossible to stop a large distributed attack, I found there are a few things that can be done to try and defend against it.

Firstly, as already has been mentioned, you should add an iptables rule that uses connlimit to limit the number of parallel connections per IP address. I found that because of ISP proxies I had to set the limit as high as 250 which does make it less effective. I would recommend that you add some logging rules to get an idea of what would be suitable for your setup. Rules like these can be used to get an idea of how often hosts hit different limits:

-N CONNLIMIT
-F CONNLIMIT
-A CONNLIMIT -m connlimit --connlimit-above 100 --connlimit-mask 32 -m limit --limit 1/s -j LOG --log-prefix "connlimit is at 100"
-A CONNLIMIT -m connlimit --connlimit-above 100 --connlimit-mask 32 -j DROP
-A CONNLIMIT -m connlimit --connlimit-above 50 --connlimit-mask 32 -m limit --limit 1/s -j LOG --log-prefix "connlimit is at 50"
-A CONNLIMIT -m connlimit --connlimit-above 50 --connlimit-mast 32 -j RETURN

-A INPUT -p tcp --syn -m multiport --dport http,https -m state --state NEW -j CONNLIMIT

You can put as many logging rules in as you like to get a feel for what connlimit setting would be appropriate for your system.

Assuming you are using the worker MPM, you can increase the MaxClients setting. I have mine set at 2000. How high you can go is limited by how much memory you have available, but each apache thread uses far less memory than each apache process. I increased threadsPerChild to 50 so that I needed fewer apache processes to reach my MaxClients. You will also need to increase your MaxServers setting.

Be aware though that having a very high MaxClients does make you vulnerable to memory exhaustion if the attackers use POST instead of GET since the POST data is buffered in memory when using PHP.

Thirdly, decrease the apache Timeout setting. This will require the attackers to send data for each connection more often which will require more bandwidth on their behalf. I think the apache docs say not to go below 60 seconds, but I've done some *limited* testing with it as low as 10. Obviously the lower the more effective against the attack, but if you go too low you risk legitimate connections getting reset connections if they are on a very poor connection. This is probably more likely with wireless broadband users as they are more likely to get lost packets which must timeout before being retransmitted.

Saving possibly the best until last, the fourth thing you can do is use the iptables recent feature. The recent module allows you to count the number of new connections made per IP in a certain time period and then drop new connections based on this.

By default, the recent module only remembers 100 IP's (ip_list_tot) and the timestamps of the last 20 packets (ip_pkt_list_tot) so you need to increase these. Depending on how many concurrent IP's you have accessing your system under attack, you might need to increase this to several thousand. I've had it as high as 65536 in the past and this consumed a few hundred MB or more (from memory) with ip_pkt_list_tot at 255. I would set ip_pkt_list_tot to 255 as it gives you the most flexibility by remembering the timestamps of the last 255 new connections. Obviously the higher you go with all these numbers, the more memory you need, so maybe start off small with say 4096 and 255 for ip_list_tot and ip_pkt_list_tot  respectively. These settings are specified when you load the module into the kernel.

Next, the iptables rules. Something like this should work:

-N RECENT
-A RECENT
-A RECENT -m recent --set
-A RECENT -m recent --rcheck --seconds 1 --hitcount 50 -m limit --limit 1/s -j LOG --log-prefix "recent is at 50/1s"
-A RECENT -m recent --rcheck --seconds 1 --hitcount 50 -j DROP
-A RECENT -m recent --rcheck --seconds 1 --hitcount 25 -m limit --limit 1/s -j LOG --log-prefix "recent is at 25/1s"

-A RECENT -m recent --rcheck --seconds 5 --hitcount 100 -m limit --limit 1/s -j LOG --log-prefix "recent is at 100/5s"
-A RECENT -m recent --rcheck --seconds 5 --hitcount 100 -j DROP
-A RECENT -m recent --rcheck --seconds 5 --hitcount 75 -m limit --limit 1/s -j LOG --log-prefix "recent is at 75/5s"

-A INPUT -p tcp --syn -m state --state NEW -j RECENT

As before, you'll want to play around with the numbers to find something that suitable for your system. You may want to go much higher with the --seconds parameter to give you a bigger time window. Just remember that the --hitcount parameter cannot be greater than ip_pkt_list_tot.

I think using the recent module as I've described will help you greatly as your attackers will most likely be trying to open lots of new connections in quick succession which will result in them getting caught by the recent module.

Please let us know how you go if you decide to implement any of the 4 suggestions that I've made. I'd really be interested to know which ones you found to be successful against the attacks as I've only tested it when I am attacking myself, not when other people are trying to attack me.

Good luck,

Brad
0
 
LVL 5

Expert Comment

by:bplant
ID: 26328894
Oh, one more thing, putting a reverse proxy such as varnish in front of your web server will most likely solve your problems provided that the attackers are using GET. If they are using POST, then I'm pretty sure the reverse proxy will just forward the connection and data onto your web server therefore not helping you at all.
0
 
LVL 5

Expert Comment

by:bplant
ID: 26366214
Hi djcybex,

Just curious if you've been able to defend off the slowloris attacks using any of the above suggestions? I'm interested in your results.

Cheers,

Brad
0

Featured Post

New benefit for Premium Members - Upgrade now!

Ready to get started with anonymous questions today? It's easy! Learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

If you are a web developer, you would be aware of the <iframe> tag in HTML. The <iframe> stands for inline frame and is used to embed another document within the current HTML document. The embedded document could be even another website.
Introduction This article is intended for those who are new to PHP error handling (https://www.experts-exchange.com/articles/11769/And-by-the-way-I-am-New-to-PHP.html).  It addresses one of the most common problems that plague beginning PHP develop…
This tutorial will teach you the special effect of super speed similar to the fictional character Wally West aka "The Flash" After Shake : http://www.videocopilot.net/presets/after_shake/ All lightning effects with instructions : http://www.mediaf…
In this video, Percona Solution Engineer Rick Golba discuss how (and why) you implement high availability in a database environment. To discuss how Percona Consulting can help with your design and architecture needs for your database and infrastr…
Suggested Courses
Course of the Month9 days, 9 hours left to enroll

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question