blocking IP of non US ip addresses
We are hosting our website on AWS using ubuntu and have notice a large number of attempts by "non" web friendly nations.
IPtables seems to be slow. Out ubuntu server has 1 cpu and 1 gig of ram. Will adding additional cpu and ram speed it up or is there a better solution ?
IPtables seems to be slow. Out ubuntu server has 1 cpu and 1 gig of ram. Will adding additional cpu and ram speed it up or is there a better solution ?
There is an RBL-type of DNS lookup that returns the country code of origin.
Look at http://countries.nerd.dk/
Essentially, you would PASS anything that is listed in us.countries.nerd.dk (or rather block any IP not in that list)
- NOTE: You would likely still do your other RBL checks
HOWEVER, I would caution that a lot of US companies use mail servers from outside the US. (For example, my hosting company is co-located in Germany and Canada).
Remember: this is fashioned like an RBL list -- so you use the IN-addr.arpa address format.
That is, to test if the address 71.2.3.4 is in the UK, you would query 4.3.2.71.uk.countries.nerd.dk (it is not -- the result is "not found")
To then test if the address 71.2.3.4 is in the US, you would query 4.3.2.71.us.countries.nerd.dk (it is -- the result is 127.0.0.2)
I hope this helps.
Dan
IT4SOHO
A single AWS CPU with 1 GB of RAM should be able to handle a reasonable amount of web traffic -- however, adding resources to such a server would undoubtedly improve performance.
Look at http://countries.nerd.dk/
Essentially, you would PASS anything that is listed in us.countries.nerd.dk (or rather block any IP not in that list)
- NOTE: You would likely still do your other RBL checks
HOWEVER, I would caution that a lot of US companies use mail servers from outside the US. (For example, my hosting company is co-located in Germany and Canada).
Remember: this is fashioned like an RBL list -- so you use the IN-addr.arpa address format.
That is, to test if the address 71.2.3.4 is in the UK, you would query 4.3.2.71.uk.countries.nerd.dk (it is not -- the result is "not found")
To then test if the address 71.2.3.4 is in the US, you would query 4.3.2.71.us.countries.nerd.dk (it is -- the result is 127.0.0.2)
I hope this helps.
Dan
IT4SOHO
A single AWS CPU with 1 GB of RAM should be able to handle a reasonable amount of web traffic -- however, adding resources to such a server would undoubtedly improve performance.
Oh, I forgot to mention (I make too many assumptions at times).
Since an RBL is intended for use with a MX server (that is, an email system), you'll need something to integrate it into iptables -- and that would be "packetbl" (see http://chiselapp.com/user/rkeene/repository/packetbl/artifact/abf8dcfcd1baaecb for a howto, and somewhere like http://freecode.com/projects/packetbl/ for the source code.
Again, I hope this helps.
Dan
IT4SOHO
Since an RBL is intended for use with a MX server (that is, an email system), you'll need something to integrate it into iptables -- and that would be "packetbl" (see http://chiselapp.com/user/rkeene/repository/packetbl/artifact/abf8dcfcd1baaecb for a howto, and somewhere like http://freecode.com/projects/packetbl/ for the source code.
Again, I hope this helps.
Dan
IT4SOHO
So here's some food for thought.
1. There are a lot of IP address ranges outside the US, and they are not all contiguous. In other words, you might have a range like 1.2.3.4 - 1.2.3.10 that belongs to a non-US country EXCEPT for 1.2.3.6. That means your firewall needs to block two ranges: 1.2.3.4 - 1.2.3.5 and 1.2.3.7 - 1.2.3.10 so it doesn't accidentally block a US IP address.
2. IP addresses can change owners all the time. You won't see a LOT of geographic ownership changes, so if an IP is a non-US IP, it'll usually stay that way, but there are still updates, and if you want to be exact about it, then you might need a database like Maxmind's GeoIP database (free), but you'll need a script to turn it into a set of iptables rules.
3. My own firewall does this country-level blocking already, although mine EXCLUDES countries that are common origins of attacks. There are always exceptions, but it does a decent job and definitely cuts down on spam, too. I've attached my country-level blocking rules, if you want to merge them into your firewall. Make sure your own IP isn't in there before you do it. :)
4. IPTables isn't really slow. It's about as fast as a firewall can possibly be.
5. Additional RAM and CPU will definitely improve performance for sure.
6. I would double-check to ensure your website itself and its code is optimized properly. You don't have a lot of RAM / CPU to start with, so even small deficiencies in the code will have a greater effect. You might want to profile some of your pages and see if they're slow (e.g. slow database connection or slow queries) and ensure there aren't any bottlenecks in the requests (e.g. Apache can be configured to allow more children than the system resources allow, which leads to children waiting around, taking up resources that otherwise would be used for a speedier response).
iptablescountryblock.txt
1. There are a lot of IP address ranges outside the US, and they are not all contiguous. In other words, you might have a range like 1.2.3.4 - 1.2.3.10 that belongs to a non-US country EXCEPT for 1.2.3.6. That means your firewall needs to block two ranges: 1.2.3.4 - 1.2.3.5 and 1.2.3.7 - 1.2.3.10 so it doesn't accidentally block a US IP address.
2. IP addresses can change owners all the time. You won't see a LOT of geographic ownership changes, so if an IP is a non-US IP, it'll usually stay that way, but there are still updates, and if you want to be exact about it, then you might need a database like Maxmind's GeoIP database (free), but you'll need a script to turn it into a set of iptables rules.
3. My own firewall does this country-level blocking already, although mine EXCLUDES countries that are common origins of attacks. There are always exceptions, but it does a decent job and definitely cuts down on spam, too. I've attached my country-level blocking rules, if you want to merge them into your firewall. Make sure your own IP isn't in there before you do it. :)
4. IPTables isn't really slow. It's about as fast as a firewall can possibly be.
5. Additional RAM and CPU will definitely improve performance for sure.
6. I would double-check to ensure your website itself and its code is optimized properly. You don't have a lot of RAM / CPU to start with, so even small deficiencies in the code will have a greater effect. You might want to profile some of your pages and see if they're slow (e.g. slow database connection or slow queries) and ensure there aren't any bottlenecks in the requests (e.g. Apache can be configured to allow more children than the system resources allow, which leads to children waiting around, taking up resources that otherwise would be used for a speedier response).
iptablescountryblock.txt
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I banged out a default "block all counties except U.S" years ago, based on IANA's allocations from here - http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.txt
If resources are already an issue, DNS resolving will only add to overhead and connection latency.
Blocking via IP is truly the best way.
It's as simple as finding which /8's you don't want to have access, and make the block to that whole network.
iptables -A INPUT -s 1.0.0.0/8 -j DROP
Wash, rinse, repeat. Can we it's own chain, or just grab it with an include on your current firewall.
If resources are already an issue, DNS resolving will only add to overhead and connection latency.
Blocking via IP is truly the best way.
It's as simple as finding which /8's you don't want to have access, and make the block to that whole network.
iptables -A INPUT -s 1.0.0.0/8 -j DROP
Wash, rinse, repeat. Can we it's own chain, or just grab it with an include on your current firewall.
No, it's not as simple as that! That is really bad advice. Blocking ranges with /8 is like trying to perform surgery with a machete. If you block things at the /8 level, you're going to block all sorts of legitimate ranges. For example:
Country = Starting IP - Ending IP
==========================
US = 4.18.68.0 - 4.28.141.255
CA = 4.28.142.0 - 4.28.142.255
US = 4.28.143.0 - 4.53.150.255
CA = 4.53.151.0 - 4.53.151.255
US = 4.53.152.0 - 4.69.153.192
NL = 4.69.153.193 - 4.69.153.193
US = 4.69.153.194 - 4.69.153.255
Allowing the 4.0.0.0/8 in its entirety would include many different IPs from non-US countries.
Similarly, blocking a common African range, like 197.0.0.0/8 would result in valid US IP address ranges being blocked, like 197.147.20.0/24.
DON'T block addresses at the /8 level. It's extremely inaccurate.
A good starting point is the free MaxMind GeoIP database located here:
http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country-CSV.zip
I run a nightly PHP script to download this file, process it, and import the data into a MySQL table that I can then query easily with my firewall-generation scripts. I've attached that script here.
GeoLiteDBImporter.php
Country = Starting IP - Ending IP
==========================
US = 4.18.68.0 - 4.28.141.255
CA = 4.28.142.0 - 4.28.142.255
US = 4.28.143.0 - 4.53.150.255
CA = 4.53.151.0 - 4.53.151.255
US = 4.53.152.0 - 4.69.153.192
NL = 4.69.153.193 - 4.69.153.193
US = 4.69.153.194 - 4.69.153.255
Allowing the 4.0.0.0/8 in its entirety would include many different IPs from non-US countries.
Similarly, blocking a common African range, like 197.0.0.0/8 would result in valid US IP address ranges being blocked, like 197.147.20.0/24.
DON'T block addresses at the /8 level. It's extremely inaccurate.
A good starting point is the free MaxMind GeoIP database located here:
http://geolite.maxmind.com/download/geoip/database/GeoLite2-Country-CSV.zip
I run a nightly PHP script to download this file, process it, and import the data into a MySQL table that I can then query easily with my firewall-generation scripts. I've attached that script here.
GeoLiteDBImporter.php
"really bad advice"....has worked very well for me for 10 years+ now.
Just sayin'.
As a rule of thumb, I block all APNIC, then go from there. Never had a problem, and we do have a customer support center to take such issues, the only issue I've ever had is, for instance, a developer in the Philippines needing their IP unblocked to place orders for a US based client.
You can UN-block specific subnets. First matching rule gets the golden candlestick. ;) With a little tweaking, I stand by this method 100%.
For triage in these minor issues, I provide the client with an un-blocked IPS to hit to gather their true "presenting as" IP, then unblock appropriately. Works extremely well for our needs (which is always subjective, granted).
It really gets down to what your web site services and target audience. I wouldn't tout ANY method as the "best way", as that is also subjective. Our service is snail-mail based to US and CA, and this method has served us very well, and saved us tons of headaches in hack attempts and the rampant APNIC bot nets.
It's certainly better than rDNS on every single connection. Just my .02.
Just sayin'.
As a rule of thumb, I block all APNIC, then go from there. Never had a problem, and we do have a customer support center to take such issues, the only issue I've ever had is, for instance, a developer in the Philippines needing their IP unblocked to place orders for a US based client.
You can UN-block specific subnets. First matching rule gets the golden candlestick. ;) With a little tweaking, I stand by this method 100%.
For triage in these minor issues, I provide the client with an un-blocked IPS to hit to gather their true "presenting as" IP, then unblock appropriately. Works extremely well for our needs (which is always subjective, granted).
It really gets down to what your web site services and target audience. I wouldn't tout ANY method as the "best way", as that is also subjective. Our service is snail-mail based to US and CA, and this method has served us very well, and saved us tons of headaches in hack attempts and the rampant APNIC bot nets.
It's certainly better than rDNS on every single connection. Just my .02.
@mugojava - Just because you haven't seen any problems doesn't mean they haven't existed. :)
From what you're saying, the only way that you would know about an issue/exception is if someone took the time to complain about it. There are a lot of people who will just abandon a site if it doesn't seem responsive and they'll assume the site is just down.
That also means that when you DO get a complaint, you have to manually tweak/adjust the firewall to allow the exception, which creates more work that you could have -mostly- avoided by just being more specific upfront - something that can easily be automated.
The iptables engine can easily process tens of thousands of rules upfront without any issues. There is no reason to over-block when you can just feed it automatically-generated rules that will stay up to date over time, and like you said, you can still introduce minor exceptions as necessary.
I completely agree that IP filtering is better than rDNS. Just saying that it's easy to be precise upfront.
From what you're saying, the only way that you would know about an issue/exception is if someone took the time to complain about it. There are a lot of people who will just abandon a site if it doesn't seem responsive and they'll assume the site is just down.
That also means that when you DO get a complaint, you have to manually tweak/adjust the firewall to allow the exception, which creates more work that you could have -mostly- avoided by just being more specific upfront - something that can easily be automated.
The iptables engine can easily process tens of thousands of rules upfront without any issues. There is no reason to over-block when you can just feed it automatically-generated rules that will stay up to date over time, and like you said, you can still introduce minor exceptions as necessary.
I completely agree that IP filtering is better than rDNS. Just saying that it's easy to be precise upfront.
Actually, these are heavily paying customers, so believe me, they would complain if they couldn't place an order. But, that decision is really up to OP. It may work for some, may not work for others.
Your method has value, but they make bazookas for a reason. ;) If you need a bazooka, use a bazooka. If you need a sniper rifle, use a sniper rifle.
Your method has value, but they make bazookas for a reason. ;) If you need a bazooka, use a bazooka. If you need a sniper rifle, use a sniper rifle.
Apache, PHP, and nginx all support GeoIP Lite based blocking
And, you're not be throttled by AWS based upon what you've purchased?