We help IT Professionals succeed at work.

Number of firewall rules cause slowness?  Firewall rules review to remove dormant rules

sunhux
sunhux used Ask the Experts™
on
Our network team raised concern that with the number of IP addresses to block
(currently we create a group & add in IP addresses) coming from cyber regulator,
it may hog/slow down our CheckPoint 12600.

Q1:
What's the rule of thumb for max number of rules for 12600?  We estimated the
number of IP to block to reach 5000 per year.

Q2:
There's currently 1 rule ie "Deny Threat_intel_list All  for all ports/protocol":
So with only 1 rule but adding IP to the "Threat_intel_list" cause slowness or
by adding the # of IP to the list will increase the latency (make it slower) as well?

Q3: *** this question is crucial ***
We plan to break down that single rule to multiple rules ie 1 rule for 1 IP from
Threat Intel to block so that we can assess if there are hits & subsequently
assess whether to remove rules that don't have hits after, say 6 months so
as to reduce the # of rules & load to the firewall: is this a good practice in
terms of making the firewall faster?  In terms of cybersecurity, was advised
by one vendor this is a safe practice as IOCs that are 'dormant' ought to be
removed, just like AV vendors removed signatures for viruses that have not
been seen in the wild for quite a while to reduce the size/length of the AV
signature file/DB.

Q4:
To do firewall rules review (ie remove rules,  permit or block rules: permit as
it means the endpoint device may have been decomm'ed & block if the IOC
is not active), reckon we review if there are hits.  Heard Tuffin can't help if
we group IP addresses into a group, so what are the practices out there?
Break down to 1 IP to 1 rule or there are better tools?

Q5:
How do we verify the firewall is beginning to cause slowness due to its number
of rules:  can we do "ping across one zone to another zone" to measure?
Eg: compare it against a firewall that has minimal rules vs one with 90000 rules?

Q6:
Last but not least, can someone recommend a firewall that can take in 150,000
rules without performance impact & offer API where outsourced vendor could
add rules to block without having access to the firewall?


I've heard of one site's Cisco ASA that support more than 1 million rules but don't
know if it affects network performance.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
Fractional CTO
Distinguished Expert 2018
Commented:
Q1:
What's the rule of thumb for max number of rules for 12600?  We estimated the
number of IP to block to reach 5000 per year.

Generally this won't be a problem. I'd open a ticket with company manufacturing your device for a 100% correct answer.

Tip: Using ipset + iptables on a Linux machine is far easier to manage + since ipset uses hashed lookups for IP, all lookups are the same for every lookup, whether you have one rule or millions of rules.

Q2:
There's currently 1 rule ie "Deny Threat_intel_list All  for all ports/protocol":
So with only 1 rule but adding IP to the "Threat_intel_list" cause slowness or
by adding the # of IP to the list will increase the latency (make it slower) as well?

No difference whether you have a setup of IPs which fire another firewall rule or inline the addresses directly into the rule.

Q3: *** this question is crucial ***
We plan to break down that single rule to multiple rules ie 1 rule for 1 IP from
Threat Intel to block so that we can assess if there are hits & subsequently
assess whether to remove rules that don't have hits after, say 6 months so
as to reduce the # of rules & load to the firewall: is this a good practice in
terms of making the firewall faster?  In terms of cybersecurity, was advised
by one vendor this is a safe practice as IOCs that are 'dormant' ought to be
removed, just like AV vendors removed signatures for viruses that have not
been seen in the wild for quite a while to reduce the size/length of the AV
signature file/DB.

This is more a management issue rather than a speed issue.

To me, it's always better to gather firewall rule logic into a single target, then have another target for all IPs.

This makes debugging much easier than one rule/IP.

Also, this is important, if you have millions of rule/IP occurrences, then have to update the actual rule logic, then doing a drop/add operation or an update operation across millions of rule/IP occurrences could potentially take a very long time.

If all your firewall logic is gathered into a single rule, where all IPs in other changes call this common rule, then you can fix rule problems or enhancements in < 1 second, for millions of IPs.

Q4:
To do firewall rules review (ie remove rules,  permit or block rules: permit as
it means the endpoint device may have been decomm'ed & block if the IOC
is not active), reckon we review if there are hits.  Heard Tuffin can't help if
we group IP addresses into a group, so what are the practices out there?
Break down to 1 IP to 1 rule or there are better tools?

No clue exactly what you're asking here that's different than above questions.

Q5:
How do we verify the firewall is beginning to cause slowness due to its number
of rules:  can we do "ping across one zone to another zone" to measure?
Eg: compare it against a firewall that has minimal rules vs one with 90000 rules?

You can't really do this on a hardware device.

With iptables you can test this + with iptables (when using ipset), there's no point because every firewall runs at the same speed, since all lookups are from hashed tables, rather than sequential table walks.

Q6:
Last but not least, can someone recommend a firewall that can take in 150,000
rules without performance impact & offer API where outsourced vendor could
add rules to block without having access to the firewall?

iptables + ipset, used on millions of machines/containers continuously for decades.

The API is the command line interface. If you prefer writing your own code to interface with the Kernel tables you can. Likely this is overkill as your API would simply mimic the existing command line interface.
Distinguished Expert 2017

Commented:
David did a good job covering the questions you posed, though the current, existing spec and utilization of your system is important to accurately answer them.
https://sc1.checkpoint.com/uc/pdf/datasheets/12600-appliance-datasheet.pdf

How much ram does your appliance have 6 or 12gb, besides the firewall feature, does it also employ anti-virus, IDS/IDP, VPN   and other options that would impact demand on CPU/memory that could explain a slow down.

The suggestion to check with the vendor could help analyze your setup and provide you a definitive answer based on your existing configuration and use.
DarinTCHSenior CyberSecurity Engineer
Commented:
So let’s start with some basic Firewall and security tenants

If you are worried about having MAX firewalls Rules for your device then your focus is misaligned
Less rules will facilitate the traffic be evaluated and leaving(drop or else) sooner
Also we normally have narrow rules at the top of the process
That being said .... the sooner the majority of your traffic is evaluated the quicker your FW will be
Most companies can get by with 500-1000 rules.....I will stand by this and can easily prove it

Now regarding IP addresses .... 5000 addresses should have near 0 impact
IP address rules are almost like ACLs and Firewall filters...or can be
In other words it’s easy to drop IP address based traffic early and with minimal impact
(If your IP address filter is part of IDP / UTM / content filter — you are doing it wrong)

I currently block 300,000,000 addresses from China alone and that rule triggers many times per minute with near 0 impact

So a properly built rule blocking IP address can trigger @ L3/4 without intensive inspection

Currently the fastest NGFW is PaloAlto....but there is a decent Checkpoint FW also....speedwise
But pure Networking it world be Juniper or Cisco......however they are significantly slower when L4-7 traffic inspection is enabled

See NSS labs for independent speed and evaluation analysis, tests, and reports

And although David’s answer for #5 is wrong ... not all FW are equal or deliver via serial processing.....

I do agree that you could break 1 rule into 5 and the see the hits and volume per rule
...most high end FW have the ability to self evaluate rule usage....hits and traffic statistics

Let us know if you need more clarification post explanation
David FavorFractional CTO
Distinguished Expert 2018

Commented:
Side Note: I ended up in a situation like this some years ago, where I... inherited a system with several 100,000s of iptables firewall rules... which was insane. Also all firewall rules were atomic (for lack of a better term)... there was no concept of a common CHAIN where a specific port/protocol landed, so one change could effect 100% of some firewall logic.

It took hours for firewall rules to load at boot up, as the system had massive resource usage also.

Changing over to ipset changed this to a boot time of a few seconds to install all firewall rules.

Suggestion: Target using hashed firewall rule matches. If you avoid all sequential table walks, you'll simply avoid all firewall performance considerations. Better to arrange to avoid performance consideration, rather than coming up with some work around.

Note: This also drives to a very important consideration. Many firewall appliances provide no hashed IP/port/protocol lookup mechanism. These devices can only be used for very simple applications. For massive numbers of rules or IP/port/protocol handling (like running Fail2Ban on machines/containers under continual + heavy attack), you'll likely have to replace your firewall appliances with using iptables + ipset at the machine/container level... to maintain any level of firewall management sanity.

Author

Commented:
>IDS/IDP, VPN ..
Yes, our Checkpoint 12600 has 12GB & also act as Network IDS/IDP & VPN

Can DarinTCH share which firewall brand was used to block the 300m IP from China?
I wish I could create rules that blocks by entire subnets by countries but I can't as we
have branches & staff travelling to China & various parts of Europe.


>If your IP address filter is part of IDP / UTM / content filter — you are doing it wrong
Don't quite understand the above line;  are you recommending that we should not
get a UTM but rather separate appliances for firewall, IDP/IDS rather than an all-in-one
appliance?  Thought Palo Alto only carries UTM ie they don't have separate products
for FW, IDS/IDP & VPN server