Linux IP tables down

To begin with, your log tells me that your IPTables is receiving more requests than it can track. When enabled through the use of NAT or other stateful inspection rules, netfilter (iptables) under Linux maintains a list of connections passing through the router. Each connection tracking entry contains defined characteristics of the packet, including the source and destination IP address and port number.

The connection tracking entries are ultimately stored in a hash table with a fixed size. For routers with stateful inspection enabled, the number of connections to track may exceed the total number of connections available in the table. If the router reaches the maximum number of connection tracking entries, it will log an error:

"ip_conntrack: table full, dropping packet"

each time that it is unable to store an entry in the connection tracking table. Each instance of this message represents a connection that the router has discarded, typically meaning that the user whose connection was dropped must re-establish their connection.

The maximum size of the connection tracking table can be increased. The maximum size value is stored in the router's proc filesystem in the file /proc/sys/net/ipv4/ip_conntrack_max. Increasing the maximum size of the connection tracking table to a value larger than the total number fo connections will eliminate the error message and prevent the router from dropping connections due to a lack of space in the connection tracking table.

Each connection uses approximately 350 bytes of memory, so 16384 connections would allocate 5.7 MB of RAM.

Following are commands, which you can use to check current configuration, as well as to modify the same.

# This tell you how many sessions arte open right now.
cat /proc/net/ip_conntrack | wc -l
# This tells you the maximum number of conntrack entries you can have in
total
cat /proc/sys/net/ipv4/ip_conntrack_max

Once the previous number hits beyond the latter, you should start seeing
these messages. I would increase the latter number by calling:

echo "<some_bigger_number>" > /proc/sys/net/ipv4/ip_conntrack_max

or if you want it to span reboots, you can place the following in
/etc/sysctl.conf

sys.net.ipv4.ip_conntrack_max =3D <some_big_number>


Hope this helps!!

Regards,

Also, on going further down the log I discovered that you are using Linux version 2.6.18-194.el5xen, which is buggy.

I would suggest you to update the kernel to minimum of 2.6.18-238.

Please use yum upgrade kernel-xenp command to upgrade your kernel, considering you are currently using a xen kernel.

our website was under attacked, but iptables went down after that attack and when there were only a few incoming requests.

I wonder why the tables went down after the attack but during the attack.

If it was not too long after the attack, it is possible that state information was still being maintained in the Iptables tracking database.


Regards,

pritamdutt,

While I am asking you the question, new issues came out (please see the log in the code snippet). I would think they are related.

Please help advise Thanks.

INFO: task syslogd:1013 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
syslogd       D ffff880001005460     0  1013      1          1016  1005 (NOTLB)
 ffff8800b8b6bd88  0000000000000282  0000000000000000  0000000000000001 
 0000000000000009  ffff8800bde5b7a0  ffffffff804f4b80  00000000000015f1 
 ffff8800bde5b988  ffff8800bfb9b860 
Call Trace:
 [<ffffffff88036d5a>] :jbd:log_wait_commit+0xa3/0xf5
 [<ffffffff8029c60a>] autoremove_wake_function+0x0/0x2e
 [<ffffffff8803178a>] :jbd:journal_stop+0x1cf/0x1ff
 [<ffffffff8023138e>] __writeback_single_inode+0x1e9/0x328
 [<ffffffff802d2ff1>] do_readv_writev+0x26e/0x291
 [<ffffffff802e555b>] sync_inode+0x24/0x33
 [<ffffffff8804c36d>] :ext3:ext3_sync_file+0xc9/0xdc
 [<ffffffff80252276>] do_fsync+0x52/0xa4
 [<ffffffff802d37f5>] __do_fsync+0x23/0x36
 [<ffffffff802602f9>] tracesys+0xab/0xb6

Select allOpen in new window