Link to home
Start Free TrialLog in
Avatar of Duncan Roe
Duncan RoeFlag for Australia

asked on

Odd IPTABLES behaviour

I forgot to allow bootps requests from bootpc but it worked anyway. The packet is logged as about to be dropped (that is the next rule), but the bootpc server responds so must have got it.
This is contrary to my understanding of what should happen (and makes it harder for me to sell management on iptables as a firewall solution).
Does anyone have any idea what could be going on?
#!/bin/sh
set -x
exec >/tmp/iptables-filter-mintec.out 2>&1
 
# Set up a firewall: drop all incoming UDP & connects except DNS UDP:-
 
#Don't set up these rules twice
/usr/sbin/iptables -L -n|grep ppptab >/dev/null ||
{
 
  # A chain to log & drop a packet, except don't log FIN pkts
  /usr/sbin/iptables -N logdrop
  /usr/sbin/iptables -A logdrop -p tcp -m tcp --tcp-flags FIN FIN -j DROP
  # Comment out logging if too much stuff gets logged
  /usr/sbin/iptables -A logdrop -j LOG --log-level debug
  /usr/sbin/iptables -A logdrop -j DROP
 
  # A chain to inspect incoming (to this box) packets from ppp connection
  /usr/sbin/iptables -N ppptab
  # Allow icmp but not too many
  /usr/sbin/iptables -A ppptab -p icmp -m limit --limit 5/second -j ACCEPT
  # Allow DNS replies and queries
  /usr/sbin/iptables -A ppptab -p udp --source-port 53 -j ACCEPT
  /usr/sbin/iptables -A ppptab -p udp --destination-port 53 -j ACCEPT
  # Allow reply packets to an outbound telnet
  /usr/sbin/iptables -A ppptab -p tcp --source-port 23 -j ACCEPT
  # Drop everything else
  /usr/sbin/iptables -A ppptab -j logdrop
 
  # Firewall rule - check incoming (to this box) packets from ppp connection
  /usr/sbin/iptables -A INPUT -i ppp0 -j ppptab
  /usr/sbin/iptables -A INPUT -i eth1 -j ppptab
}

Open in new window

SOLUTION
Avatar of WizRd-Linux
WizRd-Linux
Flag of Australia image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Duncan Roe

ASKER

Other packets *are* dropped, like attempts to connect to the FTP server. Actually the above is modelled on a firewall I have successfully run at home for years. Still, your idea has merit in he absence of anything better - I'll give it a go
Rearranging to only have 2 tables shortened the script by 3 lines, a good result.
However it doesn't fix the problem. In the log file below, you can see the rejected bootpc->bootps UDP but you can also see dhcp actioning it
15:47:19# cat iptables-filter-mintec
#!/bin/sh
set -x
exec >/tmp/iptables-filter-mintec.out 2>&1
 
# Set up a firewall: drop all incoming UDP & connects except DNS UDP:-
 
#Don't set up these rules twice
/usr/sbin/iptables -L -n|grep ppptab >/dev/null ||
{
 
  # A chain to inspect incoming (to this box) packets from ppp connection
  /usr/sbin/iptables -N ppptab
  # Allow icmp but not too many
  /usr/sbin/iptables -A ppptab -p icmp -m limit --limit 5/second -j ACCEPT
  # Allow DNS replies and queries
  /usr/sbin/iptables -A ppptab -p udp --source-port 53 -j ACCEPT
  /usr/sbin/iptables -A ppptab -p udp --destination-port 53 -j ACCEPT
  # Allow reply packets to an outbound telnet
  /usr/sbin/iptables -A ppptab -p tcp --source-port 23 -j ACCEPT
 
  # Firewall rule - check incoming (to this box) packets from ppp connection
  /usr/sbin/iptables -A INPUT -i ppp0 -j ppptab
  /usr/sbin/iptables -A INPUT -i eth1 -j ppptab
  /usr/sbin/iptables -A INPUT -i eth0 -j ACCEPT
  # Drop everything not accepted by ppptab
  /usr/sbin/iptables -A INPUT -p tcp -m tcp --tcp-flags FIN FIN -j DROP
  # Comment out logging if too much stuff gets logged
  /usr/sbin/iptables -A INPUT -j LOG --log-level debug
  /usr/sbin/iptables -A INPUT -j DROP
}
 
=======================================================================================
 
15:47:44# iptables -L -v --line-numbers
Chain INPUT (policy ACCEPT 14697 packets, 2652K bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1       41  6831 ppptab     all  --  ppp0   any     anywhere             anywhere            
2      348 53664 ppptab     all  --  eth1   any     anywhere             anywhere            
3    10008 1694K ACCEPT     all  --  eth0   any     anywhere             anywhere            
4       24  2087 ACCEPT     all  --  lo     any     anywhere             anywhere            
5        0     0 DROP       tcp  --  any    any     anywhere             anywhere            tcp flags:FIN/FIN 
6      263 47495 LOG        all  --  any    any     anywhere             anywhere            LOG level debug 
7      263 47495 DROP       all  --  any    any     anywhere             anywhere            
 
Chain FORWARD (policy ACCEPT 146 packets, 14805 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
 
Chain OUTPUT (policy ACCEPT 2412 packets, 332K bytes)
num   pkts bytes target     prot opt in     out     source               destination         
 
Chain ppptab (2 references)
num   pkts bytes target     prot opt in     out     source               destination         
1       15   900 ACCEPT     icmp --  any    any     anywhere             anywhere            limit: avg 5/sec burst 5 
2       41  6831 ACCEPT     udp  --  any    any     anywhere             anywhere            udp spt:domain 
3       70  4458 ACCEPT     udp  --  any    any     anywhere             anywhere            udp dpt:domain 
4        0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere            tcp spt:telnet 
 
=================================================================================================
 
Apr 22 15:07:10 uno kernel: IN=eth1 OUT= MAC=00:d0:c9:a6:47:17:00:c0:3a:6b:10:6c:08:00 SRC=192.168.0.249 DST=192.168.0.1 LEN=161 TOS=0x00 PREC=0x00 TTL=1 ID=30888 PROTO=UDP SPT=22124 DPT=1900 LEN=141 
Apr 22 15:07:17 uno named[1473]: client 192.168.0.1#34461: updating zone 'loco.net/IN': deleting an RR
Apr 22 15:07:17 uno kernel: IN=eth1 OUT= MAC=00:d0:c9:a6:47:17:00:c0:3a:6b:10:6c:08:00 SRC=192.168.0.249 DST=192.168.0.1 LEN=328 TOS=0x00 PREC=0x00 TTL=128 ID=30894 PROTO=UDP SPT=68 DPT=67 LEN=308 
Apr 22 15:07:17 uno dhcpd: if LocoNetServer.loco.net IN TXT "31833bf41bc38609c2d9ff0dd96ce25498" rrset exists and LocoNetServer.loco.net IN A 192.168.0.249 rrset exists delete LocoNetServer.loco.net IN A 192.168.0.249: success.
Apr 22 15:07:17 uno named[1473]: zone loco.net/IN: sending notifies (serial 105)
Apr 22 15:07:17 uno named[1473]: client 192.168.0.1#41651: updating zone 'loco.net/IN': deleting an RR
Apr 22 15:07:18 uno dhcpd: if LocoNetServer.loco.net IN A rrset doesn't exist delete LocoNetServer.loco.net IN TXT "31833bf41bc38609c2d9ff0dd96ce25498": success.
Apr 22 15:07:18 uno named[1473]: client 192.168.0.1#44873: updating zone '0.168.192.in-addr.arpa/IN': deleting rrset at '249.0.168.192.in-addr.arpa' PTR
Apr 22 15:07:18 uno dhcpd: removed reverse map on 249.0.168.192.in-addr.arpa.
Apr 22 15:07:18 uno named[1473]: zone 0.168.192.in-addr.arpa/IN: sending notifies (serial 74)
Apr 22 15:07:18 uno dhcpd: DHCPRELEASE of 192.168.0.249 from 00:c0:3a:6b:10:6c (LocoNetServer) via eth1 (found)
Apr 22 15:07:22 uno named[1473]: zone loco.net/IN: sending notifies (serial 106)
Apr 22 15:09:08 uno kernel: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:00:c0:3a:6b:10:6c:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=332 TOS=0x00 PREC=0x00 TTL=128 ID=30926 PROTO=UDP SPT=68 DPT=67 LEN=312 
Apr 22 15:09:08 uno dhcpd: DHCPDISCOVER from 00:c0:3a:6b:10:6c via eth1
Apr 22 15:09:09 uno dhcpd: DHCPOFFER on 192.168.0.249 to 00:c0:3a:6b:10:6c (LocoNetServer) via eth1
Apr 22 15:09:09 uno kernel: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:00:c0:3a:6b:10:6c:08:00 SRC=0.0.0.0 DST=255.255.255.255 LEN=355 TOS=0x00 PREC=0x00 TTL=128 ID=30927 PROTO=UDP SPT=68 DPT=67 LEN=335 
Apr 22 15:09:09 uno named[1473]: client 192.168.0.1#40859: updating zone 'loco.net/IN': adding an RR at 'LocoNetServer.loco.net' A
Apr 22 15:09:09 uno named[1473]: client 192.168.0.1#40859: updating zone 'loco.net/IN': adding an RR at 'LocoNetServer.loco.net' TXT
Apr 22 15:09:09 uno dhcpd: Added new forward map from LocoNetServer.loco.net to 192.168.0.249
Apr 22 15:09:09 uno named[1473]: zone loco.net/IN: sending notifies (serial 107)
Apr 22 15:09:09 uno named[1473]: client 192.168.0.1#34461: updating zone '0.168.192.in-addr.arpa/IN': deleting rrset at '249.0.168.192.in-addr.arpa' PTR
Apr 22 15:09:09 uno named[1473]: client 192.168.0.1#34461: updating zone '0.168.192.in-addr.arpa/IN': adding an RR at '249.0.168.192.in-addr.arpa' PTR
Apr 22 15:09:09 uno named[1473]: zone 0.168.192.in-addr.arpa/IN: sending notifies (serial 75)
Apr 22 15:09:09 uno dhcpd: added reverse map from 249.0.168.192.in-addr.arpa. to LocoNetServer.loco.net
Apr 22 15:09:09 uno dhcpd: DHCPREQUEST for 192.168.0.249 (192.168.0.1) from 00:c0:3a:6b:10:6c (LocoNetServer) via eth1
Apr 22 15:09:09 uno dhcpd: DHCPACK on 192.168.0.249 to 00:c0:3a:6b:10:6c (LocoNetServer) via eth1
Apr 22 15:09:12 uno kernel: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:00:c0:3a:6b:10:6c:08:00 SRC=192.168.0.249 DST=192.168.0.255 LEN=229 TOS=0x00 PREC=0x00 TTL=128 ID=30960 PROTO=UDP SPT=138 DPT=138 LEN=209 

Open in new window

I added the lo ACCEPT rule by hand in the above, because dynamic dhcp updates stopped working. This is the current script:
15:58:54# cat iptables-filter-mintec
#!/bin/sh
set -x
exec >/tmp/iptables-filter-mintec.out 2>&1
 
# Set up a firewall: drop all incoming UDP & connects except DNS UDP:-
 
#Don't set up these rules twice
/usr/sbin/iptables -L -n|grep ppptab >/dev/null ||
{
 
  # A chain to inspect incoming (to this box) packets from ppp connection
  /usr/sbin/iptables -N ppptab
  # Allow icmp but not too many
  /usr/sbin/iptables -A ppptab -p icmp -m limit --limit 5/second -j ACCEPT
  # Allow DNS replies and queries
  /usr/sbin/iptables -A ppptab -p udp --source-port 53 -j ACCEPT
  /usr/sbin/iptables -A ppptab -p udp --destination-port 53 -j ACCEPT
  # Allow reply packets to an outbound telnet
  /usr/sbin/iptables -A ppptab -p tcp --source-port 23 -j ACCEPT
 
  # Firewall rule - check incoming (to this box) packets from ppp connection
  /usr/sbin/iptables -A INPUT -i ppp0 -j ppptab
  /usr/sbin/iptables -A INPUT -i eth1 -j ppptab
  /usr/sbin/iptables -A INPUT -i eth0 -j ACCEPT
  /usr/sbin/iptables -A INPUT -i lo -j ACCEPT
  # Drop everything not accepted by ppptab
  /usr/sbin/iptables -A INPUT -p tcp -m tcp --tcp-flags FIN FIN -j DROP
  # Comment out logging if too much stuff gets logged
  /usr/sbin/iptables -A INPUT -j LOG --log-level debug
  /usr/sbin/iptables -A INPUT -j DROP
}

Open in new window

SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Hi Blaz, thanks for replying. The mangle table is empty. I can post the output from iptables -t nat -L -nvx --line-numbers when I'm at work tomorrow.
Here is my understanding of the log. The "other " machine is a Windows system, hence ipconfig and options:

1. In a cmd window on 192.168.0.249, I type "ipconfig /release"
2. A bunch of log entries appear at  15:07:17 not in the order in which the events they document actually happened:
2a. iptables logs that it is going to drop this packet:
Apr 22 15:07:10 uno kernel: IN=eth1 OUT= MAC=00:d0:c9:a6:47:17:00:c0:3a:6b:10:6c:08:00 SRC=192.168.0.249 DST=192.168.0.1 LEN=161 TOS=0x00 PREC=0x00 TTL=1 ID=30888 PROTO=UDP SPT=22124 DPT=1900 LEN=141
2b. dhcpd sees and actions this packet. Did dhcpd see the packet before iptables did? Maybe it did and that is the explanation. Certainly tcpdump can see all packets that are dropped by iptables. Perhaps the message below is only issued after some kind of acknowledgement
Apr 22 15:07:17 uno dhcpd: if LocoNetServer.loco.net IN TXT "31833bf41bc38609c2d9ff0dd96ce25498" rrset exists and LocoNetServer.loco.net IN A 192.168.0.249 rrset exists delete LocoNetServer.loco.net IN A 192.168.0.249: success.
2c. dhcpd instructs DNS to delete its entry for 192.168.0.249 and DNS does so
Apr 22 15:07:17 uno named[1473]: client 192.168.0.1#34461: updating zone 'loco.net/IN': deleting an RR
Apr 22 15:07:17 uno named[1473]: zone loco.net/IN: sending notifies (serial 105)
Apr 22 15:07:17 uno named[1473]: client 192.168.0.1#41651: updating zone 'loco.net/IN': deleting an RR

Actually, the more I look at this, the more the original order of the log appears logical, *if* it is the case that dhcpcd sees packets before netfilter does:

 
Apr 22 15:07:17 uno named[1473]: client 192.168.0.1#34461: updating zone 'loco.net/IN': deleting an RR
*** I typed "ipconfig/release on system 192.168.0.249. Dhcpd has seen this packet *before* iptables and has instructed named (bind) to delete the entry for 192.168.0.249. Named has done the first step of this.
Apr 22 15:07:17 uno kernel: IN=eth1 OUT= MAC=00:d0:c9:a6:47:17:00:c0:3a:6b:10:6c:08:00 SRC=192.168.0.249 DST=192.168.0.1 LEN=328 TOS=0x00 PREC=0x00 TTL=128
ID=30894 PROTO=UDP SPT=68 DPT=67 LEN=308
*** netfilter (iptables) sees the packet and drops it. Too late
Apr 22 15:07:17 uno dhcpd: if LocoNetServer.loco.net IN TXT "31833bf41bc38609c2d9ff0dd96ce25498" rrset exists and LocoNetServer.loco.net IN A 192.168.0.249 rrset exists
delete LocoNetServer.loco.net IN A 192.168.0.249: success.
*** dhcpd has some kind of acknowledge from bind that bind will delete 192.168.0.249
Apr 22 15:07:17 uno named[1473]: zone loco.net/IN: sending notifies (serial 105)
*** bind continues deleting
Apr 22 15:07:17 uno named[1473]: client 192.168.0.1#41651: updating zone 'loco.net/IN': deleting an RR
*** bind continues deleting
Apr 22 15:07:18 uno dhcpd: if LocoNetServer.loco.net IN A rrset doesn't exist delete LocoNetServer.loco.net IN TXT "31833bf41bc38609c2d9ff0dd96ce25498": success.
*** dhcpd gets another ack from bind
Apr 22 15:07:18 uno named[1473]: client 192.168.0.1#44873: updating zone '0.168.192.in-addr.arpa/IN': deleting rrset at '249.0.168.192.in-addr.arpa' PTR
*** bind deletes the reverse lookup entry
Apr 22 15:07:18 uno dhcpd: removed reverse map on 249.0.168.192.in-addr.arpa.
*** dhcpcd gets a message from bind that reverse entry is gone
Apr 22 15:07:18 uno named[1473]: zone 0.168.192.in-addr.arpa/IN: sending notifies (serial 74)
*** bind sends notifies of reverse lookup zone change to any other processes that would be interested
Apr 22 15:07:18 uno dhcpd: DHCPRELEASE of 192.168.0.249 from 00:c0:3a:6b:10:6c (LocoNetServer) via eth1 (found)
*** dhcpd finishes un-registering 192.168.0.249
Apr 22 15:07:22 uno named[1473]: zone loco.net/IN: sending notifies (serial 106)
*** bind sends notifies of loco.net zone change to any other processes that would be interested

All in all, I guess that's it. Now, does anyone know for a fact that dhcpcd listens in such a way that it will see requests before netfilter?
I meant dhcpd, not dhcpcd
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
"I believe there is no way for a program to see a packet before netfilter does."
tcpdump can see them, so there is at least 1 way.
Yes there is a dhcp server on this system, and an dns that dhcp can configure (dynamic dns).Yes the first log shows a client releasing its IP, followed shortly after by acquiring one again.
This interaction did not take place when I didn't have the rule "/usr/sbin/iptables -A INPUT -i eth0 -j ACCEPT", so DROP was working then.
To clarify the network topography: there is only 1 other system on eth1's LAN (loco.net) (at least for now) and it is connected by crossover cable.
I will try your suggestion.
I think a crack is unlikely - one can never rule it out completely  of course - we are behind an outsourced firewall to a mainly Windows network.  I built uno's kernel fairly recently on a VMWare guest in that network. The guest runs an older kernel. I'll diff the files on the build system against uno anyway.

Next post, as promised earlier the nat table

10:13:10# iptables -t nat -L -nvx --line-numbers
Chain PREROUTING (policy ACCEPT 10662 packets, 1555071 bytes)
num      pkts      bytes target     prot opt in     out     source               destination         
1           0        0 DNAT       tcp  --  ppp0   *       0.0.0.0/0            0.0.0.0/0           tcp dpt:3389 to:192.168.0.249 
2           0        0 DNAT       tcp  --  ppp0   *       0.0.0.0/0            0.0.0.0/0           tcp dpt:23 to:192.168.0.249 
3           0        0 DNAT       tcp  --  ppp0   *       0.0.0.0/0            0.0.0.0/0           tcp dpt:21 to:192.168.0.249 
 
Chain POSTROUTING (policy ACCEPT 74 packets, 8129 bytes)
num      pkts      bytes target     prot opt in     out     source               destination         
1         368    35278 SNAT       all  --  *      ppp0    0.0.0.0/0            0.0.0.0/0           to:10.254.10.37 
 
Chain OUTPUT (policy ACCEPT 256 packets, 24103 bytes)
num      pkts      bytes target     prot opt in     out     source               destination         

Open in new window

ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Guys,

Thanks for your input.

I have to say I think neither of you came close, so I am going to accept my own comment http:#24211084 as the solution.

Feel free to post an objection if you think that is unfair.

Cheers ... Duncan.
Have you given up or have you figured out the cause?  If the cause are you able to share it?
I already shared it with you - at the foot of http:#24211084.
tcpdump is able to see incoming packets before iptables(netfilter) drops them. See the lsof of tcpdump below. Tcpdump only has 1 socket, and lsof can't identify its protocol. Dhcpd has a similar socket. Therefore dhcpd could be doing the same as tcpdump, thereby preempting netfilter. The man page doesn't say anything about that, and I haven't trawled through dhcpd source to see what it does with that socket (and the sock_raw socket). Given that DROP works for everything else I have tested, I was looking for a mechanism to explain why dhcp kept working, and I have found one. The existence of these unusual sockets is good enough for me as an explanation.


20:49:22# lsof -p $(pgrep tcpdump)
COMMAND  PID USER   FD   TYPE DEVICE    SIZE     NODE NAME
tcpdump 2258 dunc  cwd    DIR    8,2   12288 16220163 /home/dunc
tcpdump 2258 dunc  rtd    DIR    8,3    4096        2 /
tcpdump 2258 dunc  txt    REG    8,3  539096  2048532 /usr/sbin/tcpdump
tcpdump 2258 dunc  mem    REG    8,3   45552  1132313 /lib/libnss_files-2.5.so.incoming
tcpdump 2258 dunc  mem    REG    8,3 1528742  1132304 /lib/libc-2.5.so.incoming
tcpdump 2258 dunc  mem    REG    8,3  131484  1132301 /lib/ld-2.5.so.incoming
tcpdump 2258 dunc    0u   CHR  136,1                3 /dev/pts/1
tcpdump 2258 dunc    1u   CHR  136,1                3 /dev/pts/1
tcpdump 2258 dunc    2u   CHR  136,1                3 /dev/pts/1
tcpdump 2258 dunc    3u  sock    0,4             4492 can't identify protocol

Open in new window