blub061198
asked on
Proxy arp with cisco 1600 router
Hi!
I have a problem here that is driving me mad:
I'm trying to set up a firewall at a local ISP. The setup should be as
follows:
x.x.x.0 -> x.x.x.5 eth0 -> x x x.3 eth1 -> x.x.x.1Cisco 1600 ->
The linux box has x.x.x.3 on eth0 and x.x.x.3 on eth1.
The problem is that the proxy arping on the Linux firewall doesn't work with the Cisco.
I have had proxy arp working successfully with a win95 box instead of the cisco connected to eth1.
With the setup above I can ping both sides of the network from the firewall box, but routing doesn't work at all.
When I have the win95 machine connected instead of the Cisco I can ping
from everywhere anywhere without problems.
The Firewall is running Debian 2.1 with linux 2.2.14.
Here is the setup on the Firewall:
ifconfig:
eth0 Link encap:Ethernet HWaddr 00:10:4B:C3:E8:DA
inet addr:195.163.187.5 Bcast:195.163.187.127
Mask:255.255.255.128
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3431 errors:0 dropped:0 overruns:0 frame:0
TX packets:663 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
Interrupt:7 Base address:0x310
eth1 Link encap:Ethernet HWaddr 00:60:8C:B3:CD:52
inet addr:195.163.187.3 Bcast:195.163.187.127
Mask:255.255.255.128
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:181 errors:0 dropped:0 overruns:0 frame:0
TX packets:183 errors:0 dropped:0 overruns:0 carrier:3
collisions:0 txqueuelen:100
Interrupt:10 Base address:0x300
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:3924 Metric:1
RX packets:11 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
route -n:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
195.163.187.1 0.0.0.0 255.255.255.255 UH 0 0 0
eth1
195.163.187.0 0.0.0.0 255.255.255.128 U 0 0 0
eth0
0.0.0.0 195.163.187.1 0.0.0.0 UG 1 0 0
eth1
Proxy arp has been enabled by issuing:
echo "1" >/proc/sys/net/ipv4/ip_for
echo "1" >/proc/sys/net/ipv4/conf/a
I don't have the slightest idea where the problem could be.. I mean proxy arp works perfectly with the win95 box connected to eth1, but it
doesn't work with the cisco 1600 connected to eth1.. this reeeeeeeeaaaaaaly drives me mad.
Thanx for any help on this!
I have a problem here that is driving me mad:
I'm trying to set up a firewall at a local ISP. The setup should be as
follows:
x.x.x.0 -> x.x.x.5 eth0 -> x x x.3 eth1 -> x.x.x.1Cisco 1600 ->
The linux box has x.x.x.3 on eth0 and x.x.x.3 on eth1.
The problem is that the proxy arping on the Linux firewall doesn't work with the Cisco.
I have had proxy arp working successfully with a win95 box instead of the cisco connected to eth1.
With the setup above I can ping both sides of the network from the firewall box, but routing doesn't work at all.
When I have the win95 machine connected instead of the Cisco I can ping
from everywhere anywhere without problems.
The Firewall is running Debian 2.1 with linux 2.2.14.
Here is the setup on the Firewall:
ifconfig:
eth0 Link encap:Ethernet HWaddr 00:10:4B:C3:E8:DA
inet addr:195.163.187.5 Bcast:195.163.187.127
Mask:255.255.255.128
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3431 errors:0 dropped:0 overruns:0 frame:0
TX packets:663 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
Interrupt:7 Base address:0x310
eth1 Link encap:Ethernet HWaddr 00:60:8C:B3:CD:52
inet addr:195.163.187.3 Bcast:195.163.187.127
Mask:255.255.255.128
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:181 errors:0 dropped:0 overruns:0 frame:0
TX packets:183 errors:0 dropped:0 overruns:0 carrier:3
collisions:0 txqueuelen:100
Interrupt:10 Base address:0x300
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:3924 Metric:1
RX packets:11 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
route -n:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
195.163.187.1 0.0.0.0 255.255.255.255 UH 0 0 0
eth1
195.163.187.0 0.0.0.0 255.255.255.128 U 0 0 0
eth0
0.0.0.0 195.163.187.1 0.0.0.0 UG 1 0 0
eth1
Proxy arp has been enabled by issuing:
echo "1" >/proc/sys/net/ipv4/ip_for
echo "1" >/proc/sys/net/ipv4/conf/a
I don't have the slightest idea where the problem could be.. I mean proxy arp works perfectly with the win95 box connected to eth1, but it
doesn't work with the cisco 1600 connected to eth1.. this reeeeeeeeaaaaaaly drives me mad.
Thanx for any help on this!
As an aid for troubleshooting could you run several pings while tcpdumping.
tcpdump -i eth1 arp > logfile
Some with eth0, too.
Thx
Rob
tcpdump -i eth1 arp > logfile
Some with eth0, too.
Thx
Rob
ASKER
I don't think the cisco has static arp, and I also tried restarting the cisco which should flush the arp cache. as for the tcpdump suggestion: I did that, but the cisco never replied to anything not sent directly from the firewall box..
What does the arp table on the 1600 contain (router# sho arp)? What does the 1600's routing table look like (router# sho ip route)? And maybe even better how about letting me see the router's config (router sho conf).
ASKER
I'd really love to show you the routers config, but the sad thing is that the router is owned by the company that provides the internet services, which means that I have to e-mail them, ask them to get the config, and send it to me. I will do that, but this will probably take untill monday or tuesday, when those people get to work :( Could you possibly send me your e-mail address to hansd@saunalahti.fi, then I'll let you know when I have added the router configuration to this page.
You can't do this:
x.x.x.n->x.x.x.5->x.x.x.3-
etho eth 1600
with a Cisco router. You have to something like:
a.b.c.n->a.b.c.5->d.e.f.3-
etho eth 1600
where a.b.c.0 and d.e.f.0 are different networks. The 1600 routes networks and either the target is on a locally attached network (d.e.f.0) or it's reachable by forwarding to the next hop. You need a route statement in the cisco that says (assuming class C's):
ip route a.b.c.0 255.255.255.0 d.e.f.3
and of course you have to tell the firewall to route between a.b.c.0 and d.e.f.0 as well as tell it that the default gateway is d.e.f.1.
Of course with a 1600 available, it would make more sense to me (from a network view) to upgrade the 1600 to an IOS with the firewall feature set and dispense with the linux box entirely. This would be the simplest and most reliable configuration, providing that you don't need or intend to install any DMZ servers.
x.x.x.n->x.x.x.5->x.x.x.3-
etho eth 1600
with a Cisco router. You have to something like:
a.b.c.n->a.b.c.5->d.e.f.3-
etho eth 1600
where a.b.c.0 and d.e.f.0 are different networks. The 1600 routes networks and either the target is on a locally attached network (d.e.f.0) or it's reachable by forwarding to the next hop. You need a route statement in the cisco that says (assuming class C's):
ip route a.b.c.0 255.255.255.0 d.e.f.3
and of course you have to tell the firewall to route between a.b.c.0 and d.e.f.0 as well as tell it that the default gateway is d.e.f.1.
Of course with a 1600 available, it would make more sense to me (from a network view) to upgrade the 1600 to an IOS with the firewall feature set and dispense with the linux box entirely. This would be the simplest and most reliable configuration, providing that you don't need or intend to install any DMZ servers.
Well, we crossed comments. If you don't own the router and the ISP doesn't want to install the firewall feature set, then that option is out. Oh yeah, I'll get an email notification automatically if you or anyone else adds a comment.
The situation became a bit more clear from your comment. I'm guessing that you've only got a single netblock from the ISP (and they probably weren't overly generous) to work with. The ideal solution, since you don't own the router, would be to do real NAT on the firewall (one-to-one address translation) and use a private network inside of the firewall. This would be trivial to to with a Cisco Pix or a router w/firewall feature set. It's not at all clear to me that this can be done with the Linux tools. Masquerading as implemented on Linux looks to me to be only a many-to-one form of address translation.
Of course you could use Masquerading, but this has implications w/respect to publically accessible servers that you might want to have inside of the firewall.
The situation became a bit more clear from your comment. I'm guessing that you've only got a single netblock from the ISP (and they probably weren't overly generous) to work with. The ideal solution, since you don't own the router, would be to do real NAT on the firewall (one-to-one address translation) and use a private network inside of the firewall. This would be trivial to to with a Cisco Pix or a router w/firewall feature set. It's not at all clear to me that this can be done with the Linux tools. Masquerading as implemented on Linux looks to me to be only a many-to-one form of address translation.
Of course you could use Masquerading, but this has implications w/respect to publically accessible servers that you might want to have inside of the firewall.
ASKER
Hmm. I don't understand why I can't do:
x.x.x.n->x.x.x.5->x.x.x.3-
When I run proxy arp on the linux box which has x.x.x.5 and x.x.x.3. The cisco which then is x.x.x.1 is in fact connected to a.b.c.d as you can see in this traceroute which is run from the Linux box:
traceroute to ftp.funet.fi (193.166.0.148), 30 hops max, 40 byte packets
1 195.163.187.1 (195.163.187.1) 68.055 ms 122.937 ms 100.197 ms
2 194.213.91.189 (194.213.91.189) 108.983 ms 9.349 ms 8.693 ms
3 tni-hel-rn01-fe00.telenord
Do you have any idea what there could be in the cisco that makes it to not accept beeing proxy arped by the Linux box?
As I said I'll send the cisco config as soon as I get it, but I simply don't understand what the cisco doesn't like about the proxy arp setup. I mean proxy arp should be transparent to the cisco..
Upgrading the hardware is not an option (at least not for me, because I'm at the company only to do the linux stuff ;))
x.x.x.n->x.x.x.5->x.x.x.3-
When I run proxy arp on the linux box which has x.x.x.5 and x.x.x.3. The cisco which then is x.x.x.1 is in fact connected to a.b.c.d as you can see in this traceroute which is run from the Linux box:
traceroute to ftp.funet.fi (193.166.0.148), 30 hops max, 40 byte packets
1 195.163.187.1 (195.163.187.1) 68.055 ms 122.937 ms 100.197 ms
2 194.213.91.189 (194.213.91.189) 108.983 ms 9.349 ms 8.693 ms
3 tni-hel-rn01-fe00.telenord
Do you have any idea what there could be in the cisco that makes it to not accept beeing proxy arped by the Linux box?
As I said I'll send the cisco config as soon as I get it, but I simply don't understand what the cisco doesn't like about the proxy arp setup. I mean proxy arp should be transparent to the cisco..
Upgrading the hardware is not an option (at least not for me, because I'm at the company only to do the linux stuff ;))
Let me think about that. There's something in the back of my mind regarding this situation that I can't quite dredge up.
ASKER
I had posted the original question to comp.os.linux.networking, too, and now I got one reply which might help you in figuring out why the cisco won't do proxy arp:
A guy replied who has the exact same configuration, but he didn't remember what modell his cisco was. He experienced the exact same problem: He was able to ping both sides of the network from his firewall, but routing didn't work. After leaving everything in the proxy arp setup "for the time of a dinner" (don't ask me how long that would be.. I don't know) everything was working nicely.
So do you know if there is some kind of timeout in the cisco, or if the cisco might be switching some kind of modes after "the time of a dinner"?
A guy replied who has the exact same configuration, but he didn't remember what modell his cisco was. He experienced the exact same problem: He was able to ping both sides of the network from his firewall, but routing didn't work. After leaving everything in the proxy arp setup "for the time of a dinner" (don't ask me how long that would be.. I don't know) everything was working nicely.
So do you know if there is some kind of timeout in the cisco, or if the cisco might be switching some kind of modes after "the time of a dinner"?
There is a time-to-live for entries in the arp table of the routers, but that can easily be bypassed by rebooting the router.
I figured out what was waving for attention from the back of my head. If you read the RFC's that apply to proxy arp, you'll notice that they consistently refer to the reason for the protocol being there is to allow a gateway in a sub-netted network to inform a host on one side that it is the route to a host on the other side. There are several sanity checks built in to the protocol and I suspect that this config is tripping over one of them.
From the 1600's perspective, it has a destination IP from an inbound packet that it needs to deliver. It knows that the destination is within the locally attached network because it's eth0 was configured with an IP within that network and a netmask that established the size of the network. The destination IP is within that range, so the machine with that IP has to be directly on the wire hooked to eth0. So it arps the IP, and the linux box dutifully responds and says that its IP is the gateway. The router looks at the response and sees a contradiction. As far as the router is concerned all of the IP's for that network are supposed to be locally connected, yet here's something claiming to be a gateway to the locally attached network. I think the router sees this as a potential routing loop and discards the arp reply.
If the network inside of the 1600 were sub-netted or two different networks, this situation doesn't occur. The destination address wouldn't be local as far as the cisco is concerned as it's IP won't lie within the address space of the locally attached network. In this case, a proxy arp would be accepted and the router would forward the packet to the gateway for delivery.
I figured out what was waving for attention from the back of my head. If you read the RFC's that apply to proxy arp, you'll notice that they consistently refer to the reason for the protocol being there is to allow a gateway in a sub-netted network to inform a host on one side that it is the route to a host on the other side. There are several sanity checks built in to the protocol and I suspect that this config is tripping over one of them.
From the 1600's perspective, it has a destination IP from an inbound packet that it needs to deliver. It knows that the destination is within the locally attached network because it's eth0 was configured with an IP within that network and a netmask that established the size of the network. The destination IP is within that range, so the machine with that IP has to be directly on the wire hooked to eth0. So it arps the IP, and the linux box dutifully responds and says that its IP is the gateway. The router looks at the response and sees a contradiction. As far as the router is concerned all of the IP's for that network are supposed to be locally connected, yet here's something claiming to be a gateway to the locally attached network. I think the router sees this as a potential routing loop and discards the arp reply.
If the network inside of the 1600 were sub-netted or two different networks, this situation doesn't occur. The destination address wouldn't be local as far as the cisco is concerned as it's IP won't lie within the address space of the locally attached network. In this case, a proxy arp would be accepted and the router would forward the packet to the gateway for delivery.
ASKER
Ok. I totally understand your last reply, and it seems logical. I just have a last question before accepting your help as the asnwer: Do you know of some kind of configuration option to turn off this sanity check in the cisco? or do you know if there is some other configuration option I could change in the cisco to make the setup work? Wouldn't it be possible to enter a host route in the cisco the the linux box, and enter it as a gateway for the rest of the network? I mean something like this routeing table in the cisco:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
x.x.x.3 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
x.x.x.0 x.x.x.3 255.255.255.128 U 0 0 0 eth0
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
x.x.x.3 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
x.x.x.0 x.x.x.3 255.255.255.128 U 0 0 0 eth0
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Ok. I'll take a look at the ipfilter stuff. Thanks a lot for all your help! The whole thing became much clearer for me now!
Thanks for the compliment, and the grade. Add a comment to this question if you get stuck someplace and we can figure ot the solution (I'll get an email notice so I'll know you've added something)
ASKER
I don't know if you still read this, since it has been some time since the discussion, but I actully got everything to work now with the original setup..
So it actually does work the way I wanted to implement the proxy arp from the beginnig.
The problem was that I shouldn't have made a host route on the linux box, but make a small subnet like this:
x.x.x.0-x.x.x.5(eth0):x.x.
I just had to change the IP address from x.x.x.3 to x.x.x.2, make x.x.x.3 the broadcast address on eth1, and make the subnet mask for eth1 255.255.255.252. This is better described in the proxy-arp-subnetting howto..
Thanx anyways for the discussion, and the help.
So it actually does work the way I wanted to implement the proxy arp from the beginnig.
The problem was that I shouldn't have made a host route on the linux box, but make a small subnet like this:
x.x.x.0-x.x.x.5(eth0):x.x.
I just had to change the IP address from x.x.x.3 to x.x.x.2, make x.x.x.3 the broadcast address on eth1, and make the subnet mask for eth1 255.255.255.252. This is better described in the proxy-arp-subnetting howto..
Thanx anyways for the discussion, and the help.
Rob