Link to home
Start Free TrialLog in
Avatar of travisryan
travisryanFlag for United States of America

asked on

Subnet advertisement issues, Cisco 2821, 1841

I have two virtual servers that sit in my DMZ subnet. Pinging back and forth between these servers and other servers/machines at my main site (Site A) just fine. Between these servers and two my other sites (Site B and Site C) however, I cannot ping. Other devices that physically sit in the DMZ can connect to machines at these remote sites.

I thought it was an issue with the ACL so I pulled it and applied an "icmp any any" ACL but this didn't change my ping results. Upon checking the learned routes for my two remote sites I found that neither site had my DMZ subnet. Here's where the confusion comes in at, I have the subnet advertised to both sites. Can someone help with me this?

Sanitized configs will be attached.

=====

Facts:
Site A Firewall - Cisco ASA 5520
Site A Router - Cisco 2821
Site A to Site B connection: Fiber running EIGRP
Site B Router - Cisco 1841
Site A to Site C connection: MPLS line running BGP
Site C Router - Cisco 1841
=====

Testing:
Ping back and forth with ACL attached to DMZ interface, pings fail.
Pull ACL off of DMZ interface, pings fail.
Put "icmp any any" ACL on DMZ, pings fail.
Packet tracer ping test comes back successfully, but I never trust packet tracer results.
SiteA-FW-912-SANI.txt
SiteA-RTR-912-SANI.txt
SiteB-RTR-912-SANI.txt
SiteC-RTR-912-SANI.txt
Avatar of mikebernhardt
mikebernhardt
Flag of United States of America image

You have not provided a topology drawing or IPs of the, but most likely the default route you have on your remote routers is what's making the routing work. You are saying that other machines in the same DMZ subnet can ping successfully, so that would  mesh with your packet trace showing echo replies coming back, assuming that the packet trace was done inside the DMZ.

I suspect the problem is with your virtual machines, not the routing or the firewall.
Avatar of travisryan

ASKER

What would I be looking for on those virtual machines?  These is my company's first test with virtual servers so I'm not too experienced in this area.
Also, I was wrong about other devices in the dmz being able to connect to remote sites. After trying to ping another device in the dmz, I cannot get to it.
It would help a lot if you'd draw a picture of your network, with IP addresses that match the above configs.

Also provide the output of "show ip route" and "show ip bgp" from all 3 routers.
A co-worker pointed out the "passive-interface dmz" line in the EIGRP section of my Site A FW config. I removed this, pings worked from Site A to Site C for a limited amount of time, then quit. Back to square one.
So I've added a static route into my Site A router (which I shouldn't have to do because the route to my DMZ shows up on the Site A firewall just fine. For some reason the route is being communicated between the router and firewall.) and now the route for the DMZ shows up in Site B and Site C routers as well, goodie.

But, when I ping back to the DMZ from Site B and Site C the pings fail. Trace routing traces back to the Site A router, but no further. This also doesn't make any sense to me because the Site A router knows the route back to the DMZ, so why isn't traffic being pushed back over the routers connection to the DMZ?

In addition, adding that default route has interupted communication the devices in the DMZ have with the local subnet.

The mystery deepens. Any help is appreciated.
As I requested before:
It would help a lot if you'd draw a picture of your network, with IP addresses that match the above configs.

Also provide the output of "show ip route" and "show ip bgp" from all 3 routers.
Make sure that if you've changed your IP addresses in the configs that they match the routing table output. You certainly don't need to change private addressing at all. Just mask the public IPs by changing one octet.
Here are the "sh ip route" and "sh ip bgp"/"sh ip eigrp neighbor" from my routers:


SiteA#sh ip rou
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 11.254.1.1 to network 0.0.0.0

     207.250.33.0/28 is subnetted, 4 subnets
D EX    207.250.33.16 [170/3072] via 11.254.1.1, 7w0d, GigabitEthernet0/0
D EX    207.250.33.0 [170/3072] via 11.254.1.1, 7w0d, GigabitEthernet0/0
D EX    207.250.33.144 [170/3072] via 11.254.1.1, 7w0d, GigabitEthernet0/0
D EX    207.250.33.128 [170/3072] via 11.254.1.1, 7w0d, GigabitEthernet0/0
     199.37.161.0/30 is subnetted, 4 subnets
B       199.37.161.64 [20/0] via 13.116.127.81, 1w4d
B       199.37.161.40 [20/0] via 13.116.127.81, 1w4d
B       199.37.161.48 [20/0] via 13.116.127.81, 1w4d
B       199.37.161.56 [20/0] via 13.116.127.81, 1w4d
     173.226.0.0/26 is subnetted, 1 subnets
D EX    173.226.50.128 [170/3072] via 11.254.1.1, 7w0d, GigabitEthernet0/0
     11.1.0.0/8 is variably subnetted, 13 subnets, 2 masks
D       11.2.2.0/24 [90/30720] via 11.253.0.2, 3d16h, FastEthernet0/3/0
C       11.2.1.0/24 is directly connected, GigabitEthernet0/1
C       11.1.1.0/24 is directly connected, GigabitEthernet0/1
B       11.8.0.0/24 [20/0] via 13.116.127.81, 7w0d
S       11.2.40.0/24 is directly connected, GigabitEthernet0/1
S       11.2.70.0/24 is directly connected, GigabitEthernet0/1
S       11.101.0.0/24 [1/0] via 11.1.1.129
S       11.101.1.0/24 [1/0] via 11.1.1.129
S       11.2.100.0/24 is directly connected, GigabitEthernet0/1
C       11.254.1.0/30 is directly connected, GigabitEthernet0/0
C       11.253.0.0/30 is directly connected, FastEthernet0/3/0
B       11.254.3.0/30 [20/0] via 13.116.127.81, 7w0d
D       11.254.2.0/30 [90/30720] via 11.253.0.2, 3d16h, FastEthernet0/3/0
     12.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C       13.116.127.80/30 is directly connected, Multilink1
B       12.38.168.0/24 [20/0] via 13.116.127.81, 7w0d
B       13.116.127.172/30 [20/0] via 13.116.127.81, 4w6d
     73.0.0.0/26 is subnetted, 1 subnets
D EX    73.44.242.64 [170/3072] via 11.254.1.1, 7w0d, GigabitEthernet0/0
     135.89.0.0/16 is variably subnetted, 4 subnets, 2 masks
B       135.89.152.56/29 [20/0] via 13.116.127.81, 7w0d
B       135.89.152.128/28 [20/0] via 13.116.127.81, 7w0d
B       135.89.154.152/29 [20/0] via 13.116.127.81, 7w0d
B       135.89.157.160/28 [20/0] via 13.116.127.81, 7w0d
S    193.169.222.0/24 [1/0] via 11.1.1.70
D*EX 0.0.0.0/0 [170/3072] via 11.254.1.1, 7w0d, GigabitEthernet0/0
==---==
SiteB#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 11.254.2.1 to network 0.0.0.0

     207.250.33.0/28 is subnetted, 4 subnets
D EX    207.250.33.16 [170/28672] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    207.250.33.0 [170/28672] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    207.250.33.144 [170/28672] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    207.250.33.128 [170/28672] via 11.253.0.1, 3d17h, FastEthernet0/1/0
     199.37.161.0/30 is subnetted, 4 subnets
D EX    199.37.161.64 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    199.37.161.40 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    199.37.161.48 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    199.37.161.56 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
     173.226.0.0/26 is subnetted, 1 subnets
D EX    173.226.50.128 [170/28672] via 11.253.0.1, 3d17h, FastEthernet0/1/0
     11.1.0.0/8 is variably subnetted, 10 subnets, 2 masks
C       11.2.2.0/24 is directly connected, FastEthernet0/1
D EX    11.2.1.0/24 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    11.1.1.0/24 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    11.8.0.0/24 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    11.2.40.0/24 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    11.2.70.0/24 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D       11.254.1.0/30 [90/28416] via 11.253.0.1, 3d17h, FastEthernet0/1/0
C       11.253.0.0/30 is directly connected, FastEthernet0/1/0
D EX    11.254.3.0/30 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
C       11.254.2.0/30 is directly connected, FastEthernet0/0
     12.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
D EX    12.38.168.0/24 [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    13.116.127.172/30
           [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
     73.0.0.0/26 is subnetted, 1 subnets
D EX    73.44.242.64 [170/28672] via 11.253.0.1, 3d17h, FastEthernet0/1/0
     135.89.0.0/16 is variably subnetted, 4 subnets, 2 masks
D EX    135.89.152.56/29
           [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    135.89.152.128/28
           [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    135.89.154.152/29
           [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
D EX    135.89.157.160/28
           [170/1709312] via 11.253.0.1, 3d17h, FastEthernet0/1/0
S*   0.0.0.0/0 [1/0] via 11.254.2.1
==--==
SiteC#sh ip rou
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 11.254.3.1 to network 0.0.0.0

     207.250.33.0/28 is subnetted, 4 subnets
B       207.250.33.16 [20/0] via 13.116.127.173, 7w0d
B       207.250.33.0 [20/0] via 13.116.127.173, 7w0d
B       207.250.33.144 [20/0] via 13.116.127.173, 7w0d
B       207.250.33.128 [20/0] via 13.116.127.173, 7w0d
     199.37.161.0/30 is subnetted, 4 subnets
B       199.37.161.64 [20/0] via 13.116.127.173, 1w4d
B       199.37.161.40 [20/0] via 13.116.127.173, 1w4d
B       199.37.161.48 [20/0] via 13.116.127.173, 1w4d
B       199.37.161.56 [20/0] via 13.116.127.173, 1w4d
     173.226.0.0/26 is subnetted, 1 subnets
B       173.226.50.128 [20/0] via 13.116.127.173, 7w0d
     11.1.0.0/8 is variably subnetted, 13 subnets, 2 masks
B       11.2.2.0/24 [20/0] via 13.116.127.173, 3d17h
B       11.2.1.0/24 [20/0] via 13.116.127.173, 7w0d
B       11.1.1.0/24 [20/0] via 13.116.127.173, 7w0d
C       11.8.0.0/24 is directly connected, FastEthernet0/1
B       11.2.40.0/24 [20/0] via 13.116.127.173, 7w0d
B       11.2.70.0/24 [20/0] via 13.116.127.173, 7w0d
B       11.101.0.0/24 [20/0] via 13.116.127.173, 7w0d
B       11.101.1.0/24 [20/0] via 13.116.127.173, 7w0d
B       11.2.100.0/24 [20/0] via 13.116.127.173, 7w0d
B       11.254.1.0/30 [20/0] via 13.116.127.173, 7w0d
B       11.253.0.0/30 [20/0] via 13.116.127.173, 7w0d
C       11.254.3.0/30 is directly connected, FastEthernet0/0
B       11.254.2.0/30 [20/0] via 13.116.127.173, 3d17h
     12.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
B       13.116.127.80/30 [20/0] via 13.116.127.173, 7w0d
B       12.38.168.0/24 [20/0] via 13.116.127.173, 7w0d
C       13.116.127.172/30 is directly connected, Serial0/0/0
     73.0.0.0/26 is subnetted, 1 subnets
B       73.44.242.64 [20/0] via 13.116.127.173, 7w0d
     135.89.0.0/16 is variably subnetted, 4 subnets, 2 masks
B       135.89.152.56/29 [20/0] via 13.116.127.173, 7w0d
B       135.89.152.128/28 [20/0] via 13.116.127.173, 7w0d
B       135.89.154.152/29 [20/0] via 13.116.127.173, 7w0d
B       135.89.157.160/28 [20/0] via 13.116.127.173, 7w0d
B    193.169.222.0/24 [20/0] via 13.116.127.173, 7w0d
S*   0.0.0.0/0 [1/0] via 11.254.3.1
==--==--==
SiteC#sh bgp
BGP table version is 792, local router ID is 13.116.127.174
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 11.1.1.0/24      13.116.127.173                         0 7018 7018 i
*> 11.2.1.0/24      13.116.127.173                         0 7018 7018 i
*> 11.2.2.0/24      13.116.127.173                         0 7018 7018 ?
*> 11.2.40.0/24     13.116.127.173                         0 7018 7018 i
*> 11.2.70.0/24     13.116.127.173                         0 7018 7018 i
*> 11.2.100.0/24    13.116.127.173                         0 7018 7018 ?
*> 11.8.0.0/24      0.0.0.0                  0         32768 i
*> 11.101.0.0/24    13.116.127.173                         0 7018 7018 ?
*> 11.101.1.0/24    13.116.127.173                         0 7018 7018 ?
*> 11.253.0.0/30    13.116.127.173                         0 7018 7018 ?
*> 11.254.1.0/30    13.116.127.173                         0 7018 7018 i
*> 11.254.2.0/30    13.116.127.173                         0 7018 7018 ?
*> 11.254.3.0/30    0.0.0.0                  0         32768 i
*> 12.38.168.0/24   13.116.127.173                         0 7018 2386 i
*> 13.116.127.80/30 13.116.127.173                         0 7018 ?
r> 13.116.127.172/30
                    13.116.127.173           0             0 7018 ?
*> 73.44.242.64/26  13.116.127.173                         0 7018 7018 ?
*> 135.89.152.56/29 13.116.127.173                         0 7018 2386 i
*> 135.89.152.128/28
                    13.116.127.173                         0 7018 2386 i
*> 135.89.154.152/29
                    13.116.127.173                         0 7018 2386 i
*> 135.89.157.160/28
                    13.116.127.173                         0 7018 2386 i
*> 173.226.50.128/26
                    13.116.127.173                         0 7018 7018 ?
*> 193.169.222.0    13.116.127.173                         0 7018 7018 ?
*> 199.37.161.40/30 13.116.127.173                         0 7018 i
*> 199.37.161.48/30 13.116.127.173                         0 7018 i
*> 199.37.161.56/30 13.116.127.173                         0 7018 i
*> 199.37.161.64/30 13.116.127.173                         0 7018 i
*> 207.250.33.0/28  13.116.127.173                         0 7018 7018 ?
*> 207.250.33.16/28 13.116.127.173                         0 7018 7018 ?
*> 207.250.33.128/28
                    13.116.127.173                         0 7018 7018 ?
*> 207.250.33.144/28
                    13.116.127.173                         0 7018 7018 ?
SiteC#
==--==--==
SiteA#sh bgp
BGP table version is 425, local router ID is 13.116.127.82
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 11.1.1.0/24      0.0.0.0                  0         32768 i
*> 11.2.1.0/24      0.0.0.0                  0         32768 i
*> 11.2.2.0/24      11.253.0.2              20         32768 ?
*> 11.2.40.0/24     0.0.0.0                  0         32768 i
*> 11.2.70.0/24     0.0.0.0                  0         32768 i
*> 11.2.100.0/24    0.0.0.0                  0         32768 ?
*> 11.8.0.0/24      13.116.127.81                          0 7018 7018 i
*> 11.101.0.0/24    11.1.1.129               0         32768 ?
*> 11.101.1.0/24    11.1.1.129               0         32768 ?
*> 11.253.0.0/30    0.0.0.0                  0         32768 ?
*> 11.254.1.0/30    0.0.0.0                  0         32768 i
*> 11.254.2.0/30    11.253.0.2              20         32768 ?
*> 11.254.3.0/30    13.116.127.81                          0 7018 7018 i
*> 12.38.168.0/24   13.116.127.81                          0 7018 2386 i
r> 13.116.127.80/30 13.116.127.81            0             0 7018 ?
*> 13.116.127.172/30
                    13.116.127.81                          0 7018 ?
*> 73.44.242.64/26  11.254.1.1              20         32768 ?
*> 135.89.152.56/29 13.116.127.81                          0 7018 2386 i
*> 135.89.152.128/28
                    13.116.127.81                          0 7018 2386 i
*> 135.89.154.152/29
                    13.116.127.81                          0 7018 2386 i
*> 135.89.157.160/28
                    13.116.127.81                          0 7018 2386 i
*> 173.226.50.128/26
                    11.254.1.1              20         32768 ?
*> 193.169.222.0    11.1.1.70                0         32768 ?
*> 199.37.161.40/30 13.116.127.81                          0 7018 i
*> 199.37.161.48/30 13.116.127.81                          0 7018 i
*> 199.37.161.56/30 13.116.127.81                          0 7018 i
*> 199.37.161.64/30 13.116.127.81                          0 7018 i
*> 207.250.33.0/28  11.254.1.1              20         32768 ?
*> 207.250.33.16/28 11.254.1.1              20         32768 ?
*> 207.250.33.128/28
                    11.254.1.1              20         32768 ?
*> 207.250.33.144/28
                    11.254.1.1              20         32768 ?
SiteA#
==--==--==
SiteA#sh ip eigrp neigh
IP-EIGRP neighbors for process 101
H   Address                 Interface       Hold Uptime   SRTT   RTO  Q  Seq
                                            (sec)         (ms)       Cnt Num
1   11.253.0.2              Fa0/3/0           12 3d17h       4   200  0  17
0   11.254.1.1              Gi0/0             10 8w4d        1   200  0  116
SiteA#
==--==--==
SiteB#sh ip eigrp neigh
IP-EIGRP neighbors for process 101
H   Address                 Interface       Hold Uptime   SRTT   RTO  Q  Seq
                                            (sec)         (ms)       Cnt Num
1   11.253.0.1              Fa0/1/0           14 3d17h       6   200  0  542
0   11.254.2.1              Fa0/0             13 3d17h      35   210  0  11
SiteB#
The picture will be forthcoming.
OK, a few comments (I finally got a chance to really look at this):
1. Passive interface should make no difference as to whether the route is advertised in EIGRP. All that does it prevent the firewall from sending EIGRP hellos into that subnet, which you probably don't want. However, the firewall's EIGRP config doesn't have a network statement for the DMZ subnet. That's probably why the route doesn't show up on your Site A router. You do have that statement on your router, but it's useless there.

2. If you are advertising the DMZ subnet route in BGP, you have to have a matching IGP route. This should be coming from EIGRP but for reasons described above it isn't. The static route resolved this. But you created a static DEFAULTt route pointing into your firewall? Remove it and do #1. In the future if you need a static route to point inside, make it specific, not default. For example
ip route 173.17.1.0 255.255.255.0 11.254.1.1

3. I'm confused because although it seems like the router is on the outside of your firewall  and connects to the internet (for example, the router is configured with BGP and there is no rule in the firewall permitting it), the firewall seems to be configured as if it could be on the outside. For example, the connection to your router is on the inside trusted interface. This is where the drawing will be very helpful.
Diagram attached.
DMZ-Network-Layout.JPG
Mike, in response to your questions:

1. I've added a static route for the DMZ into the Site A router pointing back to the Site A firewall. I also added a network advertisement into the EIGRP section. After checking the results of "sh IP route" command on Site B and Site C the route was in there. In the end pings still didn't work and now the DMZ machines didn't work on the local network.
2.  I created a static default pointing out to my firewall because if machines don't find a matching IP address in my network it's assumed they're looking for something out on the internet.
3. All the routers who's configs I've posted sit inside of their firewalls as you can see from the diagram I finally posted.

Again, any help is appreciated. This one is a stumper to me.
In addition, as stated in a previous post, after making all of those changes I can traceroute back from Site B and Site C to the Site A router, but no further. This makes me think there's something blocking communication concerning the DMZ subnet between Site A router and Site A firewall.

Not sure what though.
SOLUTION
Avatar of mikebernhardt
mikebernhardt
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
ASKER CERTIFIED SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I've requested that this question be closed as follows:

Accepted answer: 0 points for travisryan's comment #a38438568
Assisted answer: 500 points for mikebernhardt's comment #a38437488

for the following reason:

Another member's comment caused me to check something on my setup.
That's exactly what I said in my earlier post:
https://www.experts-exchange.com/questions/27867444/Subnet-advertisement-issues-Cisco-2821-1841.html?anchorAnswerId=38434861#a38434861

I thought you had done it after my comment. that's why I said later, "You shouldn't need the static route pointing to the DMZ network now that EIGRP is working correctly (at least it should be). Take it out and test again."