Solved

TTL expired in transit switch issue

Posted on 2013-01-02
23
3,219 Views
Last Modified: 2016-11-23
Attached is a switching layout that is in production. I'm having an issue with ttl expired responses that terminate at 2 newly installed Dell 8024s. When I ping an offline device with a  valid ip address from my workstation I get ttl expired errors coming from these switches. The switch/router is the default gateway for the network so I'm curious to why the error is not originating from the router. The first response will come from the .18 switch and then the final 3 from the .17????
EE8024SwitchConfig.pdf
0
Comment
Question by:cs2009
  • 10
  • 6
  • 5
  • +1
23 Comments
 
LVL 50

Expert Comment

by:Don Johnston
ID: 38738695
What is the source IP address and destination IP address of the pings?
0
 

Author Comment

by:cs2009
ID: 38738703
both in same subnet:

Host 10.10.1.100
destination 10.10.12.33
SM 255.255.0.0
0
 
LVL 50

Expert Comment

by:Don Johnston
ID: 38738714
If what you're describing is correct then the behavior is not really possible. The TTL field is only decremented by a router (layer 3 device). If the source is 10.10.1.100/16 and the destination is 10.10.12.33/16, they hosts are on the same network and no router is needed. Which means the TTL isn't being decremented so there could be no TTL expired.

Unless the sending host is transmitting with a TTL of zero which isn't likely either.

I suspect there's more to your network than meets the eye. Can you provide configs of the network devices and indicate which ports the 10.10.1.100 and 10.10.12.33 hosts are connected to.
0
 

Author Comment

by:cs2009
ID: 38738750
exactly my thought... there is no hop on the same subnet. Both hosts are connected to ports on Vlan 10 on separate switches. I expected to get a request timed out but not so. All switches have routing on Vlan 10 enabled. The dell 8024 is a new, different animal to me compared to the HPs. The previous image shows the basic topology. 8024 startconfig attached.
startup-config
0
 
LVL 50

Expert Comment

by:Don Johnston
ID: 38738756
I have no idea why a switch (or router) would respond with a ICMP TTL exceeded message for a device on the same network as the sender.
0
 

Author Comment

by:cs2009
ID: 38738762
now you know my pain. I can't find anything......
0
 
LVL 50

Expert Comment

by:Don Johnston
ID: 38739798
What happens when you do a traceroute? (with the destination on and off)
0
 

Author Comment

by:cs2009
ID: 38740976
The device is now online. Tracert has the device going thru the .17 switch?


Tracing route to ricohmpc5502-boe.test.org [10.10.12.33]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  10.10.0.17
  2    <1 ms    <1 ms    <1 ms  ricohmpc5502-boe.test.org [10.10.12.33]
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38741228
I would double check that HP2910 and 10.10.12.33 both have the subnet mask 255.255.0.0.

Can you also do:

     ping -r 9 ricohmpc5502-boe.test.org

I've seen some weird stuff when not everything has the correct subnet mask
0
 

Author Comment

by:cs2009
ID: 38741268
both devices /16


C:\Users\samuels>ping -r 9 ricohmpc5502-boe.test.org

Pinging ricohmpc5502-boe.test.org [10.10.12.33] with 32 bytes of data:
Reply from 10.10.12.33: bytes=32 time<1ms TTL=255
    Route: 10.10.12.33
Reply from 10.10.12.33: bytes=32 time=24ms TTL=255
    Route: 10.10.0.17 ->
           10.10.12.33
Reply from 10.10.12.33: bytes=32 time=1ms TTL=255
    Route: 10.10.0.17 ->
           10.10.12.33
Reply from 10.10.12.33: bytes=32 time=116ms TTL=255
    Route: 10.10.0.17 ->
           10.10.12.33
0
 
LVL 50

Expert Comment

by:Don Johnston
ID: 38741305
Could you post the output of an ipconfig (or O/S similar command) from the source host.
0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 

Author Comment

by:cs2009
ID: 38741357
Ethernet adapter vEthernet (Virtual Switch):

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Hyper-V Virtual Ethernet Adapter #2
   Physical Address. . . . . . . . . : 18-03-73-4F-82-C9
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv4 Address. . . . . . . . . . . : 10.10.1.100(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . : 10.10.0.1
   DNS Servers . . . . . . . . . . . : 10.10.1.35
                                       10.10.1.36
   NetBIOS over Tcpip. . . . . . . . : Enabled
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38741359
Also from the target host.  To me it looks like one of them may have a subnet mask other than 255.255.0.0.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38741381
You may also want to look at all the routing tables in all of the devices.  There may be a device with a entry for 10.10.0.0/24 or 10.10.1.0/24.  Which could also cause weird results.
0
 
LVL 10

Accepted Solution

by:
mat1458 earned 350 total points
ID: 38748279
I must admit that everything looks a bit weird. Can it be that the Dell switches are kind of proxy-arp'ing the printer (they shouldn't but...)? Where is the printer attached? Can you maybe clear the arp cache (or any static ARP entry) in the two Dells and see if the problem persists?
0
 

Author Comment

by:cs2009
ID: 38750914
The printer is connected to a switch down stream. I cleared cache on both switches. After I ping host an entry is written back to the cache. As before the ping request will go to the .17 switch on the first attempt and then .18 for the final 3 attempts?
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38751256
Did you check the routing tables on everything?
0
 

Author Comment

by:cs2009
ID: 38751718
I did. I'm only routing one vlan at this point in time.
0
 
LVL 57

Assisted Solution

by:giltjr
giltjr earned 150 total points
ID: 38752233
When you look at the ARP table on the source PC what MAC address does it have for the target?

The only thing I can think of is what mat1458 already mentioned, the Dell switches are proxy arp'ing.  However I have no clue why all of sudden .18 would take over for .17.
0
 

Author Comment

by:cs2009
ID: 38752344
I found parameters on routing interface and proxy arp and local proxy arp are indeed checked. If I uncheck and ping host I now get ttl timeout on last resort gateway on the actual router, which is where I thought it should be.

On the dell 8024, the 2 arp settings are enabled by default. Should I just keep the defaults and go with lesson learned? Thanks guys.
0
 
LVL 57

Expert Comment

by:giltjr
ID: 38752512
Weird.  I personally would not use proxy arp.  Since so many network monitoring products use ping as one way to see if a device is up and to measure latency issues, I would leave it off.

Otherwise your monitoring software will show the device is up when in fact it may not be.
0
 

Author Comment

by:cs2009
ID: 38752525
thanks giltjr
0
 
LVL 10

Expert Comment

by:mat1458
ID: 38753661
To be clear: this seems to be a bug of Dell. The switches must never proxy ARP for a request in the same subnet. Proxy ARP is only useful across subnet boundaries.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

#Citrix #Citrix Netscaler #HTTP Compression #Load Balance
If your business is like most, chances are you still need to maintain a fax infrastructure for your staff. It’s hard to believe that a communication technology that was thriving in the mid-80s could still be an essential part of your team’s modern I…
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now