We help IT Professionals succeed at work.

Secure Website Inaccesible - TCP Retransmit (Wireshark)

107 Views
Last Modified: 2020-07-07
We have a website that needs to be access by staff.  We have checked our Meraki firewall logs and also the Cisco Umbrella content filters to make sure nothing is being blocked, but the site is still inaccessible.  I ran a Wireshark capture and just see conintuous TCP Retransmission entries.  I am not familiar enough with Wireshark to analyze the capture and also running out of ideas on how to get this to work.

Any suggestions?

Wireshark Screenshot
Comment
Watch Question

CERTIFIED EXPERT
Distinguished Expert 2018

Commented:
According to capture:
All packets are SYN packets - first step in 3 way handshake, and there is no return traffic from server (since next packet supposed to be SYN/ACK in opposite direction). So source host is trying to reach destination server again and again since there is no SYN/ACK response.

Possibilities:
a) Traffic is not reaching destination server for any reason (ACL dropping traffic, routing issue, traffic not natted or not properly natted, traffic blocked/dropped by firewall)
b) Traffic is reaching destination server, but return traffic is blocked/dropped (the same list as for case a) except NAT part)

I am not sure about Meraki (default configuration), but typically firewalls are dropping traffic silently and there will not be trace in log if traffic is dropped.

Author

Commented:
Thank You.    I Will look closer at the firewall as a starting point.
nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
There is something dropping the data silently.
If a regular router drops traffic an ICMP return would be sent.  (That can also be dropped on the firewall on the local side).

Author

Commented:
What's the best approach to track down silent drops?  Port 443 outbound is open for the destination IP.
nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
Traceroute can help, (tcptraceroute i mean). That might give a hint where the stuff gets dropped. (no ICMP's received anymore).
Try this from several ip addresses that might identify a choke point.

Author

Commented:
Tracert times out at the same point for clients that work outside of our network and also internally that does not connect.
nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
tracert.exe uses UDP, not tcp. TCPTRACE is a different beast it uses TCP packets to see where you get.
So if all have trouble at the same time then that is the FIRST suspect place to start looking where the problem is.

Also verify (from the server side... f.e. using wireshark, or tshark it's command line companion) if traffic is arriving there.

Author

Commented:
I ran tracetcp internally and externally.  When I run it externally and monitor wireshark, there is a SYN & SYN/ACK for TCP protocol and the trace is successful.  When I run the same command internally, Wireshark shows a Name Query using UDP 137 and the trace times out.  Something is acting totally different internally on our corporate network when I try to communicate to the destination IP than it does when offsite on a public network.
CERTIFIED EXPERT
Distinguished Expert 2018

Commented:
0) Is 199.116.188.12 located on the site or on other location (other then local site)? - asking if traffic needs to have u-turn (hairpin) configured
1) is traffic reaching firewall? (firewall present as a hop in traceroute, although firewalls might be configured not to be present as a hop in traceroute)
1.1) or traffic stops before firewall (if last hop on traceroute is before firewall, then most likely firewall is dropping traffic, or on hop after firewall (if there is nat is issue - IS will drop traffic))
2) if traffic is reaching firewall can you please check created sessions on firewall (not sure how to do it on Meraki)
3) if traffic is dropped when it reaches ISP - please contact your internet provider

From last traceoute hop check next hop (most specific route) and verify next hop for traffic. Go to next hop and verify further, repeat until you find issue.
Check how to verify sessions on Meraki (I never worked with it), and how to perform packet-tracer. I guess there is something similar on Meraki firewall too.
nociSoftware Engineer
CERTIFIED EXPERT
Distinguished Expert 2019

Commented:
I ran tracetcp internally and externally.  When I run it externally and monitor wireshark, there is a SYN & SYN/ACK for TCP protocol and the trace is successful.

This is how it should look like

When I run the same command internally, Wireshark shows a Name Query using UDP 137 and the trace times out.
UDP137 is de NETBIOS name resolution..., Queer ... i would expect a udp 53 (IP stack method) to get a name translated. This may be an issue in your server/workstation setup.
Please review the setup of your network. NETBIOS has been deprecated for quite some time now.   Verify DNS resolution has been setup correctly.
(DNS Server, and clients refering to this server).

Something is acting totally different internally on our corporate network when I try to communicate to the destination IP than it does when offsite on a public network.
The equipment that does this is mostly called a Firewall.

Author

Commented:
Not sure if this helps troubleshoot the matter, but I just installed a copy of ExpressVPN and I can access the site when that is connected.  So traffic is being proxied through one of their servers and it works.
CERTIFIED EXPERT
Distinguished Expert 2018

Commented:
What happened when ExpressVPN is added - traffic is being tunneled which means that tunnel is work around.

So, whatever is the root cause in network it is still present and in the case that VPN is down it will be back. Going around issue is not long term solution, it is adding extra layer for troubleshooting and topology is getting more complicated.
Commented:
Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION
Unlock the solution to this question.
Thanks for using Experts Exchange.

Please provide your email to receive a sample view!

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.