Am I losing packets, pls see the attached ColaSoft Capsa screen capture


I need advice and translation of what this capture - attached means.  This is the situation.  I have a windows app that uses a web service on our web server - the web service accesses one of our database servers.  Recently it started to ramdomly fail to retrieve data.  I cahnged to completey different database server but still the same.  I moved web service to another server but still fails randomly.  When I say fails I get "An existing connection was forcibly closed by the remote host" In IIS 6 on the server everytime a failure occurs I can see in the logs an entry that has sc-win32-status with a value of 64.

I did an HTTP capture using  ColaSoft Capsa (Wireshark seemed too difficult to understand).  Now I got loads of TCP duplicated ACKs but I am not sure if they are indicating that there is a packet loss in my network.  PLease have a loo at the attached image.

Can someone please comment or give advice here.  Please note that when I moved the Web service to my local PC (localhost) IIS it works fine from opther machines in our LAN

Thanks in advance
 Traffic capture
LVL 12
Who is Participating?
nociConnect With a Mentor Software EngineerCommented:
1st: draw some maps of how traffic flows

along: IIS -- server -- Switch/Hub -- .... Switch -- Your PC -- Browser

For every Component (IIS, server, Switch/Hub...) determine its status, and collect possible logging...
for every connection (--) get the properties as defined, and as actualy seen on both ends...

for the 1st Component
IIS is it running, any logging,...

for the 2nd Component Switch/Hub
is it a switch, (is capable of full duplex) or a HUB cannot handle FD
is it managed (switch only)
is there logging

for the first --:
How is IIS seing it. (Port 80, IP address X ) configured.
How is the server seeing it (netstat -antb) is there a listening port on port 80, is address X valid..

for the 2nd --:
Is it single channel, multi channel (LAG or Bonded adapter)...
What is the design speed, what is the design duplex
- What is configured on the server
-  What is configured (if possible) on the switch/hum

In some cases that answer is quickly given (f.e. a HUB precludes a lot of things)..
Then you need to look for common failures or mismatched stuff.
Like a HUB on one side of a cable and a FD configured device on the other side...

When you have a complete picture you can also device tests to see if the problem is at a certain place.
Also draw a picture for a working path.

If access to your server for IIS fails BUT other access to the same server at the same moment succeeds you need to take traffic content into view (packet logging, but more detailed then just a summary of how many failed/succeed, to be able to drill down).
MikeKaneConnect With a Mentor Commented:
This doesn't really tell me much....   a wireshark capture is much better.  

However, Dupe ACKs can usually mean network congestion as the TTL on the packet may expire causing the resend of the ACK packet.      HAve you looked at your loads lately on all devices between the 2 hosts?    
gbzhhuAuthor Commented:
I have now put a trace in my app - having tried every solution I googled to no avail. The trace shows me that the web service sends the data back, the data is being transported, the trace shows the actual data, then at some point the connection is lost.

Trace records error (before the error there are thousands of lines showing the data retrieved so far)

System.Net Verbose: 0 : [9732] Exiting ConnectStream#42644125::Read() -> 16000#16000
System.Net Verbose: 0 : [9732] ConnectStream#42644125::Read()
System.Net Error: 0 : [9732] Exception in the HttpWebRequest#63539872:: - The underlying connection was closed: An unexpected error occurred on a receive.
System.Net Verbose: 0 : [9732] ConnectStream#42644125::Close()
System.Net Verbose: 0 : [9732] Exiting ConnectStream#42644125::Close()

While running the app in the trace sometimes I can get the data 50 times before a failure sometimes it fails first time!! My data sizes have shrunk too. The largest is 2.5MB

No idea what else to try now
Prepare for an Exciting Career in Cybersecurity

Help prevent cyber-threats and provide solutions to safeguard our global digital economy. Earn your MS in Cybersecurity. WGU’s MSCSIA degree program curriculum features two internationally recognized certifications from the EC-Council at no additional time or cost.

nociSoftware EngineerCommented:
any chance of auto negotiated port being mismatched.

Speed is never an issue, but fullduplex/halfduplex can. That will cause massive packetloss on mismatch with sufficient traffic.
In a low traffic situation you will not notice any problems.
gbzhhuAuthor Commented:

Thanks for the response.  How would I find out if auto negotiated port is mismatched?  I an nmot network savvy, sorry.

For info, applications between my PC and the web server that my problematic web service is running on work fine.  Previous tests showed that If I host the web service on my local PC and run the client app on my colleague's PC all works fine (no firewall/router between our PCs)
nociSoftware EngineerCommented:

One way to find out is if a tool like netio reports assymetric speeds.

Another is looking at a interface counters.  (lots of short packets (=RUNTS) & CRC errors ) esp. on the FD side.
Issues mostly occur when hubs  or hard configured interfaces are used together with auto configured interfaces.
gbzhhuAuthor Commented:
What would you do if you were me.  Could you give me steps to follow to troubleshoot the issue?

Thanks noci
gbzhhuAuthor Commented:
OK.  The reality is now hitting me... I need to do some learning.  I have a key to our server room and admin password but I can only identify servers then I see a firewall box and a couple of boxes that say SuperStack 3300 plus 5 Blackbox server switches boxes.  Don't know where the router is.  I need to figure out what is what first

I would take IIS out of the equation but include the server where IIS is running in the investigation.  IIS seems to be functioning fine and returning iis sc-win32-status of 64 when connection drops

nociSoftware EngineerCommented:
So that what should be done with all components... (I intentionaly added the IIS as a part of the chain, it's easily verifiable.)
OTOH the next step is the server interface, possibly settings on it & firewall settings on the server.
gbzhhuAuthor Commented:
I have recreated my whole service in NET 1.1 and hosted on a different server and also tried on the same server.  This suggests possible firewall/router/network issue.  

My client has been waiting for too long and I decided to patch it up so I put a retry (up to 5 times when it fails) and it now managed to run 200 times (with those retry machanism).  I am going to ahev to leave it at that as I don't have the expertese to troubleshoot the infrastructure and not much time either as my other projects are being delayed by this issue

Many thanks for your input.

I am assuming there is no easy way to get to the end of this but you think there is please let me know

nociSoftware EngineerCommented:
No easy way, but try to use this as an excersize, at least it gives you a guide how to drill down .
Learning the properties of your equipment can be beneficial too for other trouble shooting.

Also if you write the paths you start documenting you infra & environment. Which is a huge benefit with future changes...
gbzhhuAuthor Commented:
But how do I find the paths?  I have no clue.  Follow wires?
nociSoftware EngineerCommented:
Follow wires for the switched parts, study switch/host configurations.

IP Path can be fount using f.e. traceroute
nmap is a tool that can find which ports & machines on your network are accessible.
gbzhhuAuthor Commented:
Thank you noci

Where would I find the switch configuration?
nociSoftware EngineerCommented:
That depends on the switch.

- Unmanagebale switch (there is no config available)
- Web managed switch: point your browser to the ip address of the switch (the address is either hard coded (see manual), or DHCP requested, see reservations log of your DHCP server))
- Possibly you can telnet into the switch, see documentation for your switch.

A hub is more like an unmanaged switch, but all wires are cross connected here there is no packet forwarding, just electric signal forwarding. [ Hence it can only work half duplex as it has no buffers ].
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.