Help reading a Network Packet Capture related to TLS / port 443

First, please understand that I'm not an expert (at all) in networking or reading packet captures.

I'm troubleshooting an issue between a Web server and PayPal IPN.  The issue appears to have started around the time PayPal stopped supporting SSL 3.0.

The web server is IIS6 so it ONLY supports TLS 1.0.  All versions of SSL are disabled.

What I'm comparing are two wireshark traces of PayPal connecting to a "special URL" on the web server (to perform the IPN postback after the paypal transaction processes)

- One trace is of the PayPal simulator connecting to this special URL on the web server (PayPal's simulator reports this as successful)

- The other is a live paypal transaction (the step where PayPal IPN connects to the special URL to post back the transaction).  PayPal reports it fails to connect to this URL in some way (I haven't been able to get more details, their canned notification says things like check firewall, etc. but this clearly isn't that anything is blocking PayPal)

In looking at the traces, there are some differences I'm not quite clear on.  On the 'simulator' trace (the one PayPal reports as successful), the connection appears to be attempted only using TLS 1.0.

On the trace of a live PayPal transaction, the packet details are showing both TLS 1.2 and TLS 1.0  (the IIS6 server doesn't support TLS 1.2 but it does support TLS 1.0)

My first question is:   On the live transaction, is it that PayPal attempted TLS 1.2 first, and then since TLS 1.2 is not available it then attempted TLS 1.0?   (based on the order in the "Secure Sockets Layer" section, is this correct way of reading this (TLS 1.2 then TLS 1.0).  My understanding of SSL/TLS in general is that it would try the higher first and if not supported try older versions but I'm not sure if that's what the packet shows.

Or is it stating which TLS versions it supports and the next step is the web application or server to select a version? (and if so, is it the web SERVER or the web APPLICATION that makes that choice?)

Any idea what to make of these two traces, I would have expected a simulator to work the same as the live version but it seems to be clearly connecting in a different way and I'm not quite understanding it.

Any guidance would be much appreciated.

Mostly trying to determine if the issue is with the web application, the web server (IIS6), the server (Windows Server 2003)





Who is Participating?
Sean JacksonConnect With a Mentor Information Security AnalystCommented:
For the handshake to work, both the server and the client must have matching cipher algorithms in the array of usable ciphers.  As the handshake starts, the client hits the server and says "I want to initiate communication with you". The server says (if the client hit port 80), "We're going to take this over to port 443 because we're going to encrypt this traffic". The client sends over a list of ciphers it likes to use. The server starts at the top and tries to match. If it can't communicate, they do what is called a 'Downward Dance' and they choose weaker ciphers and try again, over and over, until they find a match. Then begin communicating the details of the encryption to be used, setting up secret keys and agreed secrets, until the handshake is complete and the session is begun. If the connection is interrupted, often the whole negotiation must be started over.
VasAuthor Commented:
Thanks, so by looking at the traces (and I can send screenshots of the other sections of the frame) does it look like maybe it's an issue with the cipher suites?   PayPal supports 80 ciphers based on part of the packet capture and the two ciphers that the TLS 1.0 server supports are listed in PayPal's list so I thought we were OK there.

Or does this look like some other issue?

Basically after that 'Client Hello' frame on the live transaction the next frame is a FIN/ACK.   Wouldn't a renegotiation be another frame if that was the case of it needing to try a different cipher?

Sean JacksonInformation Security AnalystCommented:
The next frame being a FIN/ACK is odd. You want a SYN/ACK.  

That sounds like a closed port (FIN) and then the next step in the handshake (ACK).
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.