Exchange 2013 email delaying

Today we faced the Email Delay issue on Exchange environment. Emails were delivering with 5 to 7 minutes delay. On investigation, we found following errors on server. It appears from the first error there could be some connectivity issue between the Exchange servers, but both the servers are located on same chassis, the chances of connectivity issue is very low. After restart the issue gone it also shows there was no connectivity issue. What could be the issue?

Error on Server Queue:
{LED=441 4.4.1 Error encountered while communicating with primary target IP address: "Failed to connect. Winsock error code: 10048, Win32 error code: 10048. "Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or deliver failed to all alternate hosts. The last endpoint attempted was x.x.x.x:475]

Warning on Server Log:
TCP/IP failed to establish an outgoing connection because the selected local endpoint was recently used to connect to the same remote endpoint. This error typically occurs when outgoing connections are opened and closed at a high rate, causing all available local ports to be used and forcing TCP/IP to reuse a local port for an outgoing connection. To minimize the risk of data corruption, the TCP/IP standard requires a minimum time period to elapse between successive connections from a given local endpoint to a given remote endpoint.