asked on TCP Sliding Window and iSCSI
The port channel spikes up to about 10Gbps and most of that will be iSCSI. So the connection initiator to target works for the most part. I've planned to enable the jumbo frames as recommended by Cisco so that the 1514 iSCSI will be better processed and not dropped.
But my question is this: With iSCSI, are TCP conversations lengthy or very brief? To what degree would some dropped frames (.003%) in-path cause an issue for iSCSI TCP conversation? Or would this percentage just be noise that TCP connection orientedness should just deal with?
Changing MTU on the switch will not hurt in any way or form on communication (buffers will get larger, also less buffers may be available).
Another thing is changing MTU on the IP stack. The IP MTU of all systems in the same broadcast domain (aka LAN) needs to be the same, or things will grind to a halt.
So if iSCSI is on a separate (V)LAN then it can be changed.
Another thing is changing MTU on the IP stack. The IP MTU of all systems in the same broadcast domain (aka LAN) needs to be the same, or things will grind to a halt.Not really, at least for TCP traffic (I have no knowledge if there is some control mechanism for UDP traffic so I will consider only TCP traffic here - also I am deliberately ignoring the fact that MTU configured on host is not the same one MTU configured on Cisco switches)...
With all that said - the only requirement for switching equipment (the same VLAN) is that size of switch MTU configured on switches along path needs to be equal or bigger than MTU on hosts. Hosts are negotiating MSS during 3 way handshake, since MSS + 40 = MTU end hosts will be aware of each other's MTU therefore MTU on hosts can be different.
MTU != MSS.
Unix interface MTU = 1500 (ethernet) is net payload.... ==> switch packetsize = 1514. (or 1518 with VLAN tags).
For any device: if a packet is larger than then MTU it is ignored. (and only counted in the statistic counter: packets too large).
TCP may have MSS negotiation UDP does not. So all UDP frames that are 1501+ bytes will be lost.
You will experience blocked sessions as at sometime one side decides to send something larger and then to wait for the result that never comes.
On switches you need enough to capture any packet...
Also if the endpoint communicate both with MTU = 9000, and there is some intermediate having less somewhere (Switch,router..) then MSS will be around 8960 range.
No communication will happen though.. (unless packets actualy are below 1500 bytes..
You mean you need to determine the PMTU (path largest MTU that still works) that should be determined BEFORE starting MSS negotiation in TCP.
The same issue will happen with devices tunnelling traffic, and some traffic having the don;t fragment bit set... been seen it. Some 1500 byte packets were not delivered across a PPPoE connection causing random breakdowns of Phone calls. Some people could call, some could not. (no Invites arriving) due to too many options and some path info in the SIP packets.
then obviously
MTU != MSS
Regarding
Unix interface MTU = 1500 (ethernet) is net payload.... ==> switch packetsize = 1514. (or 1518 with VLAN tags).I wrote:
For any device: if a packet is larger than then MTU it is ignored.
I am deliberately ignoring the fact that MTU configured on host is not the same one MTU configured on Cisco switchesMeaning MTU = 1500 on host L3 interface is not equal to MTU = 1500 on switch for L2 transport, but MTU = 1500 on SVI is the same one MTU = 1500 on host. I did not want to go into details and still not planning to do that. :)
TCP may have MSS negotiation UDP does not. So all UDP frames that are 1501+ bytes will be lost.My explanation is ignoring discussion regarding situation when hosts are sending L2 frames bigger than what switch can accept:
With all that said - the only requirement for switching equipment (the same VLAN) is that size of switch MTU configured on switches along path needs to be equal or bigger than MTU on hosts.Translation:
If switch configured L2 frame size is equal or bigger than what will be sent by end hosts in the traffic will not be dropped by switch.
If switch configured MTU is 9126 bytes than 5000 or 9000 bytes frames send by host will not be dropped (when all overhead is added).
So I guess we both agree on that.
Regarding:
UDP does not negotiate MSSThis only means that UDP has no negotiation integrated into protocol itself, however application can still negotiate MSS if needed/wanted, just as UDP has no reliability, but can be implemented into application,
You mean you need to determine the PMTU (path largest MTU that still works) that should be determined BEFORE starting MSS negotiation in TCP.No. PMTUD is related to L3 - there is no PMTDU in strict L2 (VLAN) so it is irrelevant for the topic. since in previous post I am discussing strictly Layer 2 - this specific sentence
The same issue will happen with devices tunnelling traffic, and some traffic having the don;t fragment bit set... been seen it. Some 1500 byte packets were not delivered across a PPPoE connection causing random breakdowns of Phone calls. Some people could call, some could not. (no Invites arriving) due to too many options and some path info in the SIP packets.
The IP MTU of all systems in the same broadcast domain (aka LAN) needs to be the same, or things will grind to a halt.I am stating that MTU on different hosts/systems in the same VLAN doesn't have to be the same as long as switch can support bigger L2 frames than L2 frames that are being send by end hosts. And I was explaining that on example of TCP since question is related to TCP behavior.
1. switches along path can be configured with MTU 4000, 5000, 9000 as long as end hosts L2 frames + any potential header is smaller than 4000 (the smallest configured MTU on switched path).
2. If different hosts in the same VLAN are configured with different MTU values in the case of TCP there will be no issues as long as statement 1 is satisfied. UDP could be more complicated regarding hosts, but I am network guy and that's why I stated
I will consider only TCP traffic here
Best practice is to configure all switches in domain to support the same MTU value, but TCP can still work with different MTU sizes if few simple rules are satisfied...
These were great thoughtful answers. Thank you!
I have the change planned as you describe Pedrag. Thank you for confirming it's non-impacting. Cisco said as much as well. Still with iSCSI traffic flowing through it I'll be holding my breath as I apply the policy! I wish they'd put that on another switch. But that's what I've inherited. Shared switch, share trunks in fact.