• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 802
  • Last Modified:

Slow Terminal network traffic over VPN

We have connected two offices through a Cisco Lan-to-Lan VPN over the Internet. Users in one office Telnet to the other office to log on to a Unix app. This infrastructure is mandatory and cannot easily be changed.
The problem is that the Telnet connections can get very slow on occasion.

In an attempt to speed up the connections, we upgraded the connecting office from an ADSL line (4096 kb down, 512 up, not guaranteed), to an SDSL line (2048 down, 1024 up, guaranteed).
This hasn't improved things very much though.

I have the feeling that, because the Unix app is terminal based, it initiates TCP connections for each character pressed (a delay is noticeable when typing text), and therefore ping times are much more important than total bandwidth capacity. Ping times, however, have not decreased significantly, due to the crypto overhead of the VPN connection and the Internet tunnel that has not changed.

1. Can someone confirm or deny this? Do terminal-based apps initiate TCP connections per character inputted?
2. How can I improve performance on these connections?

4 Solutions
In a VT100 (or derivative) based terminal emulation mode, typical for *nix text consoles, each character you press will be sent to the server without waiting for an 'enter' or other signal generating key. At the TCP level, this only results in session data and acknowledgment packets being transmitted - a new session is not generated for each communication.

The actual process for queuing is more complicated; it is sufficient to say that unless you are a really fast type, most TCP data packets will only contain a single keypress. These packets, however, amount to very little total traffic. Under normal operation, 50+ terminals or sessions should have no bandwidth trouble even on a 56K modem line.

As for response time, your users will notice anything more than about .02 seconds. If you are encrypting traffic through this tunnel, it will all traffic down. In general, what kind of ping responses are you seeing? Also, are you using a centralized encryption server, or is each computer handling it's own encrypted tunnel overhead?
The "on occasion" is very vague. I also have users at 5 remote offices (2 WAN's (384kbs no guarentee) and 3 56K lines) that telnet to our unix server. That complain about lag "on occasion".

What to watch for is the specific "times".
I have found that the "LAG" is tied directly into how busy the server is. If there are lots users generating reports (as we have in the morning) you will notice some lag.
The more processing the server is doing the slower its response time will be. Even on the LAN.

It could be your connection because there is no such thng a "guarenteed".
If you have access to the remote computers you could create a simple batch file that pings the server every 30 minutes and see when this is happening. This may help to confirm if its the server load.

What is the average pig time??
I have a 35ms average response time from my cable connection to my server on the net on the servers 128k connection. Which is normal for me.

And as bm stated, encrypting the data will increase the time somewhat. Our WAN's and 56k dont use encrytion so I cant recommend a particular one. But changing the level of encryption might help. Especially if the computers are doing the encryption.

Also one other question would be is it all the computers at the site or just one? If its only one, it could be tied to what ever else the user is doing on their computer.

One thing try it you a using IPSec is to goto PPTP. IPSEC is very heavy on CPU usage. Or get a faster Vpn
Who's Defending Your Organization from Threats?

Protecting against advanced threats requires an IT dream team – a well-oiled machine of people and solutions working together to defend your organization. Download our resource kit today to learn more about the tools you need to build you IT Dream Team!

Some of the current Cisco VPN routers include a crypto on silicon processor which offloads crypto from the CPU. You will get a minor amount of lag over the VPN but it shouldn't be to the point that users are complaining. Check that it's not an MTU issue - do your DSL services use PPoE? If so add 'ip tcp adjust-mss 1452' to your inside interface (eg E0 on an 837) and 'ip mtu 1492' on your Dialer interface.

Whats the latency between your clients and the server? I doubt that the VPN tunnel would have any impact on latency when dealing with telnet. Telnet is very light in terms of bandwidth usage (and hence stressing an encryption CPU).

As stated above, telnet does not initiate a new session per keypress. One tcp connection is initiated on initial startup. However each keypress is individualy sent across the wire, and requires acknowledging. If it is a Cisco router, there is a service that you can enable to "help" speed up applications like telnet, and this is called "nagle" (nagle algorithm). You can enable this by typing "service nagle" under global config, however I am not sure if this will actually achieve what you want. For more info look at: http://www.cisco.com/en/US/products/sw/iosswrel/ps1826/products_configuration_guide_chapter09186a00800d9b69.html

It basically packages lots of keypresses up and sends them in one packet.

I would more inclined to look at the server CPU etc at the times the service is slow. Or(and/Or) look at the latency at this time (i.e. do a ping). if the RTT is more than 100ms then users will notice a lot (in my experience)..

Bandwidth with telnet will do very little.

Best regards (and luck)

Just in case, check that the output from the following is the same on both ends of the VPN:

show crypto ipsec sa | include mtu

If it's not the same, then you have three options - one is to set the router with the larger MTU to have the smaller MTU on its outgoing interface (eg in BVI 1 ; ip mtu 1492), second is to use a route-map that clears the df bit, eg I use ACL 130 to define traffic for one VPN where one is DSL bridge mode, the other is DSL PPPoE:

route-map clear-df-bit permit 130
 set ip df 0
interface Ethernet 0
 ip policy route-map clear-df-bit

The third is to use the most excellent DF bit override from Cisco:

I don't think this is your problem but it's definitely worth a check.
int21dotorgAuthor Commented:
To address some of the earlier comments, changing the infrastructure (including the use of IPSec) is not an option. About the "vagueness", I'm aware of this, but we hadn't been able to measure on peak moments yet.

We've collected ping responses today, and they've been more than acceptable, i.e. on average well below 200ms. However, there were no speed issues noticeable today. Maybe people don't work as hard on Friday...
Bandwidth usage was measured by our ISP, and we consumed about 10-20% of our capacity today, so no worries there either.

After reading the comments, I'm more and more convinced that this must be a latency issue, and as soon as we have slowdowns again I'm going to test the nagle service on our Cisco, which seems to address exactly this situation. At the same time, I'll have the other end (a client of ours) test server CPU load, and I take note of the possibility of synchronizing both sides' MTU.

Thanks for all comments so far, very constructive, I'm confident I will be handing out points soon! :)


Featured Post


Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now