ISA Server keeps dropping TELNET and other TCP/IP application sessions after 66 seconds
I have an ISA 2004 Std Ed Server with 2 NICS. One internal and one connected to the Internet. On the internal network I have a Cisco router connected to a client network via 256k leased line. I have configured ISA to have our clients IP address range routed through this Cisco router.
The issue is that local PC clients on our local network keep getting their TELNET and other TCP/IP sessions dropped after 66-70 seconds. However, we have at least two remote PC clients who access our network via VPN through the same ISA server and they do not get this problem, i.e. the VPN clients can keep TELNET and other TCP/IP applications open up continuously.
I have tried to alter the TCP KeepAlive heartbeat and session time to 1 second and 2 hours consecutively with no success on the local PC clients.
The issue, that VPN clients do not loose the connection and internal clients do and ISA is affected (blocking), I assume, that the telnet session runs on the same machine than ISA?
It points me to session keep alive packets through ISA. As an VPN clients acts as internal member of the network (which means, all traffic is passed through), keep alive packets will pass the server. Internal clinets, which will touch the ISA are alos restricted by the ISA roules. That means, that all rules on ISA will be valid for them as long as traffic is routed to or through ISA. Have a look at the ISA protocol trace, what happens to SYN and AKN packets, maybe they are blocked and this is the reason why the telnet server will close the session as the packets never will reach them. Also the fact, that your changes to keep alive settings has no effect, points to this.
Also have a look to alerts is ISA, if the server interprets keep alive packets or others as spoofing. In that case, the ISA will drop subsequent connection requests for an amount of time. Within the gernal setting for ISA, you have some settings for attack investigation (Flood-Protection) and the thresholds there. You can define a computergroup (i.e. your local subnet) and add this group to the IP-exceptions. You can define normal and exceptional thresholds, which avoids ISA from regognizing internal clients as flood-source, if they send a lot of packages.
Some general remarks:
The telnet server itself will drop the session, if no activity is recognized. have a look here:
To avoid the timeout, the client has to send keep alive packets to the server. This can be made by an underlying service as well as from an application (i.e. Internet Explorer).
The session keep alive packets have to pass ISA to take effect.
If your internal clinents are also using the cisco router, keep in mind, that also the cisco router may drop the packets. Usually you can change the settings there. If VPN tunnel through the cisco router and to ISA as endpoint behaves different, as the cisco router does not see the telnet traffic but only the vpn connection. If the cisco is setup to drop telnet sessions but nor vpn traffic, thsi may also affect your problem.
Hopes this helps for futher analysis.