Browsing Network Shares & Files over VPN tunnel

Hmmm...    Using Untangle devices to connect 4 branches:   Houston, Dallas, Austin, and San Antonio.

VPN tunnels have been configured for some years and working for that same time period.   About 6 to 9 months ago, we started having problems when trying to access network shares in Dallas (from any other branch).    So, I can ping Dallas AND I CAN BROWSE some of the DALLAS shares.   But most of the shares it allows me to browse are relatively empty (very few files).   Anytime I try to drill down into a share that has a large (but not crazily so) number of files, I get an hourglass for a minute or so and then an error (\\server\share\directory is not accessible.   You might not have permission to use the network resource...   The specified nework name is no longer available.)
Of course, if I am on the LAN  in Dallas, everything works great (I can access all the shares no problem).
Also, from Dallas, I can browse and access any shares on the remote servers in Houston, or Austin, or San Antonio - going through the VPN tunnels of course.

Last little tidbit of info...      If I try to run an RDP session on the Dallas Server (from say Houston) going through the VPN tunnel, I connect but get a blank screen - no video.    I CAN do an RDP session to the Dallas Server - but I have to come in directly from the Internet (ie, use the public IP address - not the VPN tunnel).

So, what kind of strangeness could be blocking/preventing network shares from working ?!?!?   Any ideas !?!?
Who is Participating?
ponedogConnect With a Mentor Author Commented:
Solution found - with credit to Untangle Support (Chad).

An option in the Untangle box was unchecked that needed to be checked.   Specifically...
APPS --> IPsec VPN --> IPsec Options --> Traffic Processing:   Bypass all IPsec traffice - Check this BOX !!!!     Of course, we have no idea how/who unchecked this box.  Probably happened in the ancient mists of time and only showed up when someone rebooted the Untangle box.

Screen Shot of Untangle IPsec Option that MUST be checked.
skullnobrainsConnect With a Mentor Commented:
can't really tell but this looks like a common networking mess with VPNs

my guess would be

VPN is configured in udp mode so it will send 2048bytes cells whenever it has that much data to send or whenever there was data in waiting for a few ms.

this by itself induces quite poor tcp connectivity in many cases because the RTT is irregular. when the vpn is heavily used, it actually works better and quite seemlessly

then you may notice the udp cell size is bigger than the MSS ( likely around 1400-1500 ) so those cells get to be split. the split actually reduces bandwidth unless the cell size is a multiple of the MSS. additionally if either endpoint or any statefull firewall equipment is slightly saturated in the middle, it will have a hard time to cope with reconstructing the split packets an possibly drop some of them.

the above would explain situations in which small queries work but larger ones don't : basically any question or answer above the MSS is likely to be lost in space.

this can usually be confirmed with simple ping tests : it is likely that pings above a specific size work erratically.

if that is actually the case, first check the connections the vpn runs over, and possibly use smaller cells, switch to tcp mode, let the ISP handle the problem on his side.... depending on your findings
ponedogAuthor Commented:
Running Ping tests (eg: ping dallas /f /l 1418) shows that 1418 is the maximum packet size without fragmentation.

However, this is true of all the VPN tunnels to each of the branches.   So, there does not appear to be a difference between how the Untangle is set up for the different VPN's.

However, it does appear that Dallas is somehow different since we seem to have our problems "clustered" with them.    

How would we tell if the ISP (and how they were handling packets) is part of the problem ???
good to see you "solved" it and it is actually quite a frequent way to solve vpn problems. but ipsec is pretty much the security of the vpn and only needs to be disabled either when the cpu of your endpoints can't cope with the traffic or when the network is a mess ( encrypted connections break easily with little packet loss ). please make sure it was disabled in the first place and make sure this is the security policy you want.
ponedogAuthor Commented:
Answer came from Untangle - clearly the answer to my problem!
All Courses

From novice to tech pro — start learning today.