Windows XP VPN Server - Firewall and NetBIOS resolution

I have a client that doesn't have the budget for a server so they're using an XP Pro box as a file "server". They do have a static IP through an Actiontec DSL modem. I've managed to setup their XP file server as a VPN server and it works... but, I'm missing a few things I'm hoping I can get answers to from Experts.

I can get VPN to connect just fine. But, I only get to shares if I turn off the Windows XP firewall and use the  UNC "\\IP address\share" convention. I can't get NetBIOS UNC ("\\server") to do anything. I tried adding an LMHOSTS file in the "Windows\system32\drivers\etc" folder, but it doesn't seem to have done anything (with or without the firewall on). So, I know I'm missing an exception.

Port 1723 is forwarded at the Actiontec to the XP server. GRE is enabled. The XP firewall has "Incoming Connection VPN (PPTP)" checked and configured to use TCP 1723, as well.  I also have TCP Ports 139 and 445, as well as UDP ports 137 and 138 open on the Windows XP firewall and have them configured for "any" computer, not just PCs in the same Subnet.

So... I've narrowed it down to a Windows XP firewall configuartion issue, since it mostly works when I turn off the XP firewall... but, I must be missing both an exception and something else for NetBIOS name resolution over PPTP.

Any thoughts?  
Who is Participating?

Improve company productivity with a Business Account.Sign Up

Rob WilliamsConnect With a Mentor Commented:
>>"you'll see that I already have the ports you suggested open and configured for "all"... "
Sorry obviously more than just a "memory lapse"  :-)

>>"I didn't realize that you needed that LMHOSTS file on both the server and the client."
LMHosts only needs to be on the client machines. No need on VPN server end.

With the firewall enabled, and those ports open, can the VPN still connect ?, and if so does connecting by IP work?
If you can't connect we may need to look at other ports.

If you can connect by IP but not NetBIOS name, try enabling NetBIOS over TCP/IP on the WINS tab of the network adapters advanced properties.
You shouldn't need to, but try enabling both TCP  & UDP for all 135-139, 445, and 593, at least as a test.
Rob WilliamsCommented:
If you do not have any DNS or WINS servers available you are pretty well limited to connecting by IP, or using the LMHosts file. The latter actually works very well. It needs to be installed on each client/connecting computer. An outline of its use:
An option is to use the LMHosts file which creates a table of IP's and computer names. LMHosts is located in the Windows directory under c:\Windows (or WINNT)\System32\Drivers\Etc\LMHosts.sam , instructions are included within the file. Any line starting with # is just a comment and is ignored. Open the file with Notepad and add entries for your computers as below;      CompName       #PRE
Hit enter when each line is complete (important), then save the file without a file extension. To be sure there is no extension ,when saving enclose in quotations like "LMHosts". Now when you try to connect to a computer name it should find it as it will search the LMHosts file for the record before connecting.
More details regarding LMHosts file:

As for the Windows Firewall, you should only need to enable the File and Printer Sharing Exception (included TCP 139 & 445 and UDP 138 & 139, except check the scope options and set to allow all computers for each port, under the edit button.
Rob WilliamsCommented:
Sorry above ports should read TCP 139 & 445 and UDP 137 & 138  (memory lapse)
NEW Internet Security Report Now Available!

WatchGuard’s Threat Lab is a group of dedicated threat researchers committed to helping you stay ahead of the bad guys by providing in-depth analysis of the top security threats to your network.  Check out this quarters report on the threats that shook the industry in Q4 2017.

philodendrinAuthor Commented:

If you read my posting again, you'll see that I already have the ports you suggested open and configured for "all"... not just "subnet". So, I'm still stumped on that one.

I didn't realize that you needed that LMHOSTS file on both the server and the client. I created the file on the client and it worked like a charm... so, thanks for that one.

Still no luck on the firewall issue. If I turn off the XP firewall... all is well. If I turn it on, even with TCP 139, 445 and UDP 137, 138 I get denied access to shares. Very strange.

philodendrinAuthor Commented:
To simplify and elaborate on the problem...

I can "connect" to VPN regardless (firewall on or off). Once connected, I can't ping the local IP address of the VPN server if the server's XP firewall is on. Nor can I get to shares via UNC... by IP or name.

With the firewall off... UNC with IP works without anything extra and now that I understand LMHOSTS... UNC via NetBIOS also works as long as the server's firewall is off. I can also ping the local IP of the server without issue when the server's XP firewall is off.

I'll try your test suggestion... and report back.
Rob WilliamsCommented:
OK, thanks let us know how it goes. Ping is blocked by the firewall by default. There is a whole set of rules for ping (ICMP requests) on the advanced tab of the firewalls properties control panel (ICMP "settings" button). Though no need to configure that.

Do you kneed the firewall enabled? I agree it is a good idea, but most disable it when behind a hardware firewall/router and running a VPN. Wrong attitude, I know.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.