VPN L2TP connection to Zywall USG100, Windows xp error 789

Raikka_
Raikka_ used Ask the Experts™
on
Hello.

Windows XP Pro VPN L2TP connection to Zywall USG100.

error 789 : The l2tp connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer

It happened to windows xp pro dial up connection only, so when laptop is behind another zywall usg100 ADSL connection VPN works fine, but when laptop is dialup connection (mobile) error is there. Where is problem?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®

Commented:
Hi,

does the zywall show any logs on this?
Is the windows firewall off? (or any other firewall- macafee, etc) udp 4500, 500 ESP (protocol#50) traffic might be getting blocked
Also, are you using a preshared-key for the xp client and the zywall? (if so, try re-setting)


on the xp machine, make sure the following services are started, and if they are. try restarting, then try connecting:

IPSEC Services

Author

Commented:
No logs at all, windows firewall is off, yes preshared key and ipsec services is restarted. Same problem.

Behind ADSL and another zyxel USG100 workstation connects well.

Commented:
any detailed logging besides error message in windows event logs?
Ensure you’re charging the right price for your IT

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

Author

Commented:
Yes, Event ID 1002:

The IP address lease 93.106.8.170 for the Network Card with network address BAD0BEEFFACE has denied by the DHCP server 93.106.196.28 (The DHCP server sent a DHCPNACK message)

Author

Commented:
Computer IP is:
93.106.196.27

gateway:
93.106.196.29

VPN server (USG100) where trying to connect IP is:
83.145.227.xxx

Author

Commented:
Client VPN error:

Error 792

and

Error 789

Commented:
the workstation that connects successfully-  is it getting a dhcp address in the 93.106.8.X  subnet ?

Commented:
Is the xp machine using a pre-shared key?  This would be in the properties of the vpn connection > security tab>IPSec settings

you stated earlier zywall is using a preshared-key.  Did you reset this key on the client having the issues?

Author

Commented:
Client is using windows xp built-in VPN connection L2TP, when client is behind another USG100 using builtin VPN it works well, but when using telephone GPRS connection, it fails.

Yes, workstation GPRS connection is 93.106.199.x subnet... There is also 3more laptop, they all use different GPRS connection and all fails.

Only client that what is behind ADSL works well.

Author

Commented:
Preshared key is correct, bacause it works behind ADSL

Commented:
did you check all dialup settings on the zywall? Is the zywall configured to accept traffic on the 93.106.8.X
can the dialup machines ping the vpn interface of the zywall before attempting vpn connection?

Author

Commented:
No, client cannot ping Zywall WAN ip even when VPN connection is establish behind ADSL connection:
ping 83.145.227.xx

reguest timed out.

Author

Commented:
Now there is open ping from:
ANY to ZYWALL

But it doesn´t help...

Commented:
you are positive that the zywall does not have a specific dial-up profile for remote access-users?

Commented:
try this
on the properties of the vpn connection, go to security tab, then advanced
on the drop-down menu, select "optional encrytion"  to see if it connects without encryption
make sure mschap v2 is checked
we can work out encryption issue once we can connect
then on the networking tab under "type of vpn" click settings
make sure
enable lcp extensions and enable software compression is checked
on type of vpn, try "automatic"

Author

Commented:
Tested, Don´t help....

Zywall USG manual, there is exsample L2TP, that is where settings are.... Here is similar settings....

But won´t work!:!:!:!::!

Commented:
I will pick this up later this evening. I will need to look at the zywall manual for this. It might help to use a network sniffer such as Wireshark (free) to get packet captures to see what is happening inside the wire.  You can use built-in netmon in administrative tools also

and again, are these remote users using strictly dialup, ethernet cable or wireless
sorry forgot to ask earlier

Commented:
p.435-439 gives the settings for the windows client, it does not differentiate if it is strictly dialup. You can make two seperate connection, one strictly for dialup, the other for ethernet, with suggested security settings and try

Author

Commented:
>and again, are these remote users using strictly dialup, ethernet cable or wireless
>sorry forgot to ask earlier

Dialup connetion (mobile phone) This is strange, because VPN connection works well behind ADSL (cable connection) Is XP VPN client "general" setting "first connect" "dial another connection first" setting okay, it is empty? And it works behind ADSL, not working telephone connection?

Author

Commented:
USG100 manual page 447, there is L2TP VPN Example, that is using client computers, just like manual settings...

Commented:
I don't follow you on last comment--

Author

Commented:
Zywall USG 100 manual:
http://www.zyxel.com/web/support_download_list.php?indexflag=20040906173729&ModelIndexflags=0,420070329092754

Version 2.12 page 447 --->

There is L2TP VPN example, that example is used to connect client computer to USG100 VPN router, and it works through ADSL connection but not "mobile" VPN connection...

Commented:
I am downloading manual, I have an older copy and the page reference is wrong. will take a look at this

thanks

Commented:
are we looking at same thing? Am I missing something, as the example just shows a remote user. Will this firewall allow  PPTP vpn using the windows client instead of l2tp?

Author

Commented:
USG100 allow 3 different VPN settings:

IPSEC VPN
SSL VPN
L2TP VPN (inside IPSEC)

Commented:
my question is, most of these remote clients are using wireless cards at various locations, and some using PPP dialup connections?
Commented:
another question I have is that the ip addresses that are assigned to local clients should be private ip's, in a different subnet that your inside lan hosts.

 A few posts ago you stated >>The IP address lease 93.106.8.170 for the Network Card with network address BAD0BEEFFACE has denied by the DHCP server 93.106.196.28 (The DHCP server sent a DHCPNACK message) Are you referring to a dynamic ip that a remote client is getting at their remote location, or an ip the zywall is handing out?

please clarify

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial