XAUTH/MD5 fails

Posted on 2011-10-17
Last Modified: 2012-08-14

I’m trying to set up VPN between openswan (on Gentoo) and iPhone. I’m using identity certificates and XAUTH (if only because there appear to be no way to turn OFF XAUTH on the iphone).

After MUCH head-beating…I’ve gotten it to the point where the SA is formed and the iPhone passes the CORRECT XAUTH username and password (verified within ipsec.log) to ipsec in the ISAKMP_CFG_REPLY message.

The log also shows that the /etc/ipsec.d/passwd file is being opened.

HOWEVER, then the user/pass combination fails.

I used htpasswd –c –m /etc/ipsec.d/passwd <username> to create the file and encrypted password.

How can I trace what’s happening?

From ipsec.log
The lengths of XAUTH-USER and XAUTH-USER-PASSWORD are both correct. The decrypted HEX code from the packet payload shows that the correct user/pass is being sent by the iPhone

| ***parse ISAKMP Mode Attribute:
|    next payload type: ISAKMP_NEXT_NONE
|    length: 34
|    Attr Msg Type: ISAKMP_CFG_REPLY
|    Identifier: 0
| removing 6 bytes of padding
| XAUTH: HASH computed:
|   4e 66 27 7c  c3 05 8a 14  cb 5c d0 e1  76 00 45 d2
|   db 60 90 ed
| ****parse ISAKMP ModeCfg attribute:
|    ModeCfg attr type: XAUTH-USER-NAME
|    length/value: 8
| ****parse ISAKMP ModeCfg attribute:
|    ModeCfg attr type: XAUTH-USER-PASSWORD
|    length/value: 10
"just-ipsec"[22] #13: XAUTH: User mrightmi: Attempting to login
"just-ipsec"[22] #13: XAUTH: md5 authentication being called to authenticate user mrightmi
"just-ipsec"[22] #13: XAUTH: password file (/etc/ipsec.d/passwd) open.
"just-ipsec"[22] #13: XAUTH: User mrightmi: Authentication Failed: Incorrect Username or Password
| **emit ISAKMP Message:
|    initiator cookie:

Open in new window

Question by:Mike R.
    1 Comment
    LVL 3

    Accepted Solution

    The problem was the /etc/pam.d/pluto file was corrupt. I recopied the /etc/pam.d/pop file to /etc/pam.d/pluto and life is good.

    Hopefully this will help someone else who's gotten stumped by the same issue :-)

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    What Security Threats Are You Missing?

    Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

    What's a UDID? If you're involved in developing, testing, or even reviewing an iOS application that's in beta, then at some point you may need to know the UDID for any iOS devices that you'll be testing on. What's the UDID? It stands for Unique Dev…
    iCloud Drive was introduced after iOS 8 was launched last year. This drive is Apple’s online storage device that lets users sync their files and access them from all their Apple devices.   There is a lot of data that is not automatically backed up…
    After creating this article (, I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…
    After creating this article (, I decided to make a video (no audio) to show you how to configure the routers and run some trace routes and pings between the 7 sites…

    737 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    20 Experts available now in Live!

    Get 1:1 Help Now