Need help decoding VNC challenge/response string

I'm trying to run x11vnc with the --passwdfile option. This option sends a challenge string to the VNC client. The client opens a dialog on the user's screen asking for a password. The client then sends the response string back to the VNC server. The VNC server calls the program referenced by the --passwdfile parameter with this string in the following format:

A two-byte length (in this case 16), followed by the challenge string (16 bytes), followed by the response string (16 bytes). What I get back, in hex is:

 0: 0A 6E 65 78 74 0A 31 36 0A 23 36 F4 E1 03 EE 30    .next.16.#6....0
10: 16 85 FC E9 4C F1 F5 16 5C 2C D5 5C 93 C2 21 29    ....L...\,.\..!)
20: 3A DF C2 A2 7C E9 1F 1A D7                         :...|....

Select all Open in new window

The <newline>next<newline> can be ignored as this is debug stuff from my script. The strings are:

Challenge:
23 36 F4 E1 03 EE 30 16 85 FC E9 4C F1 F5 16 5C

Response:
2C D5 5C 93 C2 21 29 3A DF C2 A2 7C E9 1F 1A D7

Select all Open in new window

The manpage defines the response string as, "client's response (i.e. the challenge string encrypted via DES with the user password in the standard situation)." I have no further information or documentation. I assume one must use the challenge string to decode the response string and get the user-entered password, but I've no idea how to do that.