Solved

get dovecot working with ntlm

Posted on 2015-01-13
51
567 Views
Last Modified: 2015-03-09
Intro: Slackware64 14.1 with Samba 4.1.11 Domain Controller/Active Directory (working). Attempting to use Outlook 2007 on WIN7 domain workstation to connect with Dovecot 2.2.15 on DC/AC using ntlm mechanism

OK, newly installed Outlook 2007 on WIN7 workstation HPLAPTOP. Domain name is hprs.local. Domain user is mark. doveconf -n is:
> doveconf -n
# 2.2.15: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 3.10.17 x86_64 Slackware 14.1
auth_mechanisms = ntlm
auth_use_winbind = yes
auth_verbose = yes
auth_verbose_passwords = plain
debug_log_path = /var/log/Dovecot/dovecot_debug.log
disable_plaintext_auth = no
mail_location = maildir:~/Maildir
protocols = imap
ssl_cert = </etc/ssl/certs/dovecot.pem
ssl_key = </etc/ssl/private/dovecot.pem

Open in new window

There is no user account for mark in/etc/passwd. wbinfo -I mark give:

HPRS\mark:*:3000026:100:Mark Foley:/home/HPRS/mark:/bin/false

Note that I tried without including 10-ssl.conf (wherein are the ssl_cert/ssl_key parameters), but got dovecot error "Couldn't parse private ssl_key". So I included ssl. I created the ssl keys per dovecot instructions as:
$ openssl req -new -x509 -nodes -out /etc/ssl/certs/dovecot.pem -keyout /etc/ssl/private/dovecot.pem -days 365
$ openssl x509 -subject -fingerprint -noout -in /etc/ssl/certs/dovecot.pem

Open in new window

Hopefully that's sufficient.

When firing up Outlook for the 1st I used auto account setup (not manually configure server). I got the following messages in /etc/log/maillog:
Jan 13 22:57:50 mail dovecot: imap-login: Disconnected (disconnected before auth was ready, waited 0 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, session=<t1VLvpQMcwDAqABk>
Jan 13 22:57:51 mail dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, TLS: Disconnected, session=<okxSvpQMcgDAqABk>
Jan 13 22:57:51 mail dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, TLS: Disconnected, session=<s0xSvpQMdADAqABk>
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC backend 'gssapi_spnego' registered
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC backend 'gssapi_krb5' registered
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC backend 'gssapi_krb5_sasl' registered
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC backend 'sasl-DIGEST-MD5' registered
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC backend 'schannel' registered
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC backend 'spnego' registered
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC backend 'ntlmssp' registered
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC backend 'krb5' registered
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC backend 'fake_gssapi_krb5' registered
Jan 13 22:57:51 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 13 22:57:51 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=240
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC login failed: NT_STATUS_LOGON_FAILURE
Jan 13 22:57:51 mail dovecot: auth: ntlm(?,192.168.0.100,<topSvpQMdgDAqABk>): user not authenticated: NT_STATUS_LOGON_FAILURE
Jan 13 22:57:52 mail dovecot: imap-login: Disconnected: Too many invalid commands (no auth attempts in 0 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, session=<ESBevpQMeADAqABk>
Jan 13 22:57:52 mail dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, TLS: Disconnected, session=<RWtfvpQMeQDAqABk>
Jan 13 22:57:52 mail dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, TLS: Disconnected, session=<s3RfvpQMegDAqABk>
Jan 13 22:57:56 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 13 22:57:56 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=230
Jan 13 22:57:56 mail dovecot: auth: Error: GENSEC login failed: NT_STATUS_LOGON_FAILURE
Jan 13 22:57:56 mail dovecot: auth: ntlm(?,192.168.0.100,<gIhfvpQMfADAqABk>): user not authenticated: NT_STATUS_LOGON_FAILURE
Jan 13 22:57:57 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 13 22:57:57 mail dovecot: auth: Error: Got user=[mark] domain=[hprs.local] workstation=[HPLAPTOP] len1=24 len2=240
Jan 13 22:57:57 mail dovecot: auth: Error: GENSEC login failed: NT_STATUS_LOGON_FAILURE
Jan 13 22:57:57 mail dovecot: auth: ntlm(?,192.168.0.100,<topSvpQMdgDAqABk>): user not authenticated: NT_STATUS_LOGON_FAILURE
Jan 13 22:57:59 mail dovecot: imap-login: Disconnected (auth failed, 2 attempts in 8 secs): user=<>, method=NTLM, rip=192.168.0.100, lip=192.168.0.2, session=<topSvpQMdgDAqABk>
Jan 13 22:58:06 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 13 22:58:06 mail dovecot: auth: Error: Got user=[mark] domain=[hprs.local] workstation=[HPLAPTOP] len1=24 len2=230
Jan 13 22:58:06 mail dovecot: auth: Error: GENSEC login failed: NT_STATUS_LOGON_FAILURE
Jan 13 22:58:06 mail dovecot: auth: ntlm(?,192.168.0.100,<gIhfvpQMfADAqABk>): user not authenticated: NT_STATUS_LOGON_FAILURE
Jan 13 22:58:08 mail dovecot: imap-login: Disconnected (auth failed, 2 attempts in 16 secs): user=<>, method=NTLM, rip=192.168.0.100, lip=192.168.0.2, session=<gIhfvpQMfADAqABk>
Jan 13 22:58:37 mail dovecot: imap-login: Disconnected (no auth attempts in 46 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, TLS handshaking: SSL_accept() failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol, session=<nzwQwZQMdQDAqABk>
Jan 13 22:58:38 mail dovecot: imap-login: Disconnected (no auth attempts in 46 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, TLS handshaking: SSL_accept() failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol, session=</FMdwZQMewDAqABk>
Jan 13 22:58:38 mail sm-mta[14652]: t0E3wc3D014652: hplaptop.hprs.local [192.168.0.100] did not issue MAIL/EXPN/VRFY/ETRN during connection to MSA
(16 or more of these message)

Open in new window

Note that it did figure out the domain (line 25).
Outlook screen:
Outlook Add New Account error
0
Comment
Question by:jmarkfoley
  • 27
  • 24
51 Comments
 
LVL 76

Expert Comment

by:arnold
ID: 40548219
Not to ... Your error image reflects an incorrect domain.

do you auto discovery setup (DNS/HTTPS) that publishes the outlook settings?
Not sure, but think this was in clouded in the latter posts of the first question in which I participated.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40548222
Just for the heck of it, I clicked 'Next' on then Outlook dialog to try unencrypted connection: /var/log/maillog
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC backend 'gssapi_spnego' registered
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC backend 'gssapi_krb5' registered
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC backend 'gssapi_krb5_sasl' registered
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC backend 'sasl-DIGEST-MD5' registered
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC backend 'schannel' registered
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC backend 'spnego' registered
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC backend 'ntlmssp' registered
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC backend 'krb5' registered
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC backend 'fake_gssapi_krb5' registered
Jan 13 23:31:17 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 13 23:31:17 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 13 23:31:17 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=240
Jan 13 23:31:17 mail dovecot: auth: Error: GENSEC login failed: NT_STATUS_LOGON_FAILURE
Jan 13 23:31:17 mail dovecot: auth: ntlm(?,192.168.0.100,<whntNZUMuwDAqABk>): user not authenticated: NT_STATUS_LOGON_FAILURE
Jan 13 23:31:17 mail dovecot: auth: Error: talloc: access after free error - first free may be at ../auth/ntlmssp/ntlmssp_server.c:456
Jan 13 23:31:17 mail dovecot: auth: Error: Bad talloc magic value - access after free
Jan 13 23:31:17 mail dovecot: auth: Error: PANIC (pid 4069): Bad talloc magic value - access after free
Jan 13 23:31:17 mail dovecot: auth: Error: BACKTRACE: 14 stack frames:
Jan 13 23:31:17 mail dovecot: auth: Error:  #0 /usr/lib64/libsmbconf.so.0(log_stack_trace+0x1a) [0x7f46650cfc0a]
Jan 13 23:31:17 mail dovecot: auth: Error:  #1 /usr/lib64/libsmbconf.so.0(smb_panic_s3+0x20) [0x7f46650cfce0]
Jan 13 23:31:17 mail dovecot: auth: Error:  #2 /usr/lib64/libsamba-util.so.0(smb_panic+0x2f) [0x7f466739519f]
Jan 13 23:31:17 mail dovecot: auth: Error:  #3 /usr/lib64/libtalloc.so.2(+0x1eaf) [0x7f46668deeaf]
Jan 13 23:31:17 mail dovecot: auth: Error:  #4 /usr/lib64/libtalloc.so.2(_talloc_free+0x345) [0x7f46668e04f5]
Jan 13 23:31:17 mail dovecot: auth: Error:  #5 /usr/lib64/libsamba-util.so.0(data_blob_free+0x18) [0x7f466738abf8]
Jan 13 23:31:17 mail dovecot: auth: Error:  #6 /usr/lib64/libgensec.so.0(gensec_ntlmssp_server_auth+0xdf) [0x7f4666090e8f]
Jan 13 23:31:17 mail dovecot: auth: Error:  #7 /usr/lib64/libgensec.so.0(gensec_ntlmssp_update+0x253) [0x7f466608f413]
Jan 13 23:31:17 mail dovecot: auth: Error:  #8 /usr/lib64/libgensec.so.0(gensec_update+0xa) [0x7f4666093f2a]
Jan 13 23:31:17 mail dovecot: auth: Error:  #9 /usr/bin/ntlm_auth(+0x95c1) [0x7f4667c025c1]
Jan 13 23:31:17 mail dovecot: auth: Error:  #10 /usr/bin/ntlm_auth(+0x5b9f) [0x7f4667bfeb9f]
Jan 13 23:31:17 mail dovecot: auth: Error:  #11 /usr/bin/ntlm_auth(main+0xac3) [0x7f4667bfe3e3]
Jan 13 23:31:17 mail dovecot: auth: Error:  #12 /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f46648edd85]
Jan 13 23:31:17 mail dovecot: auth: Error:  #13 /usr/bin/ntlm_auth(+0x5639) [0x7f4667bfe639]
Jan 13 23:31:17 mail dovecot: auth: Error: Can not dump core: corepath not set up
Jan 13 23:32:04 mail sm-mta[4723]: t0E4W3hV004723: hplaptop.hprs.local [192.168.0.100] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Jan 13 23:32:04 mail sm-mta[4724]: t0E4W3MX004724: hplaptop.hprs.local [192.168.0.100] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Jan 13 23:32:04 mail sm-mta[4726]: t0E4W41Y004726: hplaptop.hprs.local [192.168.0.100] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Jan 13 23:32:15 mail sm-mta[4725]: t0E4W4Xd004725: hplaptop.hprs.local [192.168.0.100] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA

Open in new window

Then Outlook offered me a 'retry' option:
Jan 13 23:34:47 mail dovecot: imap-login: Warning: Auth process not responding, delayed sending initial response (greeting): user=<>, rip=192.168.0.100, lip=192.168.0.2, session=<//NnQpUMxQDAqABk>
Jan 13 23:34:47 mail dovecot: imap-login: Warning: Auth process not responding, delayed sending initial response (greeting): user=<>, rip=192.168.0.100, lip=192.168.0.2, session=<shJoQpUMxwDAqABk>
Jan 13 23:35:07 mail dovecot: imap-login: Error: Timeout waiting for handshake from auth server. my pid=6039, input bytes=0
Jan 13 23:35:07 mail dovecot: imap-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 30 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, session=<//NnQpUMxQDAqABk>
Jan 13 23:35:07 mail dovecot: imap-login: Error: Timeout waiting for handshake from auth server. my pid=6040, input bytes=0
Jan 13 23:35:07 mail dovecot: imap-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 30 secs): user=<>, rip=192.168.0.100, lip=192.168.0.2, session=<shJoQpUMxwDAqABk>
Jan 13 23:35:07 mail sm-mta[6699]: t0E4Z7pt006699: hplaptop.hprs.local [192.168.0.100] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Jan 13 23:35:07 mail sm-mta[6698]: t0E4Z7mL006698: hplaptop.hprs.local [192.168.0.100] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Jan 13 23:35:07 mail sm-mta[6701]: t0E4Z72T006701: hplaptop.hprs.local [192.168.0.100] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Jan 13 23:35:11 mail sm-mta[6700]: t0E4Z7sk006700: hplaptop.hprs.local [192.168.0.100] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA

Open in new window

Only remaining Outlook option is 'manually configure'.

Leaving it there until I get some feedback.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40548271
Double check your config  do you still have /usr/bin/auth_ntlm as a helper.
Add the plain methode for auth before NTLM.
On protocol, I think you are missing the imaps

Manually configure the exchange type ....

I think you may still have to look at adjusting the settings to have outlook use NTLM.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40549056
Double check your config  do you still have /usr/bin/auth_ntlm as a helper.
Yes, it shows in `doveconf -a`, but not in `doveconf -n`
Add the plain methode for auth before NTLM.
OK, will do.
On protocol, I think you are missing the imaps
Tried adding that but got the error:
doveconf: Warning: Obsolete setting in /usr/local/etc/dovecot/dovecot.conf:25: 'imaps' protocol is no longer necessary, remove it

Open in new window

I think you may still have to look at adjusting the settings to have outlook use NTLM.
Well, lines 10, 13, 16, 20, 23, 24, 27, 29 and 32 in the first /var/log/maillog list of my original post have lines of the form:
Jan 13 22:57:51 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 13 22:57:51 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=240
Jan 13 22:57:51 mail dovecot: auth: Error: GENSEC login failed: NT_STATUS_LOGON_FAILURE
Jan 13 22:57:51 mail dovecot: auth: ntlm(?,192.168.0.100,<topSvpQMdgDAqABk>): user not authenticated: NT_STATUS_LOGON_FAILURE

Open in new window

implying to me that it is trying ntlm.

I'll try suggested config changes and post back.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40549085
Your only auth method is NTLM, I do not see the domain in the AUTH parameter.

Run
ntlm_auth --username=mark --domain=HPRS
Provide password , do you get Success?
Now try the same with the --domain=
Do you get Success as well?
This deals with making sure NTLM with username/password will validate with and without the domain.

I think once you add plain to the available auth, you should be closer to the condition your system was before where the user using NTLM authenticated, but there was an access to the directory (note you reverted to using ~/Maildir instead of the last one /home/HPRS/%n/Maildir)
IF you get an error that ~/Maildir could not be interepted, only change the mail_location and nothing else and try again.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40549111
Double check which is correct on your system, auth_ntlm or ntlm_auth It could be as simple as using the wrong one as the cause of the issue that has lasted this long.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40549199
Results before your previous message ...

/var/log/maillog results pretty much the same, but after adding 'plain' to the auth_mechanisms I got messages:
Jan 14 10:53:19 mail dovecot: auth: Fatal: No passdbs specified in configuration file. PLAIN mechanism needs one

Open in new window

Since I don't have a passdb for ntlm I removed 'plain' and restarted dovecot. I selected manual configuration. Image manualconfig are the parameters I entered. I then clicked 'Test Account Settings'. I got the following messages in /var/log/maillog:
Jan 14 11:01:30 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 14 11:01:30 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=250
Jan 14 11:01:30 mail dovecot: auth: Error: GENSEC login failed: NT_STATUS_LOGON_FAILURE
Jan 14 11:01:30 mail dovecot: auth: ntlm(?,192.168.0.100,<zs1V2p4M4gDAqABk>): user not authenticated: NT_STATUS_LOGON_FAILURE

Open in new window

I was then prompted to enter a password. After doing so I got the following messages. Outlook results are in OutlookError image.
Jan 14 11:03:00 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 14 11:03:00 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=250
Jan 14 11:03:00 mail dovecot: auth: Error: NTLMSSP Sign/Seal - Initialising with flags:
Jan 14 11:03:00 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088205
Jan 14 11:03:00 mail dovecot: imap-login: Login: user=<mark@hprs>, method=NTLM, rip=192.168.0.100, lip=192.168.0.2, mpid=15880, session=<zs1V2p4M4gDAqABk>
Jan 14 11:03:00 mail dovecot: imap(mark@hprs): Error: user mark@hprs: Couldn't drop privileges: User is missing UID (see mail_uid setting)
Jan 14 11:03:00 mail dovecot: imap(mark@hprs): Error: Internal error occurred. Refer to server log for more information.
Jan 14 11:03:01 mail sm-mta[15881]: t0EG303Y015881: from=<mark@hprs.local>, size=389, class=0, nrcpts=1, msgid=<201501141603.t0EG303Y015881@mail.hprs.local>, proto=ESMTP, daemon=MTA, relay=hplaptop.hprs.local [192.168.0.100]
Jan 14 11:03:01 mail sm-mta[15895]: t0EG303Y015881: SYSERR(root): hprs.local. config error: mail loops back to me (MX problem?)
Jan 14 11:03:01 mail sm-mta[15900]: t0EG31DY015900: mail.hprs.local [192.168.0.2] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA
Jan 14 11:03:01 mail sm-mta[15895]: t0EG303Y015881: to=<mark@hprs.local>, delay=00:00:01, xdelay=00:00:00, mailer=esmtp, pri=120389, relay=hprs.local. [192.168.0.2], dsn=5.3.5, stat=Local configuration error
Jan 14 11:03:01 mail sm-mta[15895]: t0EG303Y015881: t0EG313Y015895: DSN: Local configuration error
Jan 14 11:03:01 mail sm-mta[15895]: t0EG313Y015895: to=<mark@hprs.local>, delay=00:00:00, xdelay=00:00:00, mailer=esmtp, pri=61702, relay=hprs.local., dsn=5.3.5, stat=Local configuration error
Jan 14 11:03:01 mail sm-mta[15895]: t0EG313Y015895: to=root, delay=00:00:00, xdelay=00:00:00, mailer=local, pri=61702, dsn=2.0.0, stat=Sent
Jan 14 11:03:01 mail sm-mta[15895]: t0EG313Y015895: t0EG313Z015895: return to sender: Local configuration error
Jan 14 11:03:01 mail sm-mta[15895]: t0EG313Z015895: to=root, delay=00:00:00, xdelay=00:00:00, mailer=local, pri=32726, dsn=2.0.0, stat=Sent

Open in new window

First few lines indicate (to me) that it is trying NTLM. The new message here is the one about " Error: user mark@hprs: Couldn't drop privileges: User is missing UID (see mail_uid setting)". I would think it could get the UID from ntlm_auth.

Your latest message ...
Your only auth method is NTLM, I do not see the domain in the AUTH parameter.
Yes, your're right. But it did find the domain at first. See line 80 in /var/log/maillog list in my initial post. Since I ended up removing 'plain' from auth_mechanisms and removing 'imaps' from protocols the dovecot config is back to the way I had it upon initial posting. The only thing different now is I'm continuing to a manual setup of Outlook. Maybe the auto-configuration tries other things later that do discover the domain, but the manual config has aborted by that point due to other issues.
ntlm_auth --username=mark --domain=HPRS
 Provide password , do you get Success?
$ ntlm_auth --username=mark --domain=HPRS
Password:
NT_STATUS_OK: Success (0x0)

Open in new window

Now try the same with the --domain=
 Do you get Success as well?
$ ntlm_auth --username=mark --domain=
Password:
NT_STATUS_OK: Success (0x0)

Open in new window

I think once you add plain to the available auth, you should be closer to the condition your system was before where the user using NTLM authenticated, ...
That worked before because I specified a passdb as shadow which involved creating the user mark as a local user in /etc/passwd. Since I'm now trying to authenticate without resort to a local user there is no passdb. Therefore I cannot use PLAIN as it requires a passdb.
(note you reverted to using ~/Maildir instead of the last one /home/HPRS/%n/Maildir)
Yes, it does appear that dovecot will create the directory structure. wbinfo gives /home/HPRS/%n/Maildir as the home directory, so I'm hoping dovecot will sort that out. So far, we haven't gotten to the place where it is trying to access the mail folder. Before I got a permission error (when I specified dovecot GID/UID). But no such error yet. If we get past the authentication issue and then have a problem creating the folder I'll hard-config the maildir, but for the moment I'm hoping dovecot figures out everything (user, domain, maildir, etc.). I could set the maildir for testing purposes, but again, I think I have problems ahead of that.

Do you think the "missing UID" is the current roadblock?
Double check which is correct on your system, auth_ntlm or ntlm_auth It could be as simple as using the wrong one as the cause of the issue that has lasted this long.
This system uses ntlm_auth. No auth_htlm on system. /user/bin/ntlm_auth is configured in dovecot.
manualconfig.jpg
outlookError.jpg
0
 
LVL 76

Expert Comment

by:arnold
ID: 40549291
Disable the secure password option.

rerun the wbinfo -i mark
Did it change on the UID/GID reported in the list versus the ownership of the files in
ls -l /home/HPRS/ | grep mark?
Did you try using the exchange account type versus the internet email?

NTLM_auth outputs only OK or no such user, or etc. wrong password.
Your nsswitch.conf has not been changed since the original question, correct?

passwd: compat
shadow: compat
group: compat

how about we test the comapt functionality, change your mark account shell to bash this is done by changing the /bin/false with /bin/bash

then see if you can use the mark account to login into your mail system without an error.
If you can, this would confirm that compat covers authentication.
If you get an error, try adding winbind after the compat and try again logging in with mark.
change compat to files compat winbind


What is your mail_uid setting?
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40550432
Did you try using the exchange account type versus the internet email?
I thought of trying that too. Did try. Didn't work (see image). This link shows ports used by Exchange, many of which reference LDAP. https://support.prolateral.com/index.php?/Knowledgebase/Article/View/179

Your other comments ...
Disable the secure password option.
 rerun the wbinfo -i mark
 Did it change on the UID/GID reported in the list versus the ownership of the files in
 ls -l /home/HPRS/ | grep mark?
Not sure what you mean. Do you mean to disable ntlm? Note that /home/HPRS/... does not yet exist. I'm hoping that dovecot creates this path when it connects ... like sendmail did when I sent a message. Then I'll be able to see the ownership. If dovecot cannot create the path then I expect to see a "files does not exit" error or a write permission error. I'm hesitant to create the path by hand because I might pick the wrong gid/uid. I think this is one of the problems from the previous test/message.
Your nsswitch.conf has not been changed since the original question, correct?

 passwd: compat
 shadow: compat
 group: compat
That is correct.
how about we test the comapt functionality, change your mark account shell to bash this is done by changing the /bin/false with /bin/bash
Keep in mind that I have no user mark in /etc/passwd. The `wbinfo -I` command retrieves that information from the domain controller and apparently formats the output in passwd format. As far as I know, there is no way to change the shell. This probably makes the rest of your suggested tests moot.
What is your mail_uid setting?
It is not set (in dovecot.conf). However, the dovecot wiki advised the creation of dovecot and dovenull users and groups. The dovecot process appears to be running as root while supporting processes [including ntlm_auth (!?)] are running as the dovecot user:
> ps -ef | grep dovecot
dovecot   3473     1  0 Jan13 ?        00:00:00 dovecot/anvil
root      3474     1  0 Jan13 ?        00:00:00 dovecot/log
dovecot   4067     1 99 Jan13 ?        21:29:58 dovecot/auth
dovecot   4069  4067  0 Jan13 ?        00:00:00 [ntlm_auth] <defunct>
root     14497     1  0 11:00 ?        00:00:00 /usr/local/sbin/dovecot
dovecot  14498 14497  0 11:00 ?        00:00:00 dovecot/anvil
root     14499 14497  0 11:00 ?        00:00:00 dovecot/log

Open in new window


I wonder if anyone has gotten ntlm working on Linux with Outlook? I've searched some and found several postings of people requesting help, but no actual solutions. I will search more. The following link looks useful, but I haven't read in detail. Perhaps this simply doesn't work? What do you think? Do you know of anyone who's got this mechanism working with Outlook?

http://support.microsoft.com/kb/976918
Exchange.jpg
0
 
LVL 76

Expert Comment

by:arnold
ID: 40550472
You have to create the /home/HQRS and one with lower case as a symbolic link.

sendmail will not create the directory i.e. email, mark and look at the sendmail log dealing with delivery attempt, procmail will create Maildir if missing but not the homedir, since dovecot is not configured as LDA, waiting for it to create homedirs IMHO is not advisable.
You should within procmail add the homedir creation if missing.

You have mark in AD with a current shell as /bin/false, please change this  entry to /bin/bash and see whether you can login (ssh into the mail server with the user mark and successfully login and get a shell.)
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40550594
You have to create the /home/HQRS and one with lower case as a symbolic link.

 sendmail will not create the directory i.e. email, mark and look at the sendmail log dealing with delivery attempt, procmail will create Maildir if missing but not the homedir, since dovecot is not configured as LDA, waiting for it to create homedirs IMHO is not advisable.
Yes, you're right. I'll create that now. Symbolic link is a good idea.
You have mark in AD with a current shell as /bin/false, please change this  entry to /bin/bash and see whether you can login
I have no idea how to do that. Do you?

Actually, RSAT shows completely different UID, shell and home dir that wbinfo does (see image). So I'm really confused.

Anyway, I tried logging in, even with the /bin/false and got the following error:
Jan 15 01:05:51 mail sshd[24716]: Invalid user mark from 192.168.0.100
Jan 15 01:05:51 mail sshd[24716]: input_userauth_request: invalid user mark [preauth]
Jan 15 01:05:55 mail sshd[24716]: Failed password for invalid user mark from 192.168.0.100 port 49890 ssh2

Open in new window

If I set a local user to /bin/false I get the following in the server message log:
Jan 15 01:09:06 mail sshd[26916]: Accepted password for mfoley from 96.11.168.98 port 48514 ssh2
Jan 15 01:09:13 mail sshd[27661]: Received disconnect from 96.11.168.98: 11: disconnected by user

Open in new window

and the following on the remote
$ ssh phonetree.ohprs.org
mfoley@phonetree.ohprs.org's password:
Last login: Wed Jan 14 14:48:26 2015 from 96.11.168.98
Linux 3.10.17.
Connection to phonetree.ohprs.org closed.

Open in new window

I would expect the same with the domain user with /bin/false, not the "Invalid user" response.
profile.jpg
0
 
LVL 76

Expert Comment

by:arnold
ID: 40550610
Now try having nsswitch.conf passwd, shadow, and group as
passwd: files compat winbind
shadow: files compat winbind
group: files compat winbind

and try again.  You do not need to connect from remote you can from within the shell do ssh mark@localhost and password, when works, you should get a shell.

The uid/gid homedir for unix is different than the settings that are included in the AD user profile for a windows based login.

This is similar to why windows AD DC have the Subsystem for Unix that adds the Unix/Linux based schema as well as the related components NIS server to ......

In your case, you may have the settings overriden by your winbind configuration within /etc/samba/smb.conf
 i.e. do you set idmap uid and gid settings?
template shell?
template home dir?

it seems you've discovered what is hidden on the third floor.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40550656
Made the suggested changes to nsswitch.conf. Tried `ssh mark@localhost`. Got "Permission denied, try again".

I don't think I have anything special in smb.conf. Here it is:
[global]
        workgroup = HPRS
        realm = hprs.local
        netbios name = MAIL
        interfaces = lo, eth1
        bind interfaces only = Yes
        server role = active directory domain controller
        server services = s3fs, rpc, nbt, wrepl, ldap, cldap, kdc, drepl, winbind, ntp_signd, kcc, dnsupdate
        idmap_ldb:use rfc2307 = yes

    winbind use default domain = yes

    load printers = no
    printing = bsd
    printcap name = /dev/null
    disable spoolss = yes

   log level = 3 passdb:5 auth:10 winbind:2
   max log size = 5000

[netlogon]
        path = /var/lib/samba/sysvol/hprs.local/scripts
        read only = No

[sysvol]
        path = /var/lib/samba/sysvol
        read only = No

[Users]
    path = /redirectedFolders/Users
    comment = user folders for redirection
    read only = No

Open in new window

I believe this was mostly auto-generated by the samba-tool provision task. Things added are your (or geist's) suggestion on "winbind use default domain" and I added the printer stuff to suppress error messages.

Also tried setting HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\LSA\SuppressExtendedProtection  DWORD to 1 as advertised by http://support.microsoft.com/kb/976918. This seemed very hopeful:
For Windows clients that support channel binding that are failing to be authenticated by non-Windows NTLM servers that do not handle the CBT correctly:• Set the registry entry value to "0x01." This will configure NTLM not to emit CBT tokens for unpatched applications.
For non-Windows NTLM servers or proxy servers that require LMv2:• Set to the registry entry value to “0x01.” This will configure NTLM to provide LMv2 responses.
but didn't help. Still getting error "Jan 15 01:48:24 mail dovecot: imap(mark@hprs): Error: user mark@hprs: Couldn't drop privileges: User is missing UID (see mail_uid setting)" among others.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40550662
in smb.conf try disabling (set idmap_ldb:use rfc2307 = no) the yes feature is needed only when the UNIX portion of AD schema is not present/available.

then restart smb/winbind and run wbinfo -i mark to see whether the UID/GID and home is what you have in the aduc interface.

To be clear, you only have the Samba4 setup as AD DC.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40550731
in smb.conf try disabling (set idmap_ldb:use rfc2307 = no)
did that. same wbinfo:

HPRS\mark:*:3000026:100:Mark Foley:/home/HPRS/mark:/bin/false

To be clear, you only have the Samba4 setup as AD DC.
Yes (as opposed to what? Samba shared drives? Don't have those). There is no other DC/AD in LAN.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40551113
we're smb services restarted after the change? Restart nscd as well

How about nsswitch/login into she'll with ad username/password?
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40551966
we're smb services restarted after the change? Restart nscd as well
Yes, but I'll reboot, just to make sure everything is started
How about nsswitch/login into she'll with ad username/password?
No, same "access denied, but I'll try again after reboot.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40552035
After reboot:
> su - mark
1 14:24:36 root@mail:~
> su - mark@localhost
No passwd entry for user 'mark@localhost'
1 14:24:45 root@mail:~
> su - HPRS\\mark
1 14:25:46 root@mail:~

Open in new window

No luck.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40552099
Went through manual config again in Outlook. Had to add mail_uid, mail_gid, first_first_valid_uid to get past errors. Also added:

service auth {
  user = root
}

per http://dovecot.org/pipermail/dovecot/2015-January/099261.html. Now getting errors:
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 15 14:44:24 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=186
Jan 15 14:44:24 mail dovecot: auth: Error: NTLMSSP Sign/Seal - Initialising with flags:
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088205
Jan 15 14:44:24 mail dovecot: imap-login: Login: user=<mark@hprs>, method=NTLM, rip=192.168.0.100, lip=192.168.0.2, mpid=28024, session=<EwhHFbYMqwDAqABk>
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: user mark@hprs: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: ~/Maildir
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: Invalid user settings. Refer to server log for more information.
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 15 14:44:24 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=186
Jan 15 14:44:24 mail dovecot: auth: Error: NTLMSSP Sign/Seal - Initialising with flags:
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088205
Jan 15 14:44:24 mail dovecot: imap-login: Login: user=<mark@hprs>, method=NTLM, rip=192.168.0.100, lip=192.168.0.2, mpid=28026, session=<I09HFbYMrADAqABk>
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: user mark@hprs: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: ~/Maildir
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: Invalid user settings. Refer to server log for more information.
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 15 14:44:24 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=186
Jan 15 14:44:24 mail dovecot: auth: Error: NTLMSSP Sign/Seal - Initialising with flags:
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088205
Jan 15 14:44:24 mail dovecot: imap-login: Login: user=<mark@hprs>, method=NTLM, rip=192.168.0.100, lip=192.168.0.2, mpid=28028, session=<VpxHFbYMrQDAqABk>
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: user mark@hprs: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: ~/Maildir
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: Invalid user settings. Refer to server log for more information.
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 15 14:44:24 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=186
Jan 15 14:44:24 mail dovecot: auth: Error: NTLMSSP Sign/Seal - Initialising with flags:
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088205
Jan 15 14:44:24 mail dovecot: imap-login: Login: user=<mark@hprs>, method=NTLM, rip=192.168.0.100, lip=192.168.0.2, mpid=28030, session=<zdhHFbYMrgDAqABk>
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: user mark@hprs: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: ~/Maildir
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: Invalid user settings. Refer to server log for more information.
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088207
Jan 15 14:44:24 mail dovecot: auth: Error: Got user=[mark] domain=[] workstation=[HPLAPTOP] len1=24 len2=186
Jan 15 14:44:24 mail dovecot: auth: Error: NTLMSSP Sign/Seal - Initialising with flags:
Jan 15 14:44:24 mail dovecot: auth: Error: Got NTLMSSP neg_flags=0xa2088205
Jan 15 14:44:24 mail dovecot: imap-login: Login: user=<mark@hprs>, method=NTLM, rip=192.168.0.100, lip=192.168.0.2, mpid=28032, session=<5BBIFbYMrwDAqABk>
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: user mark@hprs: Initialization failed: Initializing mail storage from mail_location setting failed: Home directory not set for user. Can't expand ~/ for mail root dir in: ~/Maildir
Jan 15 14:44:24 mail dovecot: imap(mark@hprs): Error: Invalid user settings. Refer to server log for more information.

Open in new window

It never does get the domain (not important?). The main error now seems to be "can't expand ~/". I've googled all over for this and found nothing so far. Will keep trying.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40552240
Ok, now you are back to where you were login successful, change to mail_location = maildir:/home/HPRS/%n/Maildir.

The domain based on smb.conf winbind domain yes setting. other wise, you may have to use hprs\username in outlook.

see how that works out.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40552248
not su, ssh mark@localhoist
su - mark should kick you right out unless you changed the shell, or the data from rsat is now taken effect versus the sbm/winbind idmap.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40552835
not su, ssh mark@localhoist
Yeah, tried that too, but got "Permission denied, please try again." Neither su or ssh seem to work for domain user mark.

Ok, now you are back to where you were login successful, change to mail_location = maildir:/home/HPRS/%n/Maildir.
Well, we're back to the problem we had at the end of my previous message:
Jan 16 00:14:21 mail dovecot: imap(mark@hprs): Error: user mark@hprs: Initialization failed: Initializing mail storage from mail_location setting failed: mkdir(/home/hprs/mark/Maildir) failed: Permission denied (euid=151(dovecot) egid=151(dovecot) missing +w perm: /home/hprs/mark, dir owned by 3000026:100 mode=0755)
Jan 16 00:14:21 mail dovecot: imap(mark@hprs): Error: Invalid user settings. Refer to server log for more information.

Open in new window

dovecot does not have permission to write to the home/hprs/mark folder. So, I changed home/hprs/mark to be group dovecot and made the folder group writable. Dovecot was then able to log it OK.

Next, I sent a message from an external host, but got the error:
Jan 16 00:32:33 mail sm-mta[31196]: t0G5WW59031196: from=<mfoley@server.novatec-inc.com>, size=1624, class=0, nrcpts=1, msgid=<201501160531.t0G5Vl7c032274@server.novatec-inc.com>, proto=ESMTP, daemon=MTA, relay=cdptpa-outbound-snat.email.rr.com [107.14.166.225]
Jan 16 00:32:33 mail sm-mta[31198]: t0G5WW59031196: forward /home/HPRS/mark/.forward.mail: Jan 16 00:32:33 mail sm-mta[31198]: t0G5WW59031196: forward /home/HPRS/mark/.forward: Group writable directoryGroup writable directory

Jan 16 00:32:35 mail sm-mta[31198]: t0G5WW59031196: to=<mark@phonetree.ohprs.org>, delay=00:00:02, xdelay=00:00:01, mailer=local, pri=31849, dsn=2.0.0, stat=Sent

Open in new window

Procmail (I suppose) did create the whole subordinate Maildir structure:
$ ls -lR .
.:
total 4
drwx------ 6 dovecot dovecot 4096 2015-01-16 00:35 Maildir/

./Maildir:
total 32
drwx------ 2 dovecot dovecot 4096 2015-01-16 00:22 cur/
-rw------- 1 dovecot dovecot   51 2015-01-16 00:25 dovecot-uidlist
-rw------- 1 dovecot dovecot    8 2015-01-16 00:35 dovecot-uidvalidity
-r--r--r-- 1 dovecot dovecot    0 2015-01-16 00:22 dovecot-uidvalidity.54b8a02a
-rw------- 1 dovecot dovecot  348 2015-01-16 00:25 dovecot.index.log
-rw------- 1 dovecot dovecot   24 2015-01-16 00:35 dovecot.mailbox.log
drwx------ 2 dovecot dovecot 4096 2015-01-16 00:22 new/
-rw------- 1 dovecot dovecot   12 2015-01-16 00:35 subscriptions
drwx------ 2 dovecot dovecot 4096 2015-01-16 00:22 tmp/

./Maildir/cur:
total 0

./Maildir/new:
total 0

./Maildir/tmp:
total 0

Open in new window

but would not actually deliver the mail apparenty due to the "Group writable directory" problem. You see that Maildir/new is empty. I'm not really sure which directory it's complaining about. Maildir and all subordinate are not group writable. Only /home/HPRS/mark is. I guess that must be the one it's having a problem with.

So, we have two big problems:

1. How to get around the "Group writable directory" issue.

2. Outlook is still making me log in each time it is run (see image). That rather defeats the purpose of using the AD Authentication. How do I get Outlook to auto-authenticate with ntlm?
outlookLogin.jpg
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40552897
Fixed the "Group writable directory" problem by adding the following to sendmail.cf
O DontBlameSendmail=ForwardFileInGroupWritableDirPath
[code]
Link explaining: http://etutorials.org/Server+Administration/Sendmail/Part+III+The+Configuration+File/Chapter+24.+The+O+Options+Configuration+Command/DontBlameSendmail/

Still had a procmail problem (Suspicious rcfile) so changed permission on .procmailrc to 0640, but still have the same problem:
[code]
Jan 16 01:52:13 mail procmail[31954]: Suspicious rcfile "/home/HPRS/mark/.procmailrc"
Jan 16 01:52:13 mail sm-mta[31953]: t0G6qCho031952: to=<mark@phonetree.ohprs.org>, delay=00:00:01, xdelay=00:00:01, mailer=local, pri=31850, dsn=2.0.0, stat=Sent

Open in new window

Don't know how to fix yet. Apparently, the directory containing .procmailrc cannot be group writable; http://pank.org/blog/archives/000642.html. But dovecot needs to be able to write this this directory! No mail is getting delivered nor are non-delivery messages going back to sender. Messages just disappears.

Moved .procmailrc to /etc/procmailrc

DEFAULT=$HOME/Maildir/

now no error in /etc/log/maillog:
Jan 16 02:12:54 mail sm-mta[16180]: t0G7CrbN016180: from=<mfoley@server.novatec-inc.com>, size=1625, class=0, nrcpts=1, msgid=<201501160712.t0G7C8P2023757@server.novatec-inc.com>, proto=ESMTP, daemon=MTA, relay=cdptpa-outbound-snat.email.rr.com [107.14.166.227]
Jan 16 02:12:54 mail sm-mta[16181]: t0G7CrbN016180: to=<mark@phonetree.ohprs.org>, delay=00:00:01, xdelay=00:00:00, mailer=local, pri=31850, dsn=2.0.0, stat=Sent

Open in new window

Mail not going to Maildir. In fact, all missing messages went to /var/spool/mail/HPRS\mark in mbox format. arrrgggg!

I think the problem is that procmail's notion of the user is 'HPRS\mark', not 'mark' and it therefore can't find $HOME (yet it did find the .procmailrc file in /home/HPRS/mark ... so maybe that theory is wrong)
0
 
LVL 76

Expert Comment

by:arnold
ID: 40553343
Send mail views non owner write able permissions on the homedir /home/HPRS/mark as a security violation.

The account style setup in outlook without saving the password will ask you once per session.

Your Dovecot auth process is running as Dovecot, while it should be running as root.

http://wiki2.dovecot.org/RunningDovecot
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40554106
Send mail views non owner write able permissions on the homedir /home/HPRS/mark as a security violation.
I fixed the sendmail issue with that "O DontBlameSendmail=ForwardFileInGroupWritableDirPath" option. Now the problem is procmail and I haven't found a similar setting bypass the "Suspicious rcfile" for procmail.
The account style setup in outlook without saving the password will ask you once per session.
Well, actually entering the password is not what I want since I want it to validate using AD. If I have to enter the password in Outlook the user will have to re-enter that password each time he/she changes his/her domain password.
Your Dovecot auth process is running as Dovecot, while it should be running as root.
Actually, it is running as root now. That was the tidbit I picked from the dovecot mailing list archive:

service auth {
  user = root
}
$ ps -ef | grep dov
root     15794 22733  0 00:13 ?        00:00:00 dovecot/log
dovecot  17001 22733  0 02:13 ?        00:00:00 dovecot/imap
root     22733     1  0 Jan15 ?        00:00:00 /usr/local/sbin/dovecot
dovecot  22734 22733  0 Jan15 ?        00:00:00 dovecot/anvil
root     31264 22733  0 13:37 ?        00:00:00 dovecot/config
root     31265 22733  0 13:37 ?        00:00:00 dovecot/auth

Open in new window

Now, why I had to set this specifically, and why my ps listing is different from the one in your referenced link is another question. Why would the wiki example start the auth task as root, but my implementation did not? Why do they have tasks running as dovenull and I do not?
0
Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

 
LVL 1

Author Comment

by:jmarkfoley
ID: 40554186
OK, time to punt. This project is not a mere hobby. With Microsoft dropping the Small Business Server product and not supporting Exchange on their SBS replacement Server Essentials, my office is looking for a replacement for our aging SBS 2008 host. The Samba4 DC/AD works beautifully, including redirected folders. I have been trying to find a replacement for Exchange for 5 months now, including OpenChange/SOGo.

I have to conclude that dovecot ntlm does not work.

I've found many posts from people asking for help getting ntml working, but no solutions posted. I've posted on several forums asking if anyone has accomplished this. No responses. Although you clearly have extensive knowledge in the related technologies in Linux, you have not actually set this up yourself before. I am struggling to get on the dovecot mailing list, which is a challenge all by itself. Their archived message are difficult to navigate. The dovecot documentation is lacking (as they admit on their home page) and the wiki docs, though nicely formatted, are sparse. I find dozens of arcane settings for dovecot all over the net not mentioned in the wiki (like the 'service auth' one I mentioned). One would think that with a totally out-of-the-box vanilla setup like mine there would be a basic doveconf for ntlm that just worked. But such is not to be found.

You and I have spent the better part of two weeks on this issue. I need to move on because there are lots more things I have to figure out before we have a SBS replacement (webmail, shared calendars, remote access, ...) and time is getting short.

I'm going to leave this question open and try to revisit it with new experiments to keep it from going "neglected" on EE, and continue trying to get on the dovecot mailing list.

What I am going to do it go back to my initial PLAIN mechanism, create these domain users in /etc/passwd and authenticate with shadow. All that worked flawlessly by the end of my original question http://www.experts-exchange.com/OS/Linux/Q_28591277.html. The only downside is that users will have to have a separate, non AD password for their mail. As that will be a one-time setup, I don't think it will be too tragic in the scheme of things. After getting the fundamentals of the SBS replacement going I'll have plenty of leisure to experiment with ntlm, ldap, or whatever.

I will post back here with a link to my eventual new EE question related to this. Of course, if you have any more suggestions on ntlm, please feel free to post.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40554365
In your example, you already have a functional AD schema that includes Exchange references.
I think you started with going from the begining (building a new building)
An alternative approach deals with maintining the AD structure your existing SBS provides when migrating to the newer system is deployed with the caveat that the mail server that will be used is now sednmail/or any other usin/linux using linux AD integration. (renovate the existing fourth floor

This way Outlook will have access to the AD/LDAP data that it is looking for to mimic the exchange style account.......

I've not dug into the outlook/exchange replacement to maintain a portion while going with newer.

often, when migrating due to abandoned products, or cost, ......

defining what the end result is and then assembling the opensource or commercial products to replace/accomplish it.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40554754
An alternative approach deals with maintining the AD structure your existing SBS provides
This is what I'm trying to do, though I'm not sure where you're headed with this ...
... defining what the end result is
My end objective is straightforward: replace SBS functionality with a Linux solution the result of which is essentially transparent to the end users. So far the DC/AD portion works fine. My current struggle is with email. The system I am testing this with is an isolated test network, not connected to the existing production SBS 2008 domain.

My test platform is simple, no funny stuff. I have Samba4 4.1.11 deployed as the DC/AD all I did was provision it with
samba-tool domain provision --use-rfc2307 --server-role='dc' \
  --realm=hprs.local --domain=HPRS --adminpass='cE!5ZUeL' --dns-backend=BIND9_FLATFILE \
  --option="interfaces=lo eth1" --option="bind interfaces only=yes"

Open in new window

That basically did the trick. I had to do a bit of tweaking on bind because I'm not using samba's bind, and I followed some Internet posts on creating a group policy for Remote Desktop Connection and redirected folders (using Microsoft's RSAT). Otherwise, no special tweaks. The provision process created the /etc/samba/smb.conf which I minorly modified to disable printspool error messages and set the log level. This is a toally vanilla Samba4 DC/AD!

Samba 4 has been out there since December 2012. I have to believe that somewhere on the planet someone has managed to get Outlook working with it, yet after 2 weeks of searching I have not found anyone with a lucid solution (perhaps poor searching skills on my part), nor have a found a step-by-step tutorial that works, nor have you and I managed to get this working yet. I've posted to EE, LinuxQuestions, ServerFault and other forums and have zero response aside from you. This may be the most frustrating project I've ever worked on in my career!

Rant aside, I've run into a new wrinkle. I put all the dovecot configs back to the PLAIN setup we had working 6 days ago. I removed all the related user folders and create /home/HPRS fresh. I then attempted to create local user mark as:
$ useradd -c "Domain user HPRS\\mark" -d /home/HPRS/mark -m -g 200 -u 1100 -s /bin/bash mark                   useradd: user 'mark' already exists

Open in new window

What ?! Already exists!? It is certainly NOT in /etc/passwd, yet my backup from the 10th shows:
:
pop:x:90:90:POP:/:/bin/false
nobody:x:99:99:nobody:/:/bin/false
mfoley:x:1000:100:Mark Foley:/home/mfoley:/bin/bash
mark:x:1100:200:Mark Foley:/domainusers/mark:/bin/bash
dovenull:x:150:150:dovecot untrusted IMAP login user:/:/bin/false
dovecot:x:151:151:dovecot user:/:/bin/false

Open in new window

Stumped again. Nevertheless, procmail does deliver mail OK, so the PLAIN solution seems to work without having to create an actual /etc/passwd entry. So, even though I'm totally confused as to why I could create this user in /etc/passd 6 days ago and can't today, procmail does seem to deliver OK. I did have to change the .procmailrc:
DEFAULT=/home/HPRS/mark/Maildir/
# DEFAULT= $HOME/Maildir/     # not this

Open in new window

procmail apparently can't figure out what $HOME is for some reason and creates the Maildir as a mbox file, not a Maildir folder. The hard-coded home does work.

Next challenge is to retrieve this mail with Outlook/dovecot. Will dovecot find the right userdb, passdb. Stay tooned ....
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40554765
As suspected, problems with retrieving via dovecot. I got the messages:
Jan 16 22:32:08 mail dovecot: auth-worker(27458): shadow(mark,192.168.0.100): unknown user (given password: glacon_9)
Jan 16 22:32:10 mail dovecot: auth-worker(27458): shadow(mark@hprs.local,192.168.0.100): unknown user (given password: glacon_9)

Open in new window

Of course, this makes sense. Since mark is not in /etc/passwd or /etc/shadow dovecot cannot retrieve user information from there. I was hoping dovecot would be as clever as procmail which was apparently able to find the user.

So, I removed the userdb and passdb settings from dovecot and enabled "auth_use_winbind = yes" hoping it might use winbind to get user/PW info. But no:
Jan 16 22:38:23 mail dovecot: auth: Fatal: No passdbs specified in configuration file. PLAIN mechanism needs one

Open in new window

So, I can't create a user in /etc/passwd because it "already" exists. Therefore using userdb/passdb in dovecot won't work. But if I remove the userdb/passdb dovecot still won't work because the PLAIN mechanism requires it!

I could work around this by creating different users in /etc/passwd such as mark2, and passing mark2 from Outlook to dovecot. That would probably work, but such a kludge!

Any suggestions before I look for my Russian Roulette kit?
0
 
LVL 76

Expert Comment

by:arnold
ID: 40554810
nsswitch.conf is in what state? compat, or files compat winbind?

When you tested plain before, you had mark as a local (/etc/passwd) account.

Your user add error, might be the result of using -m flag

If it still errors, see if changing nsswitch.conf back to compat resolves.  .....

procmail does not perform any lookups, sendmail sets the parameters before handing the message to procmail for final disposition.

What I meant with the transition to samba4/DC dealt with getting the current AD schema (samba4/DC as a replica) where there is the Exchange/Mailbox schema. Versus you starting with a brand new/empty schema that does not include the exchange part. export the current LDIF from the SBS, and import it on the samba4/
......

This way you will have the functionality you want, i.e. the section that Outlook will be looking for within the LDAP query when exchange type of account is setup.

While having sendmail/procmail delivering the messages based and handling the email.

I've searched and found before the "schema" update to add the exchange component to a linux/unix ldap setup.  

what modes (passdb do you have defined?)
if pam is one, then the samba/ldap/winbind should have gotten queried.
http://wiki2.dovecot.org/PasswordDatabase
0
 
LVL 76

Expert Comment

by:arnold
ID: 40554815
Look at openldap (I know you have samba4/ldap) but the exchange schema that you need to add to your existing setup, is easily found using openldap extend schema MS exchange.

MS publishes the changes in the exchange schema.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40555348
nsswitch.conf is in what state? compat, or files compat winbind?
Ah! True! I changed passwd, group and shadow to 'files comap winbind', whereas it was only compat before. Brilliant! Good memory. I changed it back and voila! I was able to create the account! I had followed up on my idea about creating an alternate email user account in /etc/passwd and that worked just fine, but this way the users can use their existing AD account, just have a different password for email. No big deal, really. That's something they can set once and forget about. So, PLAIN is back to working just fine. Can send and receive email from Outlook. Mail resides in $HOME/Maildir/ in maildir format.

procmail does not perform any lookups, sendmail sets the parameters before handing the message to procmail for final disposition.
But my understanding is that sendmail uses procmail for local delivery and procmail determines the delivery location and format (using procmairc file(s)). https://wiki.debian.org/MaildirConfiguration
If procmail can not deliver to the DEFAULT location, the mail ends up in the location specified in the ORGMAIL variable.
If procmail has problems figuring out format, it defaults to delivering in mbox format.

export the current LDIF from the SBS, and import it on the samba4/ ... I've searched and found before the "schema" update to add the exchange component to a linux/unix ldap setup.
That's worth looking into and I'll post back to this question will results (or questions if I can't figure out how to export LDIF).

what modes (passdb do you have defined?)
 if pam is one, then the samba/ldap/winbind should have gotten queried.
My current doveconf -n is:
$ doveconf -n
# 2.2.15: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 3.10.17 x86_64 Slackware 14.1
auth_verbose = yes
auth_verbose_passwords = plain
disable_plaintext_auth = no
mail_location = maildir:~/Maildir
namespace inbox {
  inbox = yes
  location =
  prefix =
}
passdb {
  driver = shadow
}
protocols = imap
ssl = no
userdb {
  driver = passwd
}

Open in new window

so the specific answer to your question is 'shadow'. Yes, I will continue to experiment with ntlm, ldap, pam, and whatever, but we have toyed with these (possibly not pam) over the past couple of weeks and come up with frustration. As I mentioned, I'm going to go ahead and continue with the PLAIN mode for the expediency of this project. The users will only be mildly impacted. Meanwhile I'll continue researching these modes and the LDIF aspect you mentioned.

Look at openldap (I know you have samba4/ldap) but the exchange schema that you need to add to your existing setup, is easily found using openldap extend schema MS exchange.

My understanding is that Samba4 cannot use OpenLDAP, "LDAP: no, you can't use OpenLDAP as a backend. And no, you can't configure Samba4 for listening on a different port than 389" [http://dev.nethserver.org/projects/nethserver/wiki/Samba4]. I've found others who have had problems trying, one example: http://dev.nethserver.org/projects/nethserver/wiki/Samba4.

Perhaps this is another thing to defer to on-going experimentation.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40555382
Starting from the end, openldap is an over encompassing/underlying.  The reason for the reference to openldap deals with locating the LDIF portion of the ms exchange schema that you would need to add to your Smaba4/LDAP AD-DC schema to allow outlook to query for the exchange section to get the information so that it could function as it is without user addition of passwords, etc.

Your current schema includes standard AD authentication/user management + unix portion. REf. your RSAT image.

You need the Exchange Mailbox portion of the RSAT (compare your RSAT display with samba4/LDAP to your SBS RSAT tabs)
once you have the MS exchange TAB, you would be in a better position to use the exchange type of service, pointing to the mail/LDAP port of samba4/ad-dc and then seeing whether the outlook/Samba4/LDAP/sendmail/or any other mail provides the user transparent way ........ you seek.

only using compat, means that winbind is not queried by the system tools which may explain other things.

While procmail is the local delivery agent, it is as part of the process start up indicated who the user to whom the message is address.
/etc/procmailrc is the default system config where certain variables are defined during compilation. procmail -v will reveal what the default location of email is.
It is often /var/spool/mail/$username
You altered the DEFAULT, but when it failed, because you did not also alter the ORGMAIL, it fell back on that setting and delivered the message, if you alter the ORGMAIL it will defer the delivery attempt. report to sendmail that there was an error. sendmail will requeue the message for later delivery.

The mechanism is that an email goes through is:
1) message delivered to sendmail (recipient, sender)
2) sendmail determines whether the recipient is local or remote deals with handling.
3) local sendmail sets the USERNAME, HOME and starts procmail passing the email message on STDIN.
4) Procmail ........



Here is another consideration, add samba4/ldap AD-replica.  Look at what the process of converting samba4/AD-replica into the samba4/AD-DC (SBS retired or if you .....)

This way the schema you have now with exchange can be configured/managed through the use of LDAP.conf on the sendmail to query the correct section for its configuration given currently you're sendmail configuration is separate from the ....
i.e. local domain is defined and any username is presumed local.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40564946
Believe it or not, I'm still struggling to get access to the dovecot mailing list. I've posted another request today. I really would like to get this working with ntlm, so I'll keep persisting.

As I wrote before, this is the hand-down most frustrating mini-project I've ever encountered (excluding the aborted attempt at OpenChange)! Seems like it should be really simple and that thousands of people have done it, but I'm at a loss to find such a person.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40565042
I think as the useradd command proved, the compat option does not query the samba/ldap setup.
i.e. with compat, you were able to add the local passwd user mark, while not being able to do so when the files compat winbind were listed in nsswitch.conf for passwd, shadow, group.

In your case, the NTLM has a different meaning that what you would like the interaction between outlook and dovecot since you manually add the username/password within the account creation. Often NTLM host to host communication deals with exchanging tokens that each can resolve to authenticate the user.

Your mini-project goal is to have this linux mail server/samba/ldap mimic SBS/Exchange functionality such that Outlook account creation actually directly queries the samba/ldap for the malibox data and go that route.

In another question on EE (http://www.experts-exchange.com/Software/Server_Software/Q_28601484.html) dealing with a Data Server where a windows server is not available, Rindi, suggested the use of zentyal www.zentyal.com which on their site claims to include the features you want (community).

You might set it up on an older workstation for purposes of a test to see whether the advertised feature match your needs.


Once you successfully conclude this project I'd suggest you write an article on the process here.

Did you look at the schema modification of your existing samba/ldap to include exchange's schema? At which point, you would open the LDAP port 389 for outlook to connect to and query as an Exchange service account to see whether that gets you further along the path.
If so, a GPO setting user outlook account creation rule will finish it.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40565338
I have explored Zentyl, but found that we have too many little nuances in DNS and AD GPO's to make that a useable solution for us. Zentyl is basically a Linux configuration integrating OpenChange. I do have to congratulate the Zentyl people for managing to get that all working. I tried for months to install/configure OpenChange and finally gave up. Getting that running (which I never did) involved installing and configuring a dozen other packages and the whole thing crumbles when a new version of one of the packages is released, but the downstream people haven't integrated (e.g. SOGo and OpenChange) ... I might have been able to get it running eventually, but the mess would have been more complicated that setting up a virtual machine running MS Server Essentials, MS Server 2012 and Exchange; and too complicated for anyone to step into my place and take over. By contract, DC/AD functionality was a snap with Samba4 -- just missing exact Exchange functionality.

So then, perhaps I am barking up the wrong tree with ntlm. The webmail client I chosen, roundCube, advertises that it supports LDAP Authentication (http://code.google.com/p/rcldapauth/), and "LDAP directory integration for address books". Maybe I should revisit LDAP Authentication and forget about ntlm? We started down that road once, but we (or maybe *I*) abandoned that effort in favor of ntlm.

I will look at the Exchange Scheme you mentioned and see if that throws any light on the matter.

What do you think about revisiting LDAP?
0
 
LVL 76

Expert Comment

by:arnold
ID: 40565636
LDAP address book would require LDAP schema changes, I have not used, so not sure what to make of it or how it might behave. Ldap auth I think simply means that the backend if the web based application will be querying the LDAP portion via a direct connection instead of using the nsswitch.con to authenticate the user whose credentials would either be provided in plain text or NTLM ..........


As you complete setting up each component, the logs help in pointing out what might be going wrong.

When possible build on the setup you have.  I.e. The web server access to the mail would go through Dovecot.
It makes it simpler to trouble shoot issues when the sequence of access is known.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40567772
Sorry, sidetracked with remote desktop issues, as you well know since you've been responding ...

So, are you suggesting I continue pursing the NTLM approach? The logs haven't helped me much so far. I finally did get on the dovecot mailing list and was considering asking the group if anyone has gotten NTLM working and if so, what are the configs.

I mention LDAP not so much because of address books (later), but because I'd like some kind of AD authentication so the users don't have to have 2 passwords: one for domain login and one for email. I was thinking LDAP because I just haven't gotten anywhere with NTLM.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40567856
To make outlook interact with your setup as if it were an exchange, would require your samba/ldap to include the exchange schema. Then configure ldap access to allow queries by users of outlook.
I've not had this, so once you have the schema in samba/ldap and outlook account to access mail.yourdomain.

Then first thing make sure outlook does not through errors on ldap access, and can get whatever info it needs. Then looking ldap, dovecot, sendmail logs to see when outlook attempts to access the emails should it error out.  If it gets to the email, you're set.

Consider exporting your SBS's AD LDIF possibly just the exchange stack.
Backup the existing samba/ldap
See if if the current SBS's AD schema can be imported, and go this route towards your goal of having a setup using samba/ldap/sendmail/dovecot as an SBS replacement.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40571866
OK, will attempt to follow these instruction and post back.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40607214
I'm going to move on and close this question. Will post new question later when I get some kind of breakthrough. I've scoured the web for postings on Dovecot and NTLM and have found nothing of any use. I finally was able to register on the dovecot maillist and I posted a message, but no response. I downloaded the entire dovecot.mbox from the maillist having something like 91,000 messages going back to 2002, but my message is not among them and it looks like the moderator has not posted my message yet ... if there is even anyone maintaining this list! I've written to the moderator for explanation. Meanwhile, there are 170 messages in this mbox with ntml in the subject, so I'll check those and see if they shed any light.

Otherwise, it looks like few people, if any, have actually gotten this working. There ought to be more fingerprints throughout the web.

We will be switching from SBS 2008/Exchange to the new Samba4 server tomorrow. You wrote, "export the current LDIF from the SBS, and import it on the samba4/". I'd like to do that before mothballing that machine, but I don't really know how export the current LDIF. Can you advise?
0
 
LVL 76

Expert Comment

by:arnold
ID: 40607292
I understand, though I wish to emphasize that NTLM has two meanings in this context.

Email client (outlook) to Dovecot authentication does not support NTLM, not part of the construct of outlook.
Dovecot to samba4/ldap using NTLM to authenticate the user works.

To achieve the goal of having outlook auto configure and authenticate without user intervention, the setup of samba4/lDAP AD DC, must have the schema mimicking/replicating the AD schema whee the exchange portion is included. And then outlook would directly access. The samba4/LDAP ........
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40607339
Email client (outlook) to Dovecot authentication does not support NTLM, not part of the construct of outlook.
Hmmm, I thought that's what Outlooks' "Secure Password Authentication" was all about. In any case it would be nice if someone at Dovecot would explicitly say so! I'm wondering if their enthusiasm for maintaining a free product has fallen off after 12 years.
To achieve the goal of having outlook auto configure and authenticate without user intervention ... outlook would directly access. The samba4/LDAP ...
Well, I'm trying to wrap my head around the fundamentals of LDAP, but having little success. I posted a question about this which perhaps you've seen: http://www.experts-exchange.com/OS/Linux/Q_28614062.html
As usual with my LDAP posts on a variety of sites, I get one response, then no follow-up to my subsequent comment. I get the feeing that posters haven't actually used LDAP, but have found some link somewhere on the web they paste into a response in hopes of getting points :( Well, we'll see how that goes.

Thanks again for you help, patience and time on this.
0
 
LVL 76

Assisted Solution

by:arnold
arnold earned 500 total points
ID: 40607422
Secure password deals with using a one way encrypted password.

NTLM functions when all that is needed is authentication/authorization. i.e. an internal web site where you want to know which user accessed but do not want to have the user type in their username/password.  When supported you would use NTLM.
When the user using IE accesses the site, it is prompted for credentials. The browser then provides the loged in user's token/credentials the webserver than validates the token/credentials against the AD, if granted, the user is "authenticated/authorized).

In the case of outlook, there is a significant amount of additional information email address, name, where to connect to send, where to connect to receive, type of incoming, etc. where a username/password or token/credentials will not provide.
0
 
LVL 1

Accepted Solution

by:
jmarkfoley earned 0 total points
ID: 40644814
Not resolved. Abandoning idea for now
0
 
LVL 76

Expert Comment

by:arnold
ID: 40644900
Mark, have you exported the LDIF from your SBS AD in particular the section dealing with exchange?
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40645009
Arnold: Don't know how to do that really. If you want to post a howto, I could try. We couldn't wait on this resolution and converted from that machine over the weekend of Feb 14th. It is now mothballed, but I could fire it up.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40645907
See if the following helps. Red ldfide.
http://support.microsoft.com/kb/555636

I'll try to find the correct reference to only extract the exchange schema.
0
 
LVL 1

Author Comment

by:jmarkfoley
ID: 40645916
OK, thanks. I'll post a new question if this yields something profitable.
0
 
LVL 76

Expert Comment

by:arnold
ID: 40645958
Searching for exporting AD schema returns many options, including sites offering somewhat similar ... To EE, the tool referenced above with the reference.

ldifde -f SavSchema.ldif -d CN=Schema,CN=Configuration,DC=HPRS,
DC=Local
0
 
LVL 1

Author Closing Comment

by:jmarkfoley
ID: 40653426
Haven't been able to get NTLM working with Outlook/Dovecot. Will explore again someday
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

New-MailboxSearch Powershell Command and step by step approach to Search and Extract Emails form Exchange 2013 Journaling server.
Email signatures have numerous marketing benefits. Here are 8 top reasons to turn your email signature into a marketing channel.
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now