Dear group,

I've been beating my head against this one for a week or so. Any
clues gratefully received.

I've got a couple of SMTP servers configured to act as SMTP routers
and spam filters for our new company mail system:

# sendmail -d0.1
Version 8.14.1
Compiled with: DNSMAP LDAPMAP LOG MAP_REGEX MATCHGECOS MILTER MIME7TO8
MIME8TO7 NAMED_BIND NETINET NETINET6 NETUNIX NEWDB
PICKY_HELO_CHECK PIPELINING SASLv2 SCANF SOCKETMAP STARTTLS
TCPWRAPPERS USERDB USE_LDAP_INIT XDEBUG

============ SYSTEM IDENTITY (after readcf) ============
(short domain name) $w = smtp0
(canonical domain name) $j = smtp0.thebunker.net
(subdomain name) $m = thebunker.net
(node name) $k = livid.thebunker.net
================================================== ======

Recipient names must be specified

This is on FreeBSD 6.2-RELEASE-p7 using sendmail from ports.

Everything is working well, *except* for SMTP AUTH. User account
information, virthosts, aliases etc. all come out of LDAP, and that's
working fine: I can send e-mail to the system happily, and from the
system so long as it's from a trusted relay that doesn't have to
authenticate.

The authentication data should all come out of the same LDAP database.
I'm happy that LDAP is working correctly, as I've got Cyrus IMAPd
authenticating against it, as well as using it for HTTP basic auth in
Apache.

However sendmail refuses to play ball. Using smtptest from Cyrus
IMAPd I see this:

% smtptest -m DIGEST-MD5 -a matthew -p 587 livid.thebunker.net
S: 220 smtp.thebunker.net ESMTP Sendmail 8.14.1/8.14.1; Sat, 1 Sep 2007 22:36:44 +0100 (BST)
C: EHLO example.com
S: 250-smtp.thebunker.net Hello splenetic.thebunker.net [213.129.65.22], pleased to meet you
S: 250-ENHANCEDSTATUSCODES
S: 250-PIPELINING
S: 250-8BITMIME
S: 250-SIZE
S: 250-DSN
S: 250-AUTH DIGEST-MD5 CRAM-MD5
S: 250-STARTTLS
S: 250-DELIVERBY
S: 250
C: AUTH DIGEST-MD5
S: 334 bm9uY2U9IlhIVXgwNVk3OC9wTFk4NnBpVDJSaFQxd3cvODUwdW h6WGM3YVNsK0JYV1E9IixyZWFsbT0ic210cC50aGVidW5rZXIu bmV0Iixxb3A9ImF1dGgsYXV0aC1pbnQsYXV0aC1jb25mIixjaX BoZXI9InJjNC00MCxyYzQtNTYscmM0LGRlcywzZGVzIixtYXhi dWY9ODE5MixjaGFyc2V0PXV0Zi04LGFsZ29yaXRobT1tZDUtc2 Vzcw==
Please enter your password:
C: dXNlcm5hbWU9Im1hdHRoZXciLHJlYWxtPSJzbXRwLnRoZWJ1bm tlci5uZXQiLG5vbmNlPSJYSFV4MDVZNzgvcExZODZwaVQyUmhU MXd3Lzg1MHVoelhjN2FTbCtCWFdRPSIsY25vbmNlPSI5QXZzZU tRZ1pYZVNBcWYvd21GSUpNMktCNmQ3eDU3RXZFT2VFMFl6NStZ PSIsbmM9MDAwMDAwMDEscW9wPWF1dGgtY29uZixjaXBoZXI9cm M0LG1heGJ1Zj0xMDI0LGRpZ2VzdC11cmk9InNtdHAvbGl2aWQu dGhlYnVua2VyLm5ldCIscmVzcG9uc2U9NWQ3YjRmYjI1YmRkNm JkYTJjOWEwMzZmMjJkNDVkMmM=
S: 535 5.7.0 authentication failed
Authentication failed. generic failure
Security strength factor: 128
C: QUIT
221 2.0.0 smtp.thebunker.net closing connection
Connection closed.

Authentication related config file stuff is pretty standard:

TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5')dnl
define(`confAUTH_REALM', `thebunker.net')dnl
define(`confAUTH_MECHANISMS', `EXTERNAL DIGEST-MD5 CRAM-MD5')dnl

I've tried setting up SASL to use:

pwcheck_method: auxprop
auxprop_plugin: ldapdb
[...]

but no dice. That gives me the following in auth.log:

Sep 1 18:31:09 livid sm-mta[49959]: Unexpectedly missing a prompt result
Sep 1 18:31:09 livid sm-mta[49959]: no secret in database

So I also tried to use 'pwcheck_method: saslauthd' (which would be
acceptable although clearly it means the SASL EXTERNAL mechanism won't
be available)

I can do test auth against saslauthd just fine:

# testsaslauthd -s sendmail -u matthew -p XXXCENSOREDXXX
0: OK "Success."

However trying the same smtptest results in the same 'no secret in
database' error as with the ldapdb auxprop_plugin, plus I cannot
detect that saslauthd is querying LDAP at all during this process.

I'm missing something obvious somewhere, but I've run out of ideas as
to what that might be. Any clues would be gratefully received,
especially any debug flags for sendmail that will make it trace out
exactly what it is doing during the authentication process.

Cheers,

Matthew

--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
Kent, CT11 9PW

On Sep 2, 5:01 am, Matthew Seaman wrote:

This has not much to do with sendmail, it is a cyrus-sasl problem,
but...

[snip]
> Everything is working well, *except* for SMTP AUTH. User account
> information, virthosts, aliases etc. all come out of LDAP, and that's
> working fine: I can send e-mail to the system happily, and from the
> system so long as it's from a trusted relay that doesn't have to
> authenticate.
>
> The authentication data should all come out of the same LDAP database.
> I'm happy that LDAP is working correctly, as I've got Cyrus IMAPd
> authenticating against it, as well as using it for HTTP basic auth in
> Apache.
>
> However sendmail refuses to play ball.

Not true, sendmail asks whatever mechanism you configured and the
error messages you get are from that mechanism.

[snip]
> Authentication related config file stuff is pretty standard:
>
> TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5')dnl
> define(`confAUTH_REALM', `thebunker.net')dnl
> define(`confAUTH_MECHANISMS', `EXTERNAL DIGEST-MD5 CRAM-MD5')dnl
>
> I've tried setting up SASL to use:
>
> pwcheck_method: auxprop
> auxprop_plugin: ldapdb
> [...]
>
> but no dice. That gives me the following in auth.log:
>
> Sep 1 18:31:09 livid sm-mta[49959]: Unexpectedly missing a prompt result
> Sep 1 18:31:09 livid sm-mta[49959]: no secret in database

That message means the user is not in the database. Which database?
that depends...

> So I also tried to use 'pwcheck_method: saslauthd' (which would be
> acceptable although clearly it means the SASL EXTERNAL mechanism won't
> be available)

But in this case you are expecting LDAP to work? saslauthd uses what
it was told to use on the parameter used to start it, did you check
the "-a ldap" parameter? and is /usr/local/etc/saslauthd.conf set
correctly?

[snip]
> However trying the same smtptest results in the same 'no secret in
> database' error as with the ldapdb auxprop_plugin, plus I cannot
> detect that saslauthd is querying LDAP at all during this process.

My guess is that saslauthd is using its own database, in /etc/sasldb2,
but to be certain you have to check both build parameters and the
options used to start saslauthd.

> I'm missing something obvious somewhere, but I've run out of ideas as
> to what that might be. Any clues would be gratefully received,
> especially any debug flags for sendmail that will make it trace out
> exactly what it is doing during the authentication process.

Sasl is pretty hard to debug, be patient.
--
René Berber

René Berber <rberber@mailandnews.com> writes:

> On Sep 2, 5:01 am, Matthew Seaman wrote:
>
> This has not much to do with sendmail, it is a cyrus-sasl problem,
> but...
>
> [snip]
>> Everything is working well, *except* for SMTP AUTH. User account
>> information, virthosts, aliases etc. all come out of LDAP, and that's
>> working fine: I can send e-mail to the system happily, and from the
>> system so long as it's from a trusted relay that doesn't have to
>> authenticate.
>>
>> The authentication data should all come out of the same LDAP database.
>> I'm happy that LDAP is working correctly, as I've got Cyrus IMAPd
>> authenticating against it, as well as using it for HTTP basic auth in
>> Apache.
>>
>> However sendmail refuses to play ball.
>
> Not true, sendmail asks whatever mechanism you configured and the
> error messages you get are from that mechanism.

Sendmail is the only SASL enabled application that is not working.
Smells like a sendmail problem to me.

> [snip]
>> Authentication related config file stuff is pretty standard:
>>
>> TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5')dnl
>> define(`confAUTH_REALM', `thebunker.net')dnl
>> define(`confAUTH_MECHANISMS', `EXTERNAL DIGEST-MD5 CRAM-MD5')dnl
>>
>> I've tried setting up SASL to use:
>>
>> pwcheck_method: auxprop
>> auxprop_plugin: ldapdb
>> [...]
>>
>> but no dice. That gives me the following in auth.log:
>>
>> Sep 1 18:31:09 livid sm-mta[49959]: Unexpectedly missing a prompt result
>> Sep 1 18:31:09 livid sm-mta[49959]: no secret in database
>
> That message means the user is not in the database. Which database?
> that depends...

Except that the user ID is very definitely in the database, and it
works perfectly well with Cyrus IMAPd and indeed OpenLDAP both of
which are specifically using SASL authentication mechanisms.

>> So I also tried to use 'pwcheck_method: saslauthd' (which would be
>> acceptable although clearly it means the SASL EXTERNAL mechanism won't
>> be available)
>
> But in this case you are expecting LDAP to work? saslauthd uses what
> it was told to use on the parameter used to start it, did you check
> the "-a ldap" parameter? and is /usr/local/etc/saslauthd.conf set
> correctly?

Yes, thank you. As I said, *everything* about user authentication
comes out of the LDAP directory. saslauthd was configured to use LDAP
as a back end within seconds of installing it.

> [snip]
>> However trying the same smtptest results in the same 'no secret in
>> database' error as with the ldapdb auxprop_plugin, plus I cannot
>> detect that saslauthd is querying LDAP at all during this process.
>
> My guess is that saslauthd is using its own database, in /etc/sasldb2,
> but to be certain you have to check both build parameters and the
> options used to start saslauthd.
>
>> I'm missing something obvious somewhere, but I've run out of ideas as
>> to what that might be. Any clues would be gratefully received,
>> especially any debug flags for sendmail that will make it trace out
>> exactly what it is doing during the authentication process.
>
> Sasl is pretty hard to debug, be patient.
> --
> René Berber
>

Indeed.

Cheers,

Matthew

--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
Kent, CT11 9PW

On Sep 3, 4:39 am, Matthew Seaman wrote:

[snip]
> Except that the user ID is very definitely in the database, and it
> works perfectly well with Cyrus IMAPd and indeed OpenLDAP both of
> which are specifically using SASL authentication mechanisms.

I should have been more explicit, the user and realm are not in the
database.

You can see what saslauthd is complaining about in its log, /var/log/
authlog (the location depends on how syslog is configured), and also
you will have to increase the log_level in /usr/lib/sasl2/
Sendmail.conf .

[snip]
> As I said, *everything* about user authentication
> comes out of the LDAP directory. saslauthd was configured to use LDAP
> as a back end within seconds of installing it.

Regards.
--
René Berber

Ren? Berber <rberber@mailandnews.com> wrote:

: On Sep 2, 5:01 am, Matthew Seaman wrote:

: This has not much to do with sendmail, it is a cyrus-sasl problem,
: but...


I'm not so sure about the above statement.

Over the long weekend we upgraded a couple of our mailers from sendmail
8.12.x to the current 8.14.1 issue and as they say "something isn't right".

The first thing we noticed was in the syslogs these messages appearing at 5
minute intervals...

Sep 4 15:45:12 e4500 sm-mta[24331]: [ID 702911 auth.notice] no secret in
database
Sep 4 15:45:30 e4500 sm-mta[24400]: [ID 702911 auth.notice] no user in db

These messages never appeared before and appear to be tied to saslauthd but
can't explain why it's being reported from sm-mta. In our case saslauthd is
using pam, there is no database but after looking around, it seems like a
common "don't worry about it" problem that can be ignored (mostly from
postfix users).

However, smtp AUTH was still working fine, except in one case...

Yesterday one person called saying his smtp program died around the time we
changed over sendmail. His software is called VPOP3, which looks a bit
hokey, but whatever floats your boat. They just relay through us.

After covering the standard "check the configuration stuff" on his end, we
even moved him to another 8.14.1 server which is basically running the
simpliest cf/mc you can think of (no options) and got the same results.

He managed to get us a copy of his logs of the smtp conversation and the one
thing that stood out was this...

4/9/2007 15:40:05.890 - 00000003<250-ETRN
4/9/2007 15:40:05.890 - 00000003<250-AUTH PLAIN LOGIN CRAM-MD5
4/9/2007 15:40:05.890 - 00000003<250-STARTTLS
4/9/2007 15:40:05.890 - 00000003<250-DELIVERBY
4/9/2007 15:40:05.890 - 00000003<250
4/9/2007 15:40:05.890 - 00000003>AUTH CRAM-MD5
4/9/2007 15:40:05.953 - 00000003<334
PDM1NjA0NzExMjQuMTQ1MzMzNjFAZW1haWwucmlwY28uY29tPg ==
4/9/2007 15:40:05.953 -
00000003>ZGF0YXdhdmUgNzBhMzRjM2Y3MTZhMjllZjliN2U0Z Tc2ZmI3YTZjOWE=
4/9/2007 15:40:06.031 - 00000003<535 5.7.0 authentication failed
4/9/2007 15:40:06.031 - 00000003>QUIT

The problem here is, his software (VPOP3), according to him, has no
reference to supporting or using MD5 anything.

Hmmmm.

For yucks I did a quick edit on our sendmail.cf clipping out the CRAM-MD5
from the AuthMechanisms= line and it not only fixed his problem but stopped
the syslog messages about the no user in db stuff.

Since it isn't going to hurt anything, I'd suggest the original poster who
lost AUTH from working, do the same and see what happens.

Just set the one line in sendmail.cf to

O AuthMechanisms=PLAIN LOGIN

restart sendmail and see how it works now.

Being we never ran into this in the past with pre 8.14.x sendmail and are
using a known working saslauthd, like I said in the beginning, I wouldn't
rule out some bug in this version.

The old 8.12.x had the line

O AuthMechanisms=PLAIN LOGIN CRAM-MD5 DIGEST-MD5

and this wasn't an issue, it is with 8.14.1.

Don't know what to make of it all but is what it is.

-bruce
bje@ripco.com

In article <fbma5a$q6a$1@e250.ripco.com>,
Bruce Esquibel <bje@e4500.ripco.com> wrote:

> He managed to get us a copy of his logs of the smtp conversation and the one
> thing that stood out was this...
>
> 4/9/2007 15:40:05.890 - 00000003<250-ETRN
> 4/9/2007 15:40:05.890 - 00000003<250-AUTH PLAIN LOGIN CRAM-MD5
> 4/9/2007 15:40:05.890 - 00000003<250-STARTTLS
> 4/9/2007 15:40:05.890 - 00000003<250-DELIVERBY
> 4/9/2007 15:40:05.890 - 00000003<250
> 4/9/2007 15:40:05.890 - 00000003>AUTH CRAM-MD5
> 4/9/2007 15:40:05.953 - 00000003<334
> PDM1NjA0NzExMjQuMTQ1MzMzNjFAZW1haWwucmlwY28uY29tPg ==
> 4/9/2007 15:40:05.953 -
> 00000003>ZGF0YXdhdmUgNzBhMzRjM2Y3MTZhMjllZjliN2U0Z Tc2ZmI3YTZjOWE=
> 4/9/2007 15:40:06.031 - 00000003<535 5.7.0 authentication failed
> 4/9/2007 15:40:06.031 - 00000003>QUIT
>
> The problem here is, his software (VPOP3), according to him, has no
> reference to supporting or using MD5 anything.
>
> Hmmmm.

CRAM-MD5 is a SASL authentication mechanism that can be used over
unencrypted channel because it never transmits the password in a
recoverable form. The flipside of that advantage is that it requires
that both the client and server need to have unencrypted forms of the
password or "shared secret." That might be considered a problem for some
people.


> For yucks I did a quick edit on our sendmail.cf clipping out the CRAM-MD5
> from the AuthMechanisms= line and it not only fixed his problem but stopped
> the syslog messages about the no user in db stuff.
>
> Since it isn't going to hurt anything, I'd suggest the original poster who
> lost AUTH from working, do the same and see what happens.

Actually, it CAN hurt. If you care about the risk of sniffed passwords,
supporting LOGIN and PLAIN over unencrypted channels is a problem. On
the other hand, if you already require TLS, then using CRAM-MD5 or
DIGEST-MD5 is pointless.

A better approach if you support unencrypted sessions is to actually
make CRAM-MD5 work. That includes building the SASL password database.

> Just set the one line in sendmail.cf to
>
> O AuthMechanisms=PLAIN LOGIN
>
> restart sendmail and see how it works now.
>
> Being we never ran into this in the past with pre 8.14.x sendmail and are
> using a known working saslauthd, like I said in the beginning, I wouldn't
> rule out some bug in this version.
>
> The old 8.12.x had the line
>
> O AuthMechanisms=PLAIN LOGIN CRAM-MD5 DIGEST-MD5
>
> and this wasn't an issue, it is with 8.14.1.
>
> Don't know what to make of it all but is what it is.

The difference is indeed odd.

--
Now where did I hide that website...

René Berber <rberber@mailandnews.com> writes:

> On Sep 3, 4:39 am, Matthew Seaman wrote:
>
> [snip]
>> Except that the user ID is very definitely in the database, and it
>> works perfectly well with Cyrus IMAPd and indeed OpenLDAP both of
>> which are specifically using SASL authentication mechanisms.
>
> I should have been more explicit, the user and realm are not in the
> database.

Hmmm... yes. In a sense. With ldapdb, the realm appears in the
LDAP DN that SASL generates, which might look like this:

uid=matthew@thebunker.net,cn=thebunker.net,cn=dige st-md5,digest=auth
^^^^^^^^^^^^^^^^
This is the realm


and you can translate that to a dn present in the DB using authz-regexp
statements in slapd.conf:

authz-regexp
uid=([^@,]*)(@[^,]*)?,(cn=thebunker.net,)?cn=digest-md5,cn=auth
ldap:///dc=thebunker,dc=net??sub?(uid=$1)

(which will match either 'thebunker.net' or a null realm) So the realm
is mentioned in the config file but doesn't actually appear in the
LDAP DB per se.


> You can see what saslauthd is complaining about in its log, /var/log/
> authlog (the location depends on how syslog is configured), and also
> you will have to increase the log_level in /usr/lib/sasl2/
> Sendmail.conf .
>
> [snip]
>> As I said, *everything* about user authentication
>> comes out of the LDAP directory. saslauthd was configured to use LDAP
>> as a back end within seconds of installing it.

Well, finally I can report success. I got it working in the end by
using 'saslauthd -a ldap' and it turned out that a lot of the problem
was that the 'smtptest' utility from Cyrus IMAPd was always returning
'Authentication Failed', which didn't one bit. (Probably more to
do with incompetance on my behalf than any problem with the software)
Testing using Thunderbird instead allowed me to make a lot more
progress.

I still haven't managed to get a direct SASL login using
'auxprop_plugin: ldapdb' in /usr/local/lib/sasl2/Sendmail.conf -- I
get traces like this in the slapd.log:

Sep 7 20:08:43 irate slapd[52308]: ==> sasl_bind: dn="" mech=DIGEST-MD5 datalen=0
Sep 7 20:08:43 irate slapd[52308]: connection_get(30)
Sep 7 20:08:43 irate slapd[52308]: connection_operation: error: SASL bind in progress (tag=66).
Sep 7 20:08:43 irate slapd[52308]: send_ldap_result: err=1 matched="" text="SASL bind in progress"
Sep 7 20:08:46 irate slapd[52308]: connection_get(30)
Sep 7 20:08:46 irate slapd[52308]: SRCH "" 0 0
Sep 7 20:08:46 irate slapd[52308]: 0 0 0
Sep 7 20:08:46 irate slapd[52308]: filter: (objectClass=*)
Sep 7 20:08:46 irate slapd[52308]: attrs:
Sep 7 20:08:46 irate slapd[52308]: supportedSASLMechanisms
Sep 7 20:08:46 irate slapd[52308]:
Sep 7 20:08:46 irate slapd[52308]: send_ldap_result: err=0 matched="" text=""
Sep 7 20:08:46 irate slapd[52308]: connection_get(30)
Sep 7 20:08:46 irate slapd[52308]: ==> sasl_bind: dn="" mech=DIGEST-MD5 datalen=0
Sep 7 20:08:46 irate slapd[52308]: connection_get(30)
Sep 7 20:08:46 irate slapd[52308]: connection_operation: error: SASL bind in progress (tag=66).
Sep 7 20:08:46 irate slapd[52308]: send_ldap_result: err=1 matched="" text="SASL bind in progress"

which I don't understand at all. It's trying to find what SASL
mechanisms it can use in the middle of doing a DIGEST-MD5 bind as ""
-- the root of the LDAP tree?

Still, now I have a working configuration, so I'm happy.

Matthew

--
Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard
Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate
Kent, CT11 9PW