How to whitelist some servers listes on dnsbl blacklists and bypassing dnsbl for some users

I am running sendmail version 8.13.1 under CentOS 4.4 . I am also running the greylist milter via a PL script as shown here:  

http://projects.puremagic.com/greylisting/

I was using several dsnbl's and they blocked a ton of connections - and the first connect point.
The problem is that some of our customer and potential customers kept finding their way onto the lists, so we would block their mail without any warning or notice on our end.

So I would like to do two things, if possible:

1) Whitelist all known customers and have the dnsbl ignore (or not do) the online lookup if the customer's IP is in the whitelist. I think this can be done by the "spamfriend" feature in sendmail, but I have never used it before.  One thing I do know is that I have to turn on FEATURE(delay_checks) to use this feature. What, if any, impact will this have on my greylist milter and/or mail server operation?  I would assume we would have to spend more time connecting and getting all the mail info before we dumped the bad guys, but there could be other problems (to greylist?) from turning on the delaychecks option.

2) I would like to bypass all dnsbl checks for certain e-mail addresses (like sales@mydomain.com) so that no matter what, sales gets the mail even if the sending server's IP is on every known online dnsbl. Can this be done? This would let potential customer (not yet in the spamfriend whitelist) get mail to us even if they are listed.

So can these be done?  Side effects of delaying the checks?  And a simple how-to to show me for sure what goes where.

Thanks!
Dennis


dlwynneAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

PsiCopCommented:
OK, sendmail v8.13.1 on Linux. Latest rev of sendmail is v8.14.2, but v8.13.1 is reasonable.

Only trouble is that virtually every Linux distro ships a really crappy default sendmail config. Check my profile for some links to some technical papers and a sendmail.mc that is actually documented.

FEATURE(delay_checks) tells sendmail to deley RBL checks until the RCPT TO: step of the SMTP conversation, rather than checking at connection time. It also enables use of the "Spam:" keyword in the access map, as you've noted. Exactly how it would interact with your greylisting MILTER depends on where in the SMTP conversation the MILTER is applied. If it is applied at connection time, at HELO or at MAIL FROM:, then it has no interaction with FEATURE(delay_checks).

Using the "Spam:" keyword in the access map has a drawback with regard to what you seem to want to do, in that it functions on entire addresses (e.g. mailbox@domain.tld), not on a per-domain basis (e.g. @domain.tld). However, it will work if you want to have all E-Mail destined to "sales@mydomain.tld" bypass RBL checking (actually, the check is performed, the result is changed). See the appropriate entry in the access map below.

Note that implementing such a bypass will result in a torrent of SPAM to that address. It's also a "standard spamming" address. Most of the Domains I host do not have "sales@thedomain.tld" as a valid address, yet spammers routinely send to that address, along with "info@", "support@", "help@", et. al. ad. nauseum.

If you're depending on greylisting and RBLs to stop spam, then I imagine quite a lot is getting through. A few statistics.

1) About 90% of E-Mail is SPAM. That means every time a remote host connects to your mailserver, there's about a 90% chance that it is someone looking to SPAM you.

2) About 50% of spammers tip their hand by HELO. That is, you can positively identify about half of connecting hosts as a spammer by the HELO step of the SMTP conversation. Long before you check RBLs.

To do that, you need a MILTER that engages at each step of the SMTP conversation. I use MIMEDefang (http://www.mimedefang.org).
Spam:whitelist.sales@mydomain.tld    FRIEND

Open in new window

0
PsiCopCommented:
Ah, you mean the "Proposed Solution" radio button when posting?

Yeah, I wondered why they brought that back.

If it has no function, why do they (EE) display it?
0
TintinCommented:
I've wondered about it as well.  Actually it does have a function.  When emails are sent out with the "Proposed Solution" selected, it's indicated in the email suject.
0
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

dlwynneAuthor Commented:
>> FEATURE(delay_checks) tells sendmail to deley RBL checks until the RCPT TO: step of the SMTP
>> conversation, rather than checking at connection time. It also enables use of the "Spam:" keyword in
>> the access map, as you've noted. Exactly how it would interact with your greylisting MILTER
>> depends on where in the SMTP conversation the MILTER is applied. If it is applied at connection time,
>> at HELO or at MAIL FROM:, then it has no interaction with FEATURE(delay_checks).

The grey list uses the IP, from, and to fields to form a triplet of info that is looked up the MYSQL database. If we have seen and accept mail with the triplet it is passed. If this is the first we have seen of that triplet, we issue a temp fail and drop the connection. It would seem that if I use delay_checks each connection will take a bit longer before it is dropped by the milter, but should have no other effect?


>> Using the "Spam:" keyword in the access map has a drawback with regard to what you seem to
>> want to do, in that it functions on entire addresses (e.g. mailbox@domain.tld), not on a per-domain
>> basis (e.g. @domain.tld). However, it will work if you want to have all E-Mail destined
>> to "sales@mydomain.tld" bypass RBL checking (actually, the check is performed, the result is
>> changed). See the appropriate entry in the access map below.
>>
>> Note that implementing such a bypass will result in a torrent of SPAM to that address. It's also
>> a "standard spamming" address. Most of the Domains I host do not have "sales@thedomain.tld" as a
>> valid address, yet spammers routinely send to that address, along
>> with "info@", "support@", "help@", et. al. ad. nauseum.
>>

I think the #1 goal of the changes would be to bypass the BRLs for the sales e-mails (not an actual sales@domain.com) since they would rather have to dump a lot of spam than to miss one sales opportunity.

If I can bypass the RBLs for their mail, then the other stuff is largely secondary.

So I turn on DELAY_CHECKS option just just add

Spam:actual_salespersons_email_addy@mydomain.tld    FRIEND

to the access file?  I don't have to add anything else (friend, etc) to the sendmail config file? I thought the option had to be set to something like:

FEATURE(`delay_checks', `friend')

I think that should "fix" the problem - the rest of us get to go back to the RBLs but the folks that have to have the mail no matter what will get their e-mails in the access.db and it will bypass the RBLs.


The 2nd part of my question is not as important, but I would like to implement it if there is a simple fix. If we get mail from any user at "Good company #1" (@goodcompany1.com) and "Good Company #2 (@goodcompany2.com) and we put the RBLs back in place then only the FRIENDS would get e-mail through from either company should they find themselves on the RBL. So If I have a list of the companies and want their e-mail to get through even if on the RBL to any recipient at our domain, how do I do that? I have the domain or even their mail server IPs I can list in a "whitelist" if I knew how to implement it.  Let the mail through (assuming it passes the other tests) from known good customer domains and/or known good customer mail servers even if the IP is on a RBL.  Can that be done simply?

Thanks,
Dennis




0
PsiCopCommented:
Quoting:

The grey list uses the IP, from, and to fields to form a triplet of info that is looked up the MYSQL database. If we have seen and accept mail with the triplet it is passed. If this is the first we have seen of that triplet, we issue a temp fail and drop the connection. It would seem that if I use delay_checks each connection will take a bit longer before it is dropped by the milter, but should have no other effect?

Response:

That still doesn't tell me WHERE IN THE SMTP CONVERSATION the greylisting check occurs.

Does it occur at connection time? At HELO? MAIL FROM? RCPT TO? DATA?

delay_checks will not have an effect one way or another.

Without delay_checks, the RBL check occurs at connection time. So if the greylist Milter is engaged at a later step, and the RBL check drops the connection, then the greylist Milter is never invoked.

With delay_checks, the RBL check occurs at the RCPT TO: step of the SMTP conversation. If the greylist Milter was engaged at an earlier step, then its already done its job, and so is not affected my delay_checks.

Unfortunately, I still don't know where in the SMTP conversation your Milter works.
0
PsiCopCommented:
To implement delay_checks, yes, you add

FEATURE(`delay_checks', `friend')

to your sendmail.mc, rebuild your sendmail.cf, and restart sendmail (to load the new config). At that point, you can add the

Spam:actual_salespersons_email_addy@mydomain.tld    FRIEND

line to /etc/mail/access and rebuild the access map.

I don't know what facilities Centos may have for automating the rebuild of sendmail.cf and/or the access map.

Changes to the access map are immediately effective and do not require restarting sendmail. Again, check my profile for some links to some technical papers and a sendmail.mc that is actually documented - specifically, look for the paper entitled "Practical Modern Sendmail Configuration".

Note that delay_checks is dependent on access_db, and both must be in sendmail.mc. See the sendmail.mc text extracts below.
dnl # Sendmail, Chap 7.5, Page 311
dnl # Turn on Access DB to accept/reject mail from selected sites, and
dnl #   specify database type, path and name; "-o" makes it optional and
dnl #   "-T<TMPF>" parameter instructs daemon to return SMTP 4xy codes
dnl #   for temporary errors
FEATURE(`access_db',`hash -o -T<TMPF> /etc/mail/access')dnl
 
dnl # Sendmail, Chap 7.5.6, Page 318
dnl # Change order of relay checks (requires "access_db" feature above)
dnl #   to check SMTP RCPT TO: first, then SMTP MAIL FROM:, and finally
dnl #   the host (via access_db and RBLs) - "friend" keyword allows 
dnl #   entries in access_db to override RBLs and "n" turns off 
dnl #   backwards-compatibility with earlier versions of sendmail
dnl # Has interaction and dependency relationships with 
dnl #   FEATURE(`conncontrol') and FEATURE(`ratecontrol')
FEATURE(`delay_checks',`friend',`n')dnl

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
PsiCopCommented:
Quoting:

I think the #1 goal of the changes would be to bypass the BRLs for the sales e-mails (not an actual sales@domain.com) since they would rather have to dump a lot of spam than to miss one sales opportunity.

Response:

I'll bet your sales people run Winblows on their workstations, don't they. They use Lookout!, right?

You think SPAM just contains SPAM? It's also *heavily* used as a malware distribution vector. Put your latest Lookout! backdoor hack and spam it to a few million E-Mail addresses. Presto! At least a few thousand 'botted Winblows PCs.

Still so eager to have every last SPAM make it thru to your sales staff mailboxes?

If you really, really want to let every last incoming E-Mail in to your doubtlessly-non-technical sales people who probably run the least-secure-possible everything, I have two words to describe the desire: In-Sane. Just go ahead and download all the latest Winblows rootkits and backdoor Trojans and install them now. At least you'll have an idea of what malware is infesting the sales PCs.
0
PsiCopCommented:
Quoting:

So If I have a list of the companies and want their e-mail to get through even if on the RBL to any recipient at our domain, how do I do that? I have the domain or even their mail server IPs I can list in a "whitelist" if I knew how to implement it

Response:

Without using a more-sophisticated Milter (like MIMEDefang), about the only way to do this is use the access map. You'd define each individual sending mailhost, or possibly network, as having either OK or RELAY in the RHS of the access map.
0
dlwynneAuthor Commented:
>> That still doesn't tell me WHERE IN THE SMTP CONVERSATION the greylisting check occurs.
>>
>> Does it occur at connection time? At HELO? MAIL FROM? RCPT TO? DATA?
>> delay_checks will not have an effect one way or another.
>>
>> Without delay_checks, the RBL check occurs at connection time. So if the greylist Milter is engaged
>> at a later step, and the RBL check drops the connection, then the greylist Milter is never invoked.
>>
>>
>> With delay_checks, the RBL check occurs at the RCPT TO: step of the SMTP conversation. If the
>> greylist Milter was engaged at an earlier step, then its already done its job, and so is not affected my
>> delay_checks.
>>
>>Unfortunately, I still don't know where in the SMTP conversation your Milter works.

When I had RBLs in use, they dropped the connection immediately upon connection of a "bad" mail host, which is a good thing.

Since the greylist requires the IP, from, and to then it is not invoked until those have been collected (but before the DATA).

Sounds like DELAY_CHECKS will just mean the RBLs tie up the server for a longer period, which they do anyway since the RBL check is now off :-) .  In my case, it would seen that we would collect the IP, from, and to then check the RBL and dump the connection if found (and not to a spam friend), if it is not on the RBL or to a friend then it would hit the greylist milter.

Thanks,
Dennis
0
dlwynneAuthor Commented:
>> Without using a more-sophisticated Milter (like MIMEDefang), about the only way to do this is use the
>> access map. You'd define each individual sending mailhost, or possibly network, as having either OK
>> or RELAY in the RHS of the access map.

Say we have 10 customers that we want to insure get the mail through no matter if they send to a spam friend or not, if I add their mail server IPs (for example) to the left hand side

Connect:[122.1.2.3]      OK

or I could use their domain name

From:goodcustomer_domain.com    OK

It mail from that IP or from that domain would get through to any local user not matter if the customer's mail server is listed in the RBL or not?

Thanks again,
Dennis

0
PsiCopCommented:
"When I had RBLs in use, they dropped the connection immediately upon connection of a "bad" mail host, which is a good thing."

Actually, no, that's not. It gives you no way for a *legitimate* sender to contact you to request white-listing. That's the whole point to delay_checks. I've seen RBLs blacklist GMail and other legitmate sources. RBLs are, at best, a blunt instrument. Also, sendmail blocks when *any* RBL hits, and gives you no ability to give one RBL more weight than another.

"Sounds like DELAY_CHECKS will just mean the RBLs tie up the server for a longer period"

delay_checks has absolutely NO EFFECT with respect to the amount of DNS traffic sendmail will generate. All it does is change when the RBL checks occur. The same checks still happen.

"the greylist requires the IP, from, and to then it is not invoked until those have been collected"

I can generally identify half of the spammers by HELO and drop them at that point. Your greylist is letting them get all the way to RCPT TO. Why let them get that far?

To let a specific IP get thru no matter what:

Connect:[122.1.2.3]      RELAY

Using

From:goodcustomer_domain.com    OK   (or RELAY)

is more dangerous because you seem to have no verification mechanism for the envelope sender. *Anyone* can send you E-Mail claiming to be from that domain.
0
dlwynneAuthor Commented:
Thanks for the info, I was pretty sure it would work as I expected but was willing to give an expert the
"points" for verifying it for me before I made the changes.

I have put the RBLs back in place and bypassed them for those that do not want them and that is working fine.

I am going to have to bypass the checks for a few customers that seem to finding themselves on RBLs from time to time. I KNOW that it is their own fault, but they are our customers and pay us good money for the privilege of having us do business with them  :-) . So I can't just argue for them to fix it on their end, if that means that some spammer can spoof their mail server's IP and get a message through then so be it.  Most of out folks will put up with some spam to be sure they missing nothing important.

Dennis
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Email Servers

From novice to tech pro — start learning today.