Solved

CentOS 6.5:  Setup DNSBL help

Posted on 2014-03-08
27
1,277 Views
Last Modified: 2014-11-12
Hi All,

I have a VPS that hosts around 20 websites.  Recently a spammer has been hammering my server.

I installed fail2ban to control the issue, which has worked well.  However looking in the log it appears all the IP addresses are listed on DNSBL.

So I think it would make more sense to enable DNSBL with postfix.

However I'm a CentOS newbie, so need a lot of help.

I have root SSH access and YUM installed.

Could someone walk me through the process?

I tried adding a couple of lines to my main.cf as per this guide, but fail2ban is still picking up loads of IP's that are on the DNSBL, so should bet getting through

http://www.howtoforge.com/block_spam_at_mta_level_postfix



Many thanks in adv.

D
0
Comment
Question by:detox1978
  • 13
  • 9
  • 4
  • +1
27 Comments
 
LVL 13

Expert Comment

by:Daniel Helgenberger
ID: 39915693
Hello detox1978,

can you please clarify some things:

- You talk about websites and SNMP (postfix) witch are two completely different things to me
- Which logs are you using with fail2ban, apache logs only (which would be the recommended way) or are you also scanning mail logs?
- Can you verify the recipient restrictions are working with mail logs?

Some further thoughts:
If you use DNSBL's with postfix, you still have a connection attempt which you see in the logs, however the mail is rejected - but always by postfix.
In general, DNSBL only work with SMTPd and not with your web server (httpd).

There is something called mod_spamhaus, which can block traffic from DNSBL in apache. Be aware this is not something you want on your web server in general. The main reason being that all major ISPs send their public DHCP ranges to the DNSBL providers to  block spammers on these dynamic IPs. But I would guess you have most of your web traffic originating from these IP address ranges.

If you want more security, you have sevral options beside fail2ban, I list them from the most common and easily to do to more complicated / sophisticated setups:

- You can always block IPs, ranges, and even geo locations with your firewall (iptables).
- You can use mod_security - witch is kind of a 'firewall' for httpd.
- If you have several hosts you want to protect in VPC, then it might be a good idea to deploy a new machine as a gateway / firewall. This could act as load balancer, reverse proxy and firewall; it makes it easy for you to centrally manage all the access to your web servers. I recommend pfSense - a AMI is readily available on AWS.
- You can use services like Cloud Flare - wich can help improve availability and security by proxying requests to your website. You can then effectively allow access to your website from only one IP (range).
0
 
LVL 2

Author Comment

by:detox1978
ID: 39915717
Sorry if it wasnt clear.  I am having issues with my SMTP (postfix) service being spammed.  Its receiving 1000's of emails an hour to random email address e.g. sdfgdfhxfd@mydomain.com

I've installed fail2ban which has built a massive list of rules in iptables blocking the ip's.  However in the last 20 minutes 346 ip addresses had been added.  Almost all of these are listed on DNSBL, so I'm looking for help to setup DNSBL.  I guess configuring it on postfix would be first port of call.  Then possibly on my DNS (named).
0
 
LVL 13

Expert Comment

by:Daniel Helgenberger
ID: 39915998
Hello,

no problem. Postfix is quite capable of filtering mails and handling many connections.
I still do not get if you see only the connections or you have problems with spam in your mailboxes (I assume the latter for now).
I did describe the problem above: The connection has to be made and the sender verified before it is dropped. It is quite normal.

Though quite powerful, there is no 'general' way to configure it - you need to know what you want to block. I can offer only some general guidelines in what I think is effective in Spam filtering. Basically, there are two restrictions, sender and recipient restrictions (MAIL FROM context, RCPT TO context).

In my environment these tests are the most effective; meaning filtering out most spam - from top to bottom:
- Recipient validation (check if there is really someone called 'joe@mycompany.com')
- DNS - Blacklists, SORBS, zen.spamhaus, CBL, SpamCop
- SPF Test
The rest of the tests contributes (like HELO, reverse DNS) - but most of the work is done by those three tests.

Later, you can add SpamAssain. This is a content based test which filters out mails witch specific words. Personally, I do not think this is effective but in the contrary has a lot of false positives.

All the rules should be tested. Postfix lets you add the directive warn_if_reject prior to any reject_* directives. This only logs errors and does not actually reject mail.  
Now, some tutorials:
Most important:
http://www.postfix.org/ADDRESS_VERIFICATION_README.html

This is an old but still good tutorial for postfix antispam:
http://www.cyberciti.biz/tips/postfix-spam-filtering-with-blacklists-howto.html

Setup SPF validations in postfix:
http://www.thenoccave.com/2013/05/08/centos-6-postfix-spf-checking/

Also, you can add grey listing. This was once considered effective, but it delays mail flow to long and nothing stops spammers from resending mail. I do not use Postgrey any more:
http://wiki.centos.org/HowTos/postgrey

Another effective way is to limit or block connections on your firewall already. This can take off some load of postfix; but be careful with this. I recommend a implementing a rate limiting if any.
Reject concurrent connections:
/sbin/iptables  -A INPUT -p tcp --syn --dport 25 -m connlimit --connlimit-above 5 -j REJECT

Open in new window

Rate limiting:
http://www.cyberciti.biz/faq/iptables-connection-limits-howto/
0
 
LVL 2

Author Comment

by:detox1978
ID: 39916345
Thanks for the info.  I will implement most of the suggestions.

I'll start with postgrey;

After following the instructions here;
http://wiki.centos.org/HowTos/postgrey

Emails are still being sent through straight away which suggests its not working.  Any ideas what I can check?  I followed the instructions and everything appeared to work.


Is there a way to check the smtpd_recipient_restrictions setting is being used?


When I check my mail log there is no reference to postgrey.
0
 
LVL 2

Author Comment

by:detox1978
ID: 39916381
Also,  I am seeing a lot of these messages in the maillog;

Mar  9 22:31:46 www postfix/smtpd[18546]: NOQUEUE: reject: RCPT from host17-105-static.58-79-b.business.telecomitalia.it[79.58.105.17]: 554 5.7.1 <gipbtnps@mydomain.co.uk>: Relay access denied; from=<gipbtnps@carlcomm.com> to=<gipbtnps@mydomain.co.uk> proto=ESMTP helo=<alicegate>

Open in new window


Does the NOQUEUE: reject:  mean the sender gets a NDR?  as I'd prefer to just discard them, as they are clearly spam and I dont want to be used for backscatter.
0
 
LVL 2

Author Comment

by:detox1978
ID: 39916403
update:

After a lot of tweaking around I have postgrey working.  I am still seeing a lot of these messages;

Mar  9 22:31:46 www postfix/smtpd[18546]: NOQUEUE: reject: RCPT from host17-105-static.58-79-b.business.telecomitalia.it[79.58.105.17]: 554 5.7.1 <gipbtnps@mydomain.co.uk>: Relay access denied; from=<gipbtnps@carlcomm.com> to=<gipbtnps@mydomain.co.uk> proto=ESMTP helo=<alicegate>

Open in new window


Is there a way to discard rather than reject?  or better still have postgrey delay the message without and NDR.
0
 
LVL 2

Author Comment

by:detox1978
ID: 39916456
I've a little bit of reading and it looks like I just need to change the SMTP return code from 554 to 550.
0
 
LVL 2

Author Comment

by:detox1978
ID: 39916468
I guess the orginal question still stands.

How do I configure postfix to use DNSBL to discard email from listed IP's


I've tried to follow the spamcop suggestion by adding them to my /etc/postfix/main.cf;

http://www.spamcop.net/fom-serve/cache/349.html

But my maillog is still full of listed IP's and fail2ban is still adding all the IP;s from the DNSBL, which i would expect to be dropped before fail2ban see's them
0
 
LVL 76

Expert Comment

by:arnold
ID: 39916520
The no queue event deals with attempts to send email through your server,  I.e spammer wants to spam someoneelsesdomain.com.  The reason you are. Seeing is either your mail server was an open relay or the IP you now have used to belong to an open relay.
0
 
LVL 13

Expert Comment

by:Daniel Helgenberger
ID: 39916951
Hello,

glad you have postgray working. Keep in mind my warnings above and the fact that postgrey only acts on new messages. After the message was resend, all subsequent messages from that particular sender are not greylisted (for a certain amount of time).

Though it seems I did paste the wrong link in my posting above, sorry for that. The 'most imported' link should read:
http://www.postfix.org/SMTPD_ACCESS_README.html

Also, I mentioned you can debug any reject statement with warn_if_reject; please also see the link above. But let me provide one example:
smtpd_recipient_restrictions = 
    ...
    warn_if_reject reject_invalid_helo_hostname, 
    ...

Open in new window


The relay access denied messages you have are actually a good thing. They tell you some of your smtpd_recipient_restrictions are working and the mail got rejected. Most likely because the mail was incorrectly formatted (= non - RFC - compliant). In almost all cases this is spam.

As long as you do not use the 'warn_if_reject' statement, you have no way of knowing which reject_* statement actually caused the relay access denied.
My theory is the DNSBL are working if you see a lot of those messages.

Fail2Ban:
I already answered this, you will always have those connections since postfix can only deny access after some initial commands have been exchanged between two mail servers (HELO, MAIL FROM, RCPT TO...). You will have connections from spammers all the time. This is fine as long as they get rejected.* My mail systems have about 60%-80% spam - but only on connection level.
IMHO fail2ban is completely useless with mail servers. Without careful teaking, you could block valid mail. Postfix itself is very capable of blocking spam if the configuration is correct. This takes a little bit of experimenting.

@arnold:
Seeing is either your mail server was an open relay or the IP you now have used to belong to an open relay
From the log snippets I have seen I cannot tell if this is the case. Do you mean open relay because 'relay access denied'? This is not the case, Relay access denied is a very general error message meaning only the connections was dropped for some reason by postfix.

-----
* It even helps - together with systems like tarpit still more - to slow down spammers and therefore reduces overall spam. The statement in main.cf would be in_flow_delay:
in_flow_delay = 1s

Open in new window

0
 
LVL 2

Author Comment

by:detox1978
ID: 39917106
Thanks for the info.  I followed the guide in your link and the NOQUEUE reject messages has changed from "Relay access denied" to "Client host rejected: Access denied;"

I think we've made a lot of progress, however i would still like to see a list of emails (inc NDR) that my server is sending.  is that possible?
0
 
LVL 13

Expert Comment

by:Daniel Helgenberger
ID: 39917219
Hm, I do not follow you here. /var/log/maillog contains all the messages your server sends and receives. Adding the -v option to smdpd in master.cf makes it also more verbouse.

To debug, it is usually a good way to tail -f your maillog to see what is going on:
tail -f /var/log/maillog
# show all messages currently in the quere
mailq

Open in new window

Also, greping might make sense:
# shows all valid mail
grep "Message Accepted" /var/log/maillog
# show bounced messages, this is usually bad. Message was not delivered.
grep status=bounced /var/log/maillog
# deferred shows if something got wrong temporally, can mean greylisting 
grep status=deferred /var/log/maillog

Open in new window

"Message queued for delivery" - This means the transaction between your server and the target server has yet to transpire for that particular message, this usually means something just sent the message, and your SMTP server is acknowledging it's existence

"Message Accepted" - This means your SMTP server acknowledges that the target mailbox (email address) is valid and it will take it from there (It doesn't indicate read)

"Bounced" - This typically means that something went wrong - either the email was rejected from the target email server because the email address didn't exist, OR it could be rejected due to being on an RBL. This also means the email will NOT be delivered, nor handled anymore by the server. AKA: The message is dead in the water.

"Deferred" - This means that something temporary has happened to cause the message to not be delivered, but the server (yours) hasn't given up and will try again later. This is also common to see when the target SMTP server uses an anti-spam technique known as 'greylisting'.

This might be helpful:
http://en.redinskala.com/postfix-maillog-interpretation/

I also find it helpful to log message subjects. Here a tutorial how to do so; but be aware that this works by trowing a warning message for every email:
http://davidsj.co.uk/linux/log-subject-lines-in-postfix/

Do log even more, like whole messages, you would need something called after quere filters. This is build in,
http://www.postfix.org/FILTER_README.html

As for the NDR's - be very careful with those. The default config in postfix should have them disabled for external addresses. The point here is something called backscatter. If a spammer uses forged MAIL FROM and sends to an unknown recipient in your domain, your mail server would flood external (existing) domains with NDR's. This is not only embarrassing, but could have your mailserver listed in DNSBL:
http://www.postfix.org/BACKSCATTER_README.html
0
 
LVL 2

Author Comment

by:detox1978
ID: 39924556
Thanks for the tips on searching the logs.  I sent myself an email and looked at what was logged.  Then used that to grep forwarded emails.

Postfix is still sending emails from IP's that are listed on DNSBL, so it's not setup correctly.

Also every 10 minutes fail2ban is emailing me because an IP has been added.

Is there a way to check DNSBL is working on postifx?
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 13

Expert Comment

by:Daniel Helgenberger
ID: 39925922
Sorry, you have me confused here:
Postfix is still sending emails from IP's that are listed on DNSBL, so it's not setup correctly.
Postfix is sending you emails? Or are these mails processed by postfix and you have them in your inbox? I am asking because DNSBL only matter when postfix is processing incoming mails witch are then put in your mailbox.

Also every 10 minutes fail2ban is emailing me because an IP has been added.
What IP is where added?

Is there a way to check DNSBL is working on postfix?
Of course there is. Please consider my earlier recommendation regarding warn_if_reject.

To help you any further, please post:
- your main.cf or atleast restrictions from your main.cf as well as
- an example log entry from a mail; a log snippet from postfix where you did receive a spam mail (please only that snippet, not the whole log, since it will be impossible for me to find that mail).*
- the headers of your spam mail for the log entry.

* Hint: In the mail header you will find a from - ip address of the originating mail server. Just 'grep' your mail log with this ip address. Exapmple: You found the IP 1.2.3.4 in the header of your spam mail (including 5 lines before and after the find):
grep -C 5 1.2.3.4 /var/log/maillog

Open in new window

0
 
LVL 76

Expert Comment

by:arnold
ID: 39926370
If your influx of spam mailing has gone down, that means that dnsbl and fail2ban are doing their job.

There are layers of techniques that address spam, fail2ban and dnsbl are the least intrusive and are the first stage of combating spam.
The next step, deals with having postfix configured to use a Local Delivery mechanism that analyzes content.
Using SpamAssasin, etc.

http://www.postfix.org/FILTER_README.html

You can add an anti-virus ........
0
 
LVL 76

Assisted Solution

by:arnold
arnold earned 100 total points
ID: 39926376
Fail2ban and dnsbl are just the first stage in the battle against spam.

If you want to go to the next level of spam fighting, http://www.postfix.org/FILTER_README.html

using a content filter adds overhead to the system, but is "more" precise.

You can scan your logs for events that deal with postfix rejecting a message because it is on a DNSBL list.
Depending on which lists you use, the message displayed to the sender references the list that IP is on.
0
 
LVL 27

Expert Comment

by:serialband
ID: 39926563
You could also use amavis.  It plugs right into postfix and uses clamav and spamassassin to scan for spam and block it.
0
 
LVL 2

Author Comment

by:detox1978
ID: 39933106
Fail2ban is definatly working as I cab see ip's being added.

I don't believe dnsbl are working as all of the UP's fail2ban is adding to iptables are ip's listed on spamcop.

How can I test dnsbl?
0
 
LVL 13

Expert Comment

by:Daniel Helgenberger
ID: 39933240
Detox,

if I may, let me I try one last time, I do not think you tried this already:
warn_if_reject

Open in new window


in front of the dnsbl rejects. please see my example here, where I mentioned you can do this with every reject statement you like:
http://www.experts-exchange.com/OS/Linux/Q_28383789.html#a39916951

Then, every reject will be logged. If you see warnings in maillog then dnsbl's are working. If not, then they don't. For further analysis please post the config and logs I asked.

In your other question I already answered how to test it in general using dig.

IMHO we should also try to stick to the topic and get DNSBLs working for you. Fighting spam is broad topic; I fear the discussing every Linux antispam tool here is counterproductive.
0
 
LVL 2

Author Comment

by:detox1978
ID: 39933599
Yes I have that in my config.  However fail2ban is still banning ip's that are on dnsbl.

Is there a wat to drop the connection instead if warning?

How do I install dig?
0
 
LVL 13

Expert Comment

by:Daniel Helgenberger
ID: 39933824
Hello detox,

I suppose this is because of the warn_if_reject then. Please remember, you wanted to test it.

Carefully read my comments as well as the links I posted.

Bottom line: warn_if_reject only logs the reject but does not actually do the reject.
Here the relevant part from the man pages:
warn_if_reject
A safety net for testing. When "warn_if_reject" is placed before a reject-type restriction, access table query, or check_policy_service query, this logs a "reject_warning" message instead of rejecting a request [ ... ].

As you can see, DNSBL's are working perfectly. Remove warn_if_rejects; and fail2ban should not complain any more.
Also, you can define more than one DNSBL. I already wrote witch one I think are the best. You order them from the best to the worst (most entries to lowest number of entries) to keep the DNS queries to a minimum (as recommended by the man pages.)
Also, I recommend Vamsofts ORF statistics, as they keep an up to date list.
0
 
LVL 2

Author Comment

by:detox1978
ID: 39934012
This is what i have in my main.cf

smtpd_recipient_restrictions =
            reject_invalid_hostname,
            reject_unknown_recipient_domain,
            reject_unauth_pipelining,
            permit_mynetworks,
            permit_sasl_authenticated,
            reject_unauth_destination,
            reject_rbl_client multi.uribl.com,
            reject_rbl_client dsn.rfc-ignorant.org,
            reject_rbl_client dul.dnsbl.sorbs.net,
            reject_rbl_client list.dsbl.org,
            reject_rbl_client sbl-xbl.spamhaus.org,
            reject_rbl_client bl.spamcop.net,
            reject_rbl_client dnsbl.sorbs.net,
            reject_rbl_client cbl.abuseat.org,
            reject_rbl_client ix.dnsbl.manitu.net,
            reject_rbl_client combined.rbl.msrbl.net,
            reject_rbl_client rabl.nuclearelephant.com,
            check_policy_service unix:postgrey/socket,
            permit

Open in new window



There is no warn_if_reject and fail2ban is still adding ip's from DNSBL
0
 
LVL 76

Expert Comment

by:arnold
ID: 39934046
the dnsbl works by shortening the smtp session up to the point where the recipient id designated.  those connections are still being logged without regard to whether the message was accepted or rejected. the packets are still flowing in.

IMHO, it is better to allow a shortened smtp session at the conclusion of which the sending server gets a 5xx error to not try again than the firewall route, which means the sending server will re-queue the messages such that should an ip get delisted, with fail2ban the ip is removed and all the queued up message are retransmitted.
0
 
LVL 13

Accepted Solution

by:
Daniel Helgenberger earned 400 total points
ID: 39934086
Hello detox1978,

as arnold correctly stated - and if I may I quote my first comment about your fail2ban/DNSBL setup:
If you use DNSBL's with postfix, you still have a connection attempt which you see in the logs, however the mail is rejected - but always by postfix.

Please keep in mind, fail2ban acts on logs - but you will always need an active TCP connection for postfix do apply any filtering at all. So there will always be a log entry form a blacklisted server.

Even if I repeat myself again, since postfix is very well capable of handling spam, fail2ban is NOT an advantage. You may even block legitimate mail.

You should, however, optimize your restrictions.
You are not even using the 'best' DNSBL Providers.
Three blacklists are more then enough, you should also start with list which has the most entries. Currently, this will be Spamcop. I took this info form vamsofts site:
smtpd_recipient_restrictions =
            reject_invalid_hostname,
            reject_unknown_recipient_domain,
            reject_unauth_pipelining,
            permit_mynetworks,
            permit_sasl_authenticated,
            reject_unauth_destination,
            reject_rbl_client bl.spamcop.net,
            reject_rbl_client zen.spamhaus.org,
            check_policy_service unix:postgrey/socket,
            permit

Open in new window

0
 
LVL 2

Author Comment

by:detox1978
ID: 39934580
Daniel,

I've updated my smtpd_recipient_restrictions to the ones you suggest.  Which seems to be rejecting all the emails listed on DNSBL, so that appears to be now working.

One last thing.

I'm seeing a lot of "address not listed for hostname", e.g.

Mar 17 15:58:26 www postfix/smtpd[344]: warning: 80.111.225.227: address not listed for hostname cm-80.111.225.227.ntlworld.ie

Open in new window


I'm guessing this is where reverse DNS doesnt match.  If so, how can I reject these emails?


Many thanks
0
 
LVL 13

Assisted Solution

by:Daniel Helgenberger
Daniel Helgenberger earned 400 total points
ID: 39934660
Hello detox,

I am glad you have the DNSBL's finally working.

I'm guessing this is where reverse DNS doesnt match.
You've guessed correctly.
Did this mail get rejected?

In this particular case, a forward address for the reverse does not exist. Try looking up cm-80.111.225.227.ntlworld.ie.

Normally you have that warning only if you have one of the following restrictions in your main.cf:
reject_unknown_reverse_client_hostname
reject_unknown_client
reject_unknown_reverse_client

Can you check this?

Using reject_unknown_client in your main.cf will reject non-maching reverse lookups; this technique is called Forward-confirmed reverse DNS.

BUT: Be careful with Forward-confirmed reverse DNS. Some mail servers, especially MS Exchange, do not implement that correctly. I advise to use warn_if_reject for that.
0
 
LVL 2

Author Comment

by:detox1978
ID: 39934779
That's worked very well.

I've enabled reject for now, when the spammers stopping using my server I'll take it off.
0

Featured Post

Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

This article provides a guide on how to optimise your costs within your AWS infrastructure when using some of the common services such as EC2, EBS, S3, Glacier, CloudFront, EIP & ELB.
Security is one of the biggest concerns when moving and migrating your data from your on-premise location to the Public Cloud.  Where is your data? Who can access it? Will it be safe from accidental deletion?  All of these questions and more are imp…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now