too long to scan thousands entries in mailq : can Linux SMTP postsuper move to another mail queue


I have a case where tens of thousands of spam & spoof emails jammed up my
Linux SMTP mail queue :
mailq | grep MAILER | wc -l    ==> tens of thousands

This caused genuine emails to queue up & not being processed because the
it will take forever to scan from the top to the end of the mailq with tens of
thousands of emails.

However, I don't want to delete away those spam/spoof mails, I just to move them
to somewhere else (say another queue) so that I can review & in case there are
genuine ones, I'll redeliver them later.

I thought of:
mailq | grep -i mailer | awk '{print ($1)}' | grep -v "@"  > listofsuspectmails.txt

for each queue id or mail entry id in listofsuspectmails.txt,  I'll do a
   postsuper -h entry_id
to hold the suspect email.

Question is :
Are the 'held' mails still something that will be scanned (& thus prolonged the
scan time) or scanning of mail queue will skip 'witheld' mails?  If I have 100000
'witheld' emails & they're still being scanned, it will defeat this purpose of me
witholding the mails.

Currently we manually do a " postsuper -d entry_id " to delete away emails
(which can be automated with a Shell script but I don't want to automate
deletion as we may delete possibly genuine emails) after examining the emails

Any other solution are welcome
sunhuxAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Hugh FraserConsultantCommented:
Messages in the held state are not processed until they're released. They won't consume any resources other than disk apace. But keep in  mind that they're also not subject to delivery timeouts, so in effect they'll be kept forever until you do something about them. With no automatic pruning, disk space can become an issue if nobody's on top of things.

Also, a lot of these messages might be undeliverable and could be filtered further upstream at the SMTP dialog with a whitelist. One site I've been involved with received 100,000 deliverable messages a day, with 75% being spam, a typical value. But there were an additional 2,400,000 messages addressed to non-existent users that were dropped during the "RCPT TO" part of the SMTP dialog, rather thanactually  receiving and processing the email.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
sunhuxAuthor Commented:

As long as 'held' mails  (ie those mails being set by "postsuper -h" ) won't be scanned in
the mailq & allows genuine emails to be processed (or genuine mails not be hogged") then
that's what I want.  Just to reconfirm again with you, 'held' mails won't be scanned & even
with 1 million mails being 'held', genuine mails can still be processed, right?

No issue with disk space as we have a lot of it & we'll be doing housekeeping weekly.

Btw, I curious to know how you handle undeliverable & bounced emails as these can
come up to quite a huge amount too in my environment
Hugh FraserConsultantCommented:
Correct. Held messages are not processed and will not delay other messages. The caveat about  message lifetime is that held messages will stay in the queue indefinitely, and require manual intervention to clean up.

Inbound mail is received by an SMTP daemon, processed, and delivered. Usually, the SMTP daemon performs basic protocol verification, and often some RBL checks. It's not until the mail's received by the daemon, added to an inbound mail queue, and processed  that a non-existent recipient is identified, often after AV and spam checks are done. And of course, the resources consumed to check a message that can't ever be delivered delay legitimate traffic.

Using a whitelist, mail can be dropped by the SMTP daemon before the body of the mail is accepted. The site I'm familiar with uses sendmail, not postfix, but the same feature is available in postfiix. Typically, the whitelist is generated periodically from an internal directory service (like Active Directory) and uploaded to the SMTP daemon.  The SMTP dialog usually looks like this:

HELO
MAIL FROM: sender_address
RCPT TO: recipient_address
BODY
header and body of email message


The SMTP daemon performs an on-the-fly check of the recipient email address when it receives the "RCPT TO" part of the SMTP dialog, and issues and error response and closes the connection if the check fails. It's up to the sender to decide what to do with the error status. And since the body of the message comes after the RCPT TO step, you even save on network bandwidth.
sunhuxAuthor Commented:

Gee, thanks.

Bear with me, just one last question:

I'm using postfix on Redhat 4.x.
Which Linux OS file(s) is the mail queue file as I would like to backup the file(s)
before issuing "postsuper -d ..." (which I'll probably put in a Shell script's repeated
loop to do mass deletion).  In case I regret my decision, I can restore back the file(s)
& recover the 'deleted' mails
sunhuxAuthor Commented:

excellent
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Email Servers

From novice to tech pro — start learning today.