qmail: forward existing mail in mailbox to other MTA

Never tried to do this but here is a link that may get you started down the right path.

http://www.damnsemicolon.com/php/parse-emails-in-php-with-email-piping-part-1

http://www.damnsemicolon.com/php/parse-emails-in-php-with-email-piping-part-2

Everything you're asking for here is reasonably easy except for one thing that you're asking, which is fundamentally impossible.

You cannot "resend" or "forward" an email without modifying the headers.  Because this operation is by definition creating a new email with its own headers and simply attaching the original.  The "Redirect" feature you see in some mail clients is a fairly good effort at faking it, but if it were reported as spam for instance, the host that redirected it would look like the guilty party.

The best bet to accomplish this is if both mailboxes are a common format (aka Unix mbox or Maildir.)  If that is the case, I would look at literally copying the messages into the other mailbox instead of attempting to put them there via SMTP.

If you can live with the shortcomings in that step, I can bang together a pretty easy php script for you to accomplish what you're asking for.

Hi xterm,

how does (old) bsmtp work then? That stores mail until the destination host is accessible and then start relaying mail. Does that change headers too?

thanks for your reply

"Store and forward" is a misnomer - I think you are referring to secondary MX:

You set up (in DNS zone for your domain) MX priority 10 for server A, MX priority 20 for server B.

Mail comes in to server A which is down, so flows to server B where it goes into the queue.  Server A comes back up online, contacts server B on port 25 and issues the ETRN command which tells server B to flush it's queues, and they get delivered to server A as if server B was never in the picture.

you got it, that's exactly what I want to do. Quoting you:

'....and they get delivered to server A as if server B was never in the picture'

Doesn't that mean that server B doesn not change anything to the headers... because that would break the situation that server B was never in the picture...

I'm not sure about the ETRN function but I'm kind of sure I cannot make your mailflow happen with my plesk server. Only thing I can do as backup mx is drop the mail in a mailbox, not store it in a queue until the server is online again. And then ETRN would not work anymore because it's pop mail then and not (b)smtp.

Okay, setting up secondary MX is a no-brainer, but you asked about taking mail that was already delivered out of mailboxes and sending them to another server in your question.  Which of the two do you want to do?

The function of secondary MX is not meant to be utilized to help you migrate customers to new providers.  It's supposed to be for instance if you have a mail server in a location that has spotty network, or internet, or power interruptions, or whatever - pretty much anything that would keep it from having 100% availability to the Internet.  So for the domain (or domains) on that server, you'd configure a secondary MX record to point to a machine that has high availability.  That system would never actually put the mail in a mailbox - it just holds it in a queue, and no user has any ability to get to it until that queue gets flushed, and the mail gets delivered to the primary MX.  This actually happens automatically because mail server queues are flushed at interval.  However, the primary MX is permitted to issue the ETRN command if the next queue iteration on the secondary is too far away.  So that part is totally optional.

Does that make more sense?

Jup, makes sense... the problem is:

I only have our Plesk hosting server that I can use as secondary MX and I don't think is has the option to keep mail into it's queue and then deliver it to another mailserver. That would be relaying and ofcourse our hosting server can't be configured to relay mail.

So that's why I wanted to create a script that would resend the mail in a specific mailbox to mimic the function of what you're describing above, like it was still in the mailserver queue.

1.  Secondary MX is not relaying.  The secondary MX queue is a destination, and then later it will do a direct delivery to the primary MX.  It's more like two direct sends.

2.  I can help you with the script, but if you use SMTP to deliver them, the headers will all change.  They really need to be copied as files to retain all original info.

Sorry if I keep misunderstanding you ;-) Just trying to make sure we are talking about the same thing:

Imagine the following situation:

1) Our internetdomain (company.com) is hosted on our hosting server that has Linux with Plesk 10 for the hosting
2) I have enabled the mailsystem for that domain in plesk (otherwise it will reject mail for that domain)
3) I have set our company server as primary mx 10 (ip 100.1.1.1) in the DNS of our internet domain
4) I have set our hosting server as secondary mx 20 (ip 200.2.2.2) in the DNS of our internet domain
5) I have to configure what to do with incoming mail for company.com on our mailserver. The options I have in Plesk:
    a) Drop mail for specific addresses in a mailbox
    b) Drop all mail for the domain in a catch-all mailbox
    c) Reject the mail for that domain

---- So I have no option to say 'queue the mail and send it to the following host: 200.2.2.2). ---

Okay, you're misunderstanding how this works.

In your scenario, you're looking for a setting on your primary MX to hold mail and then send it to your secondary MX.  That is backwards.

The way you have it set up right now (as described above in steps 3 and 4), if you TURN OFF mail on the primary MX (or take it offline to do maintenance or something), mail should go to your hosting server at 200.2.2.2.  That is the server you will have to enable as a secondary MX.  The mail will queue up on 200.2.2.2.  If that server is running sendmail, you would put "domain.com  RELAY" in /etc/mail/access and you would NOT put the name in /etc/mail/local-host-names.  This tells sendmail "this is not one of my domains, I'm just holding mail for it".  Then 200.2.2.2 will do a standard queue flush at interval (normally every 4 hours for sendmail) and if 100.1.1.1 is back up, all the mail will flow back in.  If not, it will keep trying for up to 5 days, and then if 100.1.1.1 is not back up, it will return the mail to the senders.

So, in a nutshell, mail always go to primary MX (lowest numbered priority) server first, but if it is down, it will attempt to go to the next highest numbers in order until it finds a server that is up and will take the mail.

We clearer now? :)

Hi,

I think we are indeed misunderstanding, I don't want to send mail to secondary mx:

- my company server is primary MX (where mail should end up finally)
- my hosting server is secondary MX (which should only hold mail until company server is back online)

I have Exchange 2007 at my company server which works fine
I have PLESK (a webserver management package) as the hosting server

When my company server is down I want mail to go to the webhosting server. Whenever my company server is backup I want the webserver to send mail to my company server as if was delivered normally.

So the above is clear, we are both talking about the same solution.

The problem is that our webserver uses PLESK 10. There is a 90% change that this management software regenerates the /etc/mail/xxxx files whenever we change something to the mailboxes, so my changes will be overwritten.

Okay, then you were simply misstating it above, but in actuality your DNS is set up correctly, because the primary MX is pointed at your company Exchange server, and your secondary MX record is pointed at your hosted Plesk server.  That part is complete and doesn't need to be messed with.

All that needs to happen is to set the PLESK 10 server to be a secondary mail exchanger and then it will work exactly as you'd hoped.

In light of the fact that the changes might be wiped out, it seems like a bad idea to start changing things.  I would suggest simply opening a ticket with your web host and asking them how to configure that system to be a secondary MX/mail exchanger for your domain mail hosted elsewhere (use those exact words.)  Your key questions boil down to:

1)  Where do I tell the MTA to permit mail to come in and be queued temporarily  for mydomain.com
2)  Is there a button somewhere that I can use to release the held mail after my primary MX comes back up?

Let's start there - maybe there is a real simple answer, and we just haven't asked the right question...

My web host is just selling the PLESK software on the servers they have up for rent.... They have no control over the PLESK software as it is from Parallels. All configuration needs to be done through the Plesk software, wich doesn't have an option to accept mail for a domain but not put it in a mailbox....

So that's why I needed a script to do it for my, independant of the qmail configuration that Plesk generated...

------------

If there isn't an option doing it without changing qmail config: is there a possibility to do it with a script and make sure all mail is delivered as it should, CC's and BCC's are also delivered and for the receiving server there is no difference... it appears to be delivered directly? So if headers need to be changed, fine, but make that change invisible for the end receiver? (again, they have to be able to see who is cc's and if the user is in the bcc the mail needs to reach their mailbox!)

If you cannot modify the PLESK host to accept the mail, then what is there to script?  Remember, for this to take place, you have to actually GET the mail at the PLESK host.

If that is the case, you'd have to configure yourdomain.com on the PLESK host, and then add all the users that are in your Exchange server to the PLESK host.  Then if the Exchange server went down, all the mail would be delivered there.

Then there are options:
1)  A script could look in every user directory for the mail, and then try it's best to recreate the original mail and resend it when the primary MX is up  (CCs are irrelevant, because those go where they need to go at the time of the send as separate messages, and same with BCC but of course, it never makes it into the headers by design)
2)  A script could copy the actual mail messages to the primary MX, however this is not an option since you run Exchange, and they don't store messages in that way.  This option is only good if you moved your company mail to a Unix environment

#1 is possible though.  Is this enough of a problem (your primary going down) to where you'd want to do all that work to keep both systems cloned every time a mailbox is added or deleted?

Keep in mind, that the SMTP protocol already does what you need.  If your server went down right now and you didn't have a secondary MX record, the mail would just queue on the sender's servers until your server came back.  The Internet at large is already serving as a secondary MX for you.   Is there any specific reason you need something better than this?  The only advantage of having your own one rather than letting the Internet queue it up is that you'd be able to control when the held mail got redelivered (and know how much of it was waiting).   You really may be embarking on kind of a tough journey with little payoff here...

Hi, thank you for you long reponse, I appreciate your efforts. just some clarification again ;-)

1) Our webserver (Plesk) DOES accept mail, but only if I create mailboxes for it using the PLESK software. All changes made directly to the configuration files of qmail are overwritten by the qmail configuration files that PLESK generated everytime something changes.

So mail CAN be accepted at our Plesk LINUX Webserver as Secondary MX but only when I have qmail put them in a mailbox

2) The CC part is important for the following fact: When I receive a mail in outlook where I wasn't in the TO field but in the CC field, I can see that. My name is in the CC part and not in the TO part. How is this replicated when a mail is forwarded by a script?

3) The internet is serving as a secondary MX for just a few hours. Some mailservers stop retrying after X times and some mailserver are stopping after X times, and then sending a 'I gave up' notice to the sender. That's no good is a server has crashed and has to be rebuild during the day.

4) I just want a script, that will scan a mailbox on our Plesk webserver, and if there is mail, see if the primary MX is accepting mail and then RECREATE THE ORIGINAL MESSAGE (should be able to, all heaers are intact, right?) and send it to the primary MX.

-----

Bottom line: is #4 possible or should I even go down the path of installing a seperate maildeamon on a different IP thata does allow the mail to be queued until delivery is possible? (option 2 is not going to happen, just wondering if that is the only option).

You have to imagine we are a domainhoster and have 100+ domains for which I would like to offer secondary MX services, but with BSTMP characteristics. Now all clients have to buy a pop3-downloader and that doesn't work all the time, especially with people in the BCC field.

Hi, thank you for your explanation. I think we agree about options and possibilities. What I want to do is impossible it seems, so I'm going to put this idea in a box until we have to need to implement it again. Just one more addition to make it clear what the bottleneck is about the proposed solution:

Plesk is an integrated management tool for hosting servers that had been integrated with the OS. It's closed source and we cannot just disable mail services and configure it manually. The qmail uses in this installation has been altered by Plesk and everytime we create a domain in Plesk for hosting, the configuration for qmail is also regenerated.

Our qmail therefore automatically accepts mail for domains created within Plesk, unless we disable mail for that domain, but then qmail is automatically configured to reject mail for it. So I have three options: put mail in a mailbox, forward it to another address or drop it. No option to manually alter it as Plesk will override that settings whenever this or another domain has changes in the mail config.

So in short: there is NO way I can start making manual changes to qmail without the danger of causing trouble for the whole plesk installation or risk of losing them....

About the secondary MX ("if the mail server is set up correctly as a secondary mail exchanger for....."), I think you are talking about the technical configuration of qmail so it can act as a secondary mx, I'm talking about settings DNS MX records so it is registered as a secondary MX. So your step is the one I can't take, my step is one I can as it's only DNS.

----

Thank you for brainstorming, It has been a pleasure. Points are for the effort!

My first google search on the subject turned this up, so I would say it's not impossible at all:

http://netweblogic.com/linux/mail/how-tobackup-mx-mail-server-plesk/

Best of luck to you in making this work.

Hmmm... a classic example of putting the right words in the searchbar ;-) So still no script but indeed an option to achieve what I want. Still not sure if the files will not be overwritten but we'll see if it will work. Thanks a lot!!