Build an archiving server

It can be done with Postfix VERY easily.

All you need to do is use the "always_bcc=archive@localhost" parameter where archive@localhost is a mailbox that will receive the BCC'd messages.

Be warned though that this form or archive is not very easy to search through and you will truly be better off using either specialist kit (read email archiving software/server) or a cloud service (caveat - I work for Mimecast who provide a cloud based archive)

depending on your requirements you can easily determine what the best course of action for you is.

What kind of message volumes do you handle and what sort of user count are you?
I ask as the most critical thing to consider when doing this yourself is what your scalability is.

Oh yes, MAKE SURE YOU SET POSTFIX TO DELIVER TO A SERVICE THAT USES MAILDIR NOT MBOX.
Dovecot can easily use Maildir instead of mbox, just that mbox files lock and cause massive disk i/o issues when they get too large.

How about using actual mail archiving software? I use MailArchiva - http://www.mailarchiva.com/ - (there is a free open-souce edition) that works well. You don't need to direct mail to it either. I have it set to download copies of mail from a journal mailbox setup on Exchange 2003.

always_bcc is as close to journalling as you are gonna get on Linux :)

What I am really interested in doing is setting up this topography:

[Internet] -----> [archiver] -----> [Exchange]

What's the name of the open source package?

The Open Source Edition of MailArchiver that jar3817 is talking about only supports message journalling in Exchange 2000 and 2003, it does not support envelope journaling of Exchange 2007 and 2010.

This setup of retrieving data out of stream works well, it adds less complexity to your local environment.

Do you have a particular reason for wanting it to be inline?

I ask as at Mimecast we do it inline as we store alot of rich meta data around delivery and transmission events for non-repudiation as well as providing AV/AS, but a basic system won't do that, so you lose the benefit of it.

Actually it does support 2007 and 2010 if you set it up as an IMAP client to the journal mailbox on the exchange server. With that setup it'll work with ANY email server.

I have been looking at MailArchiva. It seems to be pretty solid. The one thing I want to be able to do is re-deliver mail. Does it have that functionlaity?

I was looking at the demo, which allows me to export all the .eml files as a zip file, which is fine, but not sure how to import that into Outlook.

I was looking for an inline solution because I am trying to come up with a third line of defense. Our current scheme is this:

Email is delivered directly to exchange. If exchange is down, then it rolls over to an alternate server, and when exchange comes back up, the pop3 connector goes and gets the mail.

Backups are done routinely on the exchange box. If a crash happens in teh no-man's land of between backups and "now" then those mails are potentially lost. Now, normally, we rely on the client's cached copies of the mail server, which has worked fine for years.

...util...

One of my users decide to 1) switch to Mac, and 2) not use Entourage.

So, she lost mail this week because the exchange box crashed. Everything from the backup until "now" is lost for her because she's using iMail or some other BS client that only looks at the webmail. She has no cached copies.

Thus, her 1.2GB backup of her mailbox is fine, but there are about 6-10 emails that were lost that had important information.

Had she been using Outlook, this would be a moot point.

But users will be users. Thus, I need an archiving solution to deal with customers who don't follow directions.

I hate doing product punts (which is why I caveat that I work for Mimecast) but it sounds to me that a more holistic approach including continuity archiving and additional security is exactly what you need.

Go check out Mimecast - www.mimecast.com - and have a look, we provide everything leaving you with just an Exchange server to manage. This greatly reduces your complexity, therefore your risk, and in most cases, your overall costs too.

Unfortunately, our security policies, which are audited by the government, require all communications storage to be done in house. So a 3rd part solution is not really in the cards.

Thus... why I was thinking I could do this with postfix.

It seems it would be simpel to do:

mail comes in, gets stored in mysql db, then is forwarded on using send mail and the original headers. That's really all I want...

The problem with that (and the reason there are so many successful archive products) is that is is not so easy to do this with a robust scalable system.

The Postfix always_bcc feature will do exactly what youneed, you can use postfix to send mail to an API and use mySQL as the receiving app. you wil have to build the DB first though...

will always bcc maintain the original headers?

I just got to thinking about it, it seems I could use fetchmail to relay...

bcc adds a recipient and that recipient is removed from the headers, so yes, it does.

and if you use fetchmail PLEASE PLEASE PLEASE use Maildir - I cannot stress this enough.
Even medium volume on an mbox file will kill your server over a short space of time.

if I use bcc, what happens when I try to re-deliver a message? Is it forwarded or is it delivered with the original headers?