Solved

PHP: Process an incoming email and forward it to a mailing list

Posted on 2010-09-08
29
480 Views
Last Modified: 2012-06-27
Dear all,
sending a given message text to a list of users on a mailing list is easy using the mail() function.

But how would I set up a PHP script that RECEIVES an email at the address mailbot@mydomain.com (for example) and automatically dispatches its message to the list of current participants in a mailing list, which are stored in a MySQL table?

I'm currently working on a shared webspace. Can this still be done without having to resort to system ressources that are only available if you have root access?

Best,
Rob
0
Comment
Question by:robofix
  • 10
  • 9
  • 9
  • +1
29 Comments
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
The thing you want is called PIPE.  Ask your hosting provider for help setting this up.  Here is an example that works for me.  It should be easy to adapt this to use the data base instead of the hard-coded address on line 63.

Something that may save you some time... CRON jobs and PIPE jobs may not have the same PHP variables as regular web scripts.  Look out for dependencies on the values in $_SERVER, etc.

Good luck with it, ~Ray
#!/usr/bin/php -q
<?php // /email_pipe/index.php

// THIS IS AN EMAIL PIPE SCRIPT.
// THIS SCRIPT IS STARTED AUTOMATICALLY FOR EACH MESSAGE.
// NOTE THAT THIS SCRIPT IS ABOVE THE /public_html/ DIRECTORY TO PREVENT ACCIDENTAL EXECUTION


// --> HOW DO WE KNOW WHICH EMAIL MESSAGES GET SENT HERE?
// THIS SCRIPT RECEIVES MESSAGES SENT TO email_pipe@your.org
// CREATE AN EMAIL MAILBOX EXCLUSIVELY FOR AUTOMATED PROCESSING.
// SET UP AN EMAIL FORWARD FOR THAT MAILBOX IN cPANEL->EMAIL LIKE THIS:
// 1...5...10...15...20...25...
// |/home/{account}/email_pipe/index.php


// --> WHEN YOU UPLOAD, THIS SCRIPT WILL BE MARKED RW-R-R BUT THAT IS WRONG
// THIS SCRIPT MUST BE MARKED EXECUTABLE x0755
// YOU CAN USE FTP SOFTWARE TO CHMOD TO RWX-RX-RX


// --> NOTE THE FIRST LINE OF THIS SCRIPT MUST SAY #!/usr/bin/php -q STARTING IN COLUMN ONE
// 1...5...10...15...20...25...
// #!/usr/bin/php -q
// <?php ... PROGRAM CODE FOLLOWS


error_reporting(E_ALL);

// USE THE OUTPUT BUFFER - THIS DOES NOT HAVE BROWSER OUTPUT
ob_start();

// COLLECT THE INFORMATION HERE
$raw_email = '';


// TRY TO READ THE EMAIL FROM STDIN
if (!$stdin = fopen("php://stdin", "R"))
{
    echo "ERROR: UNABLE TO OPEN php://stdin \n";
}

// ABLE TO READ THE MAIL
else
{
    while (!feof($stdin))
    {
        $raw_email .= fread($stdin, 4096);
    }
    fclose($stdin);
}


// REMOVE MULTIPLE BLANKS - AND OTHER PROCESSING AS MIGHT BE NEEDED
$raw_email = preg_replace('| +|', ' ', $raw_email);


// SPEW WHAT WE GOT, IF ANYTHING, INTO THE OUTPUT BUFFER
var_dump($raw_email);

// CAPTURE THE OUTPUT BUFFER AND SEND IT VIA EMAIL
$buf = ob_get_contents();
mail ('you@your.org', 'EMAIL PIPE DATA', $buf);
ob_end_clean();

Open in new window

0
 
LVL 8

Expert Comment

by:shaunak
Comment Utility
In cpanel create a forwarder as mailbot@mydomain.com and it will ask you the destination. Select PIPE to a program and in that text box put:

php -q /home/user/www/pathtoscript.php

What ever mails are sent to mailbot@mydomain.com will be piped to your script and your script will parse the content of the mail.

Check this http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_24821459.html for a my script to parse the mail contents

0
 
LVL 26

Expert Comment

by:EddieShipman
Comment Utility
You can also create an IMAP listener. See some of my working code here:
http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_26360709.html

The code shown checks email in a cron-job and then opens the messages based on teh sender and handles zip attachments.
0
 
LVL 3

Author Comment

by:robofix
Comment Utility
@eddie:
ok but that works only for pre-defined email addresses with established mailboxes right?
I'd like to be able to pass arguments to the listener via the mail address itself, e.g., clients-501@mydomain.com, where I would address all clients of some 501st mailing list.
0
 
LVL 26

Expert Comment

by:EddieShipman
Comment Utility
What I use is an xml file that contains the email addresses I need to check in it, below is a sample. The entire script is also located below.

What the script does is it checks the IMAP INBOX for unseen messages and if any are from addresses in the XML file, it looks for attachments and saves them, then if the attachment is a zip file, it unzips it. It then adds a record to the file_uploads table so it can be processed by another script.

It then moves the message to the completed_uploads folder.

<?xml version="1.0" encoding="windows-1252"?>

<items>

  <item>

    <LI_email>charlie@myemail.com</LI_email>

    <LI_user_id>114</LI_user_id>

  </item>

  <item>

    <LI_email>helen@heremail.com</LI_email>

    <LI_user_id>114</LI_user_id>

  </item>

</items>



Full Script:

<?php



function showProgressMsg($msg)

{

  echo $msg."<br>";

  ob_flush();

	flush();

  usleep(2500);

}

  $mailpass="mailpass";

  $mailhost="{localhost/imap/ssl/novalidate-cert}";

  $mailuser="address@mysite.com";

  include "libs/database/db_mysql.php";

  $db = new Database;



  $debug=false;

  if($_GET['debug'] == 'yes') {

    $debug = true;

    ini_set('output_buffering', 'Off');

    ob_start();

  }

  

  if($debug) {showProgressMsg('Opening Mailbox');}

  

  $mailbox=imap_open($mailhost,$mailuser,$mailpass) or die("<br />\nFAILLED! ".imap_last_error());



  // The IMAP.xml file contains the email address and user_id of the users that we accept 

  // their files via email 

  if($debug) {showProgressMsg('Reading IMAP.xml');}

  $xml = simplexml_load_string(file_get_contents('IMAP.xml'));

  $result = $xml->xpath('item');

  while(list( , $node) = each($result)) {

    $email   = $node->LI_email;

    $user_id = $node->LI_user_id;

    $search  = "UNSEEN FROM \"$email\"";



    if($debug) {showProgressMsg('Searching for '.$email);}

    $result2 = imap_search($mailbox, $search);

    if($result2) {

      $index = $result2[0];

      $structure = imap_fetchstructure($mailbox, $index);

      

      $attachments = array();

      if(isset($structure->parts) && count($structure->parts)) {

        if($debug) {showProgressMsg('Handling attachments');}

        for($i = 0; $i < count($structure->parts); $i++) {

          $attachments[$i] = array(

            'is_attachment' => false,

            'filename' => '',

            'name' => '',

            'attachment' => '');

          

          if($structure->parts[$i]->ifdparameters) {

            foreach($structure->parts[$i]->dparameters as $object) {

              if(strtolower($object->attribute) == 'filename') {

                $attachments[$i]['is_attachment'] = true;

                $attachments[$i]['filename'] = $object->value;

              }

            }

          }

          

          if($structure->parts[$i]->ifparameters) {

            foreach($structure->parts[$i]->parameters as $object) {

              if(strtolower($object->attribute) == 'name') {

                $attachments[$i]['is_attachment'] = true;

                $attachments[$i]['name'] = $object->value;

              }

            }

          }

          

          if($attachments[$i]['is_attachment']) {

            $attachments[$i]['attachment'] = imap_fetchbody($mailbox, $index, $i+1, FT_PEEK);

            if($structure->parts[$i]->encoding == 3) { // 3 = BASE64

              $attachments[$i]['attachment'] = base64_decode($attachments[$i]['attachment']);

            }

            elseif($structure->parts[$i]->encoding == 4) { // 4 = QUOTED-PRINTABLE

              $attachments[$i]['attachment'] = quoted_printable_decode($attachments[$i]['attachment']);

            }

          }             

        } // for($i = 0; $i < count($structure->parts); $i++)

      } // if(isset($structure->parts) && count($structure->parts))



      // Now add a record into the file_upload table

      for($i = 0; $i < count($attachments); $i++) {

        if (strlen(trim($attachments[$i]['filename'])) > 0) {

          $path_parts = pathinfo($attachments[$i]['filename']);

          if($debug) {showProgressMsg('Processing '.$attachments[$i]['filename']);}

          if(strtolower($path_parts['extension']) == 'zip') {

            // I am going to do something different with ziped files            

            $filename = 'file_uploads/temp/'.$user_id.'_'.$path_parts['filename'].'_'.date('m_d_Y').'.'.$path_parts['extension'];

            $fp = fopen($filename, "w");

            fwrite($fp, $attachments[$i]['attachment']);

            fclose($fp);                       

            $zip = new ZipArchive();

            if ($zip->open($filename) !== TRUE) {

              die ('Could not open archive');

            }

            $zippedfile = $zip->statIndex(0);

            $path_parts = pathinfo($zippedfile['name']);

            $newfilename = 'file_uploads/'.$user_id.'_'.$path_parts['filename'].'_'.date('m_d_Y').'.'.$path_parts['extension'];

            $zip->extractTo('file_uploads/', $zippedfile['name']);

            $zip->close();           

            unlink($filename);

            rename('file_uploads/'.$zippedfile['name'], $newfilename);

            $filestr = preg_replace('`[\r\n]+`',"\n", file_get_contents($newfilename));

            $fp = fopen($newfilename, "w+");

            fwrite($fp, $filestr);

            fclose($fp);                       

          } else {

            $filename = 'file_uploads/'.$user_id.'_'.$path_parts['filename'].'_'.date('m_d_Y').'.'.$path_parts['extension'];

            $fp = fopen($filename, "w");

            $attachments[$i]['attachment'] = preg_replace('`[\r\n]+`',"\n",$attachments[$i]['attachment']);

            fwrite($fp, $attachments[$i]['attachment']);

            fclose($fp);

          }

          // remove the directory from the filename

          $filename = $user_id.'_'.$path_parts['filename'].'_'.date('m_d_Y').'.'.$path_parts['extension'];

          // Now insert into file_upload table

          $insert_sql = "INSERT INTO `file_upload` VALUES(NULL, $user_id, '$filename', 0, NOW(), 'N')";

          $db->query($insert_sql) or die("Can't insert record:".mysql_error());

        } // if (strlen(trim($attachments['name'])) > 0

      } // for($i = 0; $i < count($attachments); $i++)

      // This is a stop gap to circumvent the message being processed twice

      if($debug) {showProgressMsg('Setting \Seen Flag');}

      imap_setflag_full($mailbox, "$index", "\Seen");

      if($debug) {showProgressMsg('Moving Message from:'.$email);}

      // Now move the message to completed uploads mailbox

      // imap_mail_move copies the message and then sets the deleted flag.

      imap_mail_move($mailbox, "$index", "INBOX.completed uploads") or die("can't move: ".imap_last_error()); 

    } // if($result2)

    // Now, move the message to completed uploads 

  } // while(list( , $node) = each($result))

  if($debug) {showProgressMsg('clearing out moved messages and Closing mailbox');}

  // This clears out the deleted messages

  imap_expunge($mailbox);

  imap_close($mailbox);  

  ob_flush();

  flush();

  ob_end_clean();

  ini_set('output_buffering', 4096);

?>

Open in new window

0
 
LVL 3

Author Comment

by:robofix
Comment Utility
but mail addresses listed in the xml file have to be set up as separate IMAP inboxes beforehand, right?
0
 
LVL 26

Expert Comment

by:EddieShipman
Comment Utility
If you want to check the inboxes, yes. Why would they not already be setup?
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
That's one example of why I would prefer a PIPE instead of an IMAP listener - it decouples the logic of receiving the email from the logic of re-sending the email.  You can add new users to the list, wherever they are, whenever you want.
0
 
LVL 3

Author Comment

by:robofix
Comment Utility
@eddie:
I'd like to be able to pass arguments to the listening script via the mail address itself, e.g., clients-501@mydomain.com, while I DON'T NEED this to be an inbox at the same time. There might be thousands of possible "mail addresses" (in fact arguments for the script), but an inbox shouldn't have to be setup beforehand for each of these.
0
 
LVL 108

Accepted Solution

by:
Ray Paseur earned 300 total points
Comment Utility
Suggest you pass arguments in the Subject line or the Body of the email, or maybe in a BCC field, but not in the To address.  Your mail server will need to be able to route the email on the basis of the To address, and whether you use a PIPE or any other technique, you will need the To address to be consistent.

Of course you can set up several PIPE scripts with several different email addresses.  But the idea of using the email To address to carry variable arguments is not something I would recommend.  Email addresses do not work like a URL GET argument.
0
 
LVL 26

Expert Comment

by:EddieShipman
Comment Utility
I did that to update scores for a baseball league using basically the same script by using the subject.
0
 
LVL 3

Author Comment

by:robofix
Comment Utility
I basically want my users to be able to set up their own mailing lists and my script to be able to route all mass-mailings automatically, without manual intervention, like having to set up inboxes one by one, for each new mailing list. Further, I don't want to bother my users with specific rules on how to set up their mass mailings (like how they would have to put a specific code in the subject line). They should just be given an individual mail address they send their messages to, and the script at the other hand takes care of dispatching the message to all users in the mailing list.
Maybe the solution is to give them a control panel where they set up their mailing lists, and the PHP script automatically creates a new individual inbox - is that possible? do some webspace providers put limitations on setting up an IMAP inbox from within a php script?
0
 
LVL 26

Expert Comment

by:EddieShipman
Comment Utility
Some do, yes, but you have to realize that unless your host allows unlimited accounts, you will run into problems.
Even with the explanation above, I'm still not clear what it is you really want, how about you, Ray?

Can you give a little clearer scenario?

0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
I think you want to ask your hosting company about setting up an inbox from within a PHP script.  It can probably be done, but the details are going to be somewhat host-dependent.  From the sound of this new information, it sounds like you are trying to set up a relay for spam.  If that's NOT what you're trying to do, you need to think about how the system might be abused and put very tight controls on it (maybe put it behind HTTPS with authentication).  Are you trying to set up a service like Constant Contact?
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 3

Author Comment

by:robofix
Comment Utility
Ok, I'll try to give you guys a clearer scenario. Sorry if my statements were confusing.

Think of a college or university, where students select which courses they are about to take the following semester via a webpage. Let's say student A register for a Statistics course by supplying his name, student id number, mail address and so on. All information is saved in a SQL database. The system should give each teacher an individual mailing list address, for example stat101-mailinglist@domain.com, which he or she can use to quickly communicate with all enrolled students of his course. The process of setting up these mailing list addresses should be automated in the sense that the address is created at the same moment a specific course a set up by an admin through a front-end, or the teacher himself.
So the application should be able to create the mailing list email addresses by herself.

As a security measure in order to prevent spam attacks on any of these mailing lists, the script would of course have to check that the mail to be relayed stems from the teacher, an admin or any other legitimate person.

I hope this makes things a bit more tangible.
0
 
LVL 26

Assisted Solution

by:EddieShipman
EddieShipman earned 200 total points
Comment Utility
Well, what you are describing is EXACTLY what mailing lists are for.
I would check into using mailman: http://www.gnu.org/software/mailman/index.html

This is not trivial and would be better served by an application designed for the job.
0
 
LVL 3

Author Comment

by:robofix
Comment Utility
I just had a look at the mailman homepage. It looks like a very good and reliable piece of software, but maybe a bit overdimensioned for my needs, requiring to be installed on the webserver, configuring it and so on. I'm looking for an as simple as possible solution. Has anybody another idea?
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
Meetup and Google Groups implement this kind of functionality.  Here is what I might do.

Each teacher gets one pre-defined email address for each class she teaches.  Each of these addresses is a PIPE that is aware of the class it serves.

The data base coordinates the student's email address and the class matriculation.

The PIPE uses the data base to find all students who are in the class and relays its emails to the students.

There is some risk of tampering, since basic email is completely unauthenticated.  You might want to build into the PIPE some SPF or password authentication.  Checking the originating email address is good, but may not be sufficient to prevent mischief.  Look at all the headers in the email from the teachers and choose carefully.  Maybe consider using a secret password in the BCC field.

Another way to handle this is to give the teacher an HTML form for input of the email.  Then just repeat the contents to each of the student email addresses.  If the action script is password-protected you're probably safe.
0
 
LVL 3

Author Comment

by:robofix
Comment Utility
@ray: ok, does every pre-defined email address need to be an IMAP inbox, or is it just a PIPE? This is just a general question on how PIPEs work. As I understand it, a PIPE mail address is an email alias being forwarded to a PHP script, so there is no inbox? Please correct me if I'm wrong.
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
The PIPE needs to have an associated POP inbox (at least on my hosting service) and SMTP is not required.  As I understand the process, the internet MX records route the mail to the local server.   The local MX records route the mail to the mail server.  The mail server knows about the PIPE script, as well as all of the regular inboxes.
0
 
LVL 3

Author Comment

by:robofix
Comment Utility
thx for the explanation. do you know if it is technically possible to create a pop inbox from within a php script?
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
That's a good question for the hosting company or the guys who run your servers.  I have always used the cPanel or equivalent.  So I assume it is possible, but I don't have a code example that shows how to do it in PHP.  This link might be helpful:
http://us.php.net/manual/en/function.imap-createmailbox.php
0
 
LVL 26

Expert Comment

by:EddieShipman
Comment Utility
Well, why not have forwarders setup for each class email to each student's inbox, instead?
IMHO PIPES are overkill for what you want to do here.

Also, if you have a cPanel host, they usually have mailman capable of being installed in the extra packages and it is
just a simple little configuration.
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
Yeah, forwarders might be the most useful answer.  The original question was, "how would I set up a PHP script that RECEIVES an email..." but upon further clarification, it does not look like there is any need to receive the email - it's simply to make sure the email gets routed to several different mail boxes.  Forwarders are fine for that.

Of course there is the question of maintenance and garbage collection, but that is a different matter.
0
 
LVL 3

Assisted Solution

by:robofix
robofix earned 0 total points
Comment Utility
Each class has between 30 and 200 students, so I don't see myself  (or the admin) writing every student's mail address one by one into the KAS system (similar to cpanel) my webspace provider offers. An incoming mail by a professor should be relayed to all students having opted for his course according to the database .
I have had a chat with the tech support of my provider, and they don't offer PIPEs, but each mailbox can have custom filters written in Procmail language. According to them and some google research, Procmail allows to pipe an incoming message to a script.
My plan of the moment is the following: Put some Procmail commands into my CATCH ALL mailbox that automatically relay incoming messages exhibting some predefined pattern (e.g. list-class_xxx@mydomain.com) to a php script. If this worked out, I would finally have realized what my initial goal was - pass arguments to the script via the To-address.
0
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
You can pass arguments to a script via the To-address if you want, and you can drive nails with a wrench if you want.  From a computer-science perspective those are roughly equivalent.

Why not tell the students to sign up for the teacher's emails?  They have a very high incentive to do this for themselves.  You could just use a Google Group.  We do this for the Washington, DC PHP Users Group and it works perfectly.  I am sure it would work for your application as well.
0
 
LVL 26

Expert Comment

by:EddieShipman
Comment Utility
You can use the code I provided to build a forwarder, you just need to have some way to get a list of names for each class, whether it be in XML or a database.
You would just have to setup a mailbox for each class and then the forwarder script would
open each email in that box and "forward" it to each student in the list.
0
 
LVL 26

Expert Comment

by:EddieShipman
Comment Utility
Also, at least in cPanel, you can setup forwarders based on a list you can build in a text file.
0
 
LVL 26

Expert Comment

by:EddieShipman
Comment Utility
BTW, I wasn't aware of this open source mailing list software but it looks promising:
http://www.phplist.com/
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Join & Write a Comment

Using SQL Scripts we can save all the SQL queries as files that we use very frequently on our database later point of time. This is one of the feature present under SQL Workshop in Oracle Application Express.
Although it can be difficult to imagine, someday your child will have a career of his or her own. He or she will likely start a family, buy a home and start having their own children. So, while being a kid is still extremely important, it’s also …
Viewers will get an overview of the benefits and risks of using Bitcoin to accept payments. What Bitcoin is: Legality: Risks: Benefits: Which businesses are best suited?: Other things you should know: How to get started:
The viewer will learn how to count occurrences of each item in an array.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now