Solved

PHP: Process an incoming email and forward it to a mailing list

Posted on 2010-09-08
29
505 Views
Last Modified: 2012-06-27
Dear all,
sending a given message text to a list of users on a mailing list is easy using the mail() function.

But how would I set up a PHP script that RECEIVES an email at the address mailbot@mydomain.com (for example) and automatically dispatches its message to the list of current participants in a mailing list, which are stored in a MySQL table?

I'm currently working on a shared webspace. Can this still be done without having to resort to system ressources that are only available if you have root access?

Best,
Rob
0
Comment
Question by:robofix
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 10
  • 9
  • 9
  • +1
29 Comments
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 33631451
The thing you want is called PIPE.  Ask your hosting provider for help setting this up.  Here is an example that works for me.  It should be easy to adapt this to use the data base instead of the hard-coded address on line 63.

Something that may save you some time... CRON jobs and PIPE jobs may not have the same PHP variables as regular web scripts.  Look out for dependencies on the values in $_SERVER, etc.

Good luck with it, ~Ray
#!/usr/bin/php -q
<?php // /email_pipe/index.php

// THIS IS AN EMAIL PIPE SCRIPT.
// THIS SCRIPT IS STARTED AUTOMATICALLY FOR EACH MESSAGE.
// NOTE THAT THIS SCRIPT IS ABOVE THE /public_html/ DIRECTORY TO PREVENT ACCIDENTAL EXECUTION


// --> HOW DO WE KNOW WHICH EMAIL MESSAGES GET SENT HERE?
// THIS SCRIPT RECEIVES MESSAGES SENT TO email_pipe@your.org
// CREATE AN EMAIL MAILBOX EXCLUSIVELY FOR AUTOMATED PROCESSING.
// SET UP AN EMAIL FORWARD FOR THAT MAILBOX IN cPANEL->EMAIL LIKE THIS:
// 1...5...10...15...20...25...
// |/home/{account}/email_pipe/index.php


// --> WHEN YOU UPLOAD, THIS SCRIPT WILL BE MARKED RW-R-R BUT THAT IS WRONG
// THIS SCRIPT MUST BE MARKED EXECUTABLE x0755
// YOU CAN USE FTP SOFTWARE TO CHMOD TO RWX-RX-RX


// --> NOTE THE FIRST LINE OF THIS SCRIPT MUST SAY #!/usr/bin/php -q STARTING IN COLUMN ONE
// 1...5...10...15...20...25...
// #!/usr/bin/php -q
// <?php ... PROGRAM CODE FOLLOWS


error_reporting(E_ALL);

// USE THE OUTPUT BUFFER - THIS DOES NOT HAVE BROWSER OUTPUT
ob_start();

// COLLECT THE INFORMATION HERE
$raw_email = '';


// TRY TO READ THE EMAIL FROM STDIN
if (!$stdin = fopen("php://stdin", "R"))
{
    echo "ERROR: UNABLE TO OPEN php://stdin \n";
}

// ABLE TO READ THE MAIL
else
{
    while (!feof($stdin))
    {
        $raw_email .= fread($stdin, 4096);
    }
    fclose($stdin);
}


// REMOVE MULTIPLE BLANKS - AND OTHER PROCESSING AS MIGHT BE NEEDED
$raw_email = preg_replace('| +|', ' ', $raw_email);


// SPEW WHAT WE GOT, IF ANYTHING, INTO THE OUTPUT BUFFER
var_dump($raw_email);

// CAPTURE THE OUTPUT BUFFER AND SEND IT VIA EMAIL
$buf = ob_get_contents();
mail ('you@your.org', 'EMAIL PIPE DATA', $buf);
ob_end_clean();

Open in new window

0
 
LVL 8

Expert Comment

by:shaunak
ID: 33665763
In cpanel create a forwarder as mailbot@mydomain.com and it will ask you the destination. Select PIPE to a program and in that text box put:

php -q /home/user/www/pathtoscript.php

What ever mails are sent to mailbot@mydomain.com will be piped to your script and your script will parse the content of the mail.

Check this http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_24821459.html for a my script to parse the mail contents

0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 33795016
You can also create an IMAP listener. See some of my working code here:
http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/Q_26360709.html

The code shown checks email in a cron-job and then opens the messages based on teh sender and handles zip attachments.
0
Simple, centralized multimedia control

Watch and learn to see how ATEN provided an easy and effective way for three jointly-owned pubs to control the 60 televisions located across their three venues utilizing the ATEN Control System, Modular Matrix Switch and HDBaseT extenders.

 
LVL 3

Author Comment

by:robofix
ID: 33908542
@eddie:
ok but that works only for pre-defined email addresses with established mailboxes right?
I'd like to be able to pass arguments to the listener via the mail address itself, e.g., clients-501@mydomain.com, where I would address all clients of some 501st mailing list.
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 33909507
What I use is an xml file that contains the email addresses I need to check in it, below is a sample. The entire script is also located below.

What the script does is it checks the IMAP INBOX for unseen messages and if any are from addresses in the XML file, it looks for attachments and saves them, then if the attachment is a zip file, it unzips it. It then adds a record to the file_uploads table so it can be processed by another script.

It then moves the message to the completed_uploads folder.

<?xml version="1.0" encoding="windows-1252"?>
<items>
  <item>
    <LI_email>charlie@myemail.com</LI_email>
    <LI_user_id>114</LI_user_id>
  </item>
  <item>
    <LI_email>helen@heremail.com</LI_email>
    <LI_user_id>114</LI_user_id>
  </item>
</items>

Full Script:
<?php

function showProgressMsg($msg)
{
  echo $msg."<br>";
  ob_flush();
	flush();
  usleep(2500);
}
  $mailpass="mailpass";
  $mailhost="{localhost/imap/ssl/novalidate-cert}";
  $mailuser="address@mysite.com";
  include "libs/database/db_mysql.php";
  $db = new Database;

  $debug=false;
  if($_GET['debug'] == 'yes') {
    $debug = true;
    ini_set('output_buffering', 'Off');
    ob_start();
  }
  
  if($debug) {showProgressMsg('Opening Mailbox');}
  
  $mailbox=imap_open($mailhost,$mailuser,$mailpass) or die("<br />\nFAILLED! ".imap_last_error());

  // The IMAP.xml file contains the email address and user_id of the users that we accept 
  // their files via email 
  if($debug) {showProgressMsg('Reading IMAP.xml');}
  $xml = simplexml_load_string(file_get_contents('IMAP.xml'));
  $result = $xml->xpath('item');
  while(list( , $node) = each($result)) {
    $email   = $node->LI_email;
    $user_id = $node->LI_user_id;
    $search  = "UNSEEN FROM \"$email\"";

    if($debug) {showProgressMsg('Searching for '.$email);}
    $result2 = imap_search($mailbox, $search);
    if($result2) {
      $index = $result2[0];
      $structure = imap_fetchstructure($mailbox, $index);
      
      $attachments = array();
      if(isset($structure->parts) && count($structure->parts)) {
        if($debug) {showProgressMsg('Handling attachments');}
        for($i = 0; $i < count($structure->parts); $i++) {
          $attachments[$i] = array(
            'is_attachment' => false,
            'filename' => '',
            'name' => '',
            'attachment' => '');
          
          if($structure->parts[$i]->ifdparameters) {
            foreach($structure->parts[$i]->dparameters as $object) {
              if(strtolower($object->attribute) == 'filename') {
                $attachments[$i]['is_attachment'] = true;
                $attachments[$i]['filename'] = $object->value;
              }
            }
          }
          
          if($structure->parts[$i]->ifparameters) {
            foreach($structure->parts[$i]->parameters as $object) {
              if(strtolower($object->attribute) == 'name') {
                $attachments[$i]['is_attachment'] = true;
                $attachments[$i]['name'] = $object->value;
              }
            }
          }
          
          if($attachments[$i]['is_attachment']) {
            $attachments[$i]['attachment'] = imap_fetchbody($mailbox, $index, $i+1, FT_PEEK);
            if($structure->parts[$i]->encoding == 3) { // 3 = BASE64
              $attachments[$i]['attachment'] = base64_decode($attachments[$i]['attachment']);
            }
            elseif($structure->parts[$i]->encoding == 4) { // 4 = QUOTED-PRINTABLE
              $attachments[$i]['attachment'] = quoted_printable_decode($attachments[$i]['attachment']);
            }
          }             
        } // for($i = 0; $i < count($structure->parts); $i++)
      } // if(isset($structure->parts) && count($structure->parts))

      // Now add a record into the file_upload table
      for($i = 0; $i < count($attachments); $i++) {
        if (strlen(trim($attachments[$i]['filename'])) > 0) {
          $path_parts = pathinfo($attachments[$i]['filename']);
          if($debug) {showProgressMsg('Processing '.$attachments[$i]['filename']);}
          if(strtolower($path_parts['extension']) == 'zip') {
            // I am going to do something different with ziped files            
            $filename = 'file_uploads/temp/'.$user_id.'_'.$path_parts['filename'].'_'.date('m_d_Y').'.'.$path_parts['extension'];
            $fp = fopen($filename, "w");
            fwrite($fp, $attachments[$i]['attachment']);
            fclose($fp);                       
            $zip = new ZipArchive();
            if ($zip->open($filename) !== TRUE) {
              die ('Could not open archive');
            }
            $zippedfile = $zip->statIndex(0);
            $path_parts = pathinfo($zippedfile['name']);
            $newfilename = 'file_uploads/'.$user_id.'_'.$path_parts['filename'].'_'.date('m_d_Y').'.'.$path_parts['extension'];
            $zip->extractTo('file_uploads/', $zippedfile['name']);
            $zip->close();           
            unlink($filename);
            rename('file_uploads/'.$zippedfile['name'], $newfilename);
            $filestr = preg_replace('`[\r\n]+`',"\n", file_get_contents($newfilename));
            $fp = fopen($newfilename, "w+");
            fwrite($fp, $filestr);
            fclose($fp);                       
          } else {
            $filename = 'file_uploads/'.$user_id.'_'.$path_parts['filename'].'_'.date('m_d_Y').'.'.$path_parts['extension'];
            $fp = fopen($filename, "w");
            $attachments[$i]['attachment'] = preg_replace('`[\r\n]+`',"\n",$attachments[$i]['attachment']);
            fwrite($fp, $attachments[$i]['attachment']);
            fclose($fp);
          }
          // remove the directory from the filename
          $filename = $user_id.'_'.$path_parts['filename'].'_'.date('m_d_Y').'.'.$path_parts['extension'];
          // Now insert into file_upload table
          $insert_sql = "INSERT INTO `file_upload` VALUES(NULL, $user_id, '$filename', 0, NOW(), 'N')";
          $db->query($insert_sql) or die("Can't insert record:".mysql_error());
        } // if (strlen(trim($attachments['name'])) > 0
      } // for($i = 0; $i < count($attachments); $i++)
      // This is a stop gap to circumvent the message being processed twice
      if($debug) {showProgressMsg('Setting \Seen Flag');}
      imap_setflag_full($mailbox, "$index", "\Seen");
      if($debug) {showProgressMsg('Moving Message from:'.$email);}
      // Now move the message to completed uploads mailbox
      // imap_mail_move copies the message and then sets the deleted flag.
      imap_mail_move($mailbox, "$index", "INBOX.completed uploads") or die("can't move: ".imap_last_error()); 
    } // if($result2)
    // Now, move the message to completed uploads 
  } // while(list( , $node) = each($result))
  if($debug) {showProgressMsg('clearing out moved messages and Closing mailbox');}
  // This clears out the deleted messages
  imap_expunge($mailbox);
  imap_close($mailbox);  
  ob_flush();
  flush();
  ob_end_clean();
  ini_set('output_buffering', 4096);
?>

Open in new window

0
 
LVL 3

Author Comment

by:robofix
ID: 33909521
but mail addresses listed in the xml file have to be set up as separate IMAP inboxes beforehand, right?
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 33912582
If you want to check the inboxes, yes. Why would they not already be setup?
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 33912649
That's one example of why I would prefer a PIPE instead of an IMAP listener - it decouples the logic of receiving the email from the logic of re-sending the email.  You can add new users to the list, wherever they are, whenever you want.
0
 
LVL 3

Author Comment

by:robofix
ID: 33913225
@eddie:
I'd like to be able to pass arguments to the listening script via the mail address itself, e.g., clients-501@mydomain.com, while I DON'T NEED this to be an inbox at the same time. There might be thousands of possible "mail addresses" (in fact arguments for the script), but an inbox shouldn't have to be setup beforehand for each of these.
0
 
LVL 110

Accepted Solution

by:
Ray Paseur earned 300 total points
ID: 33913446
Suggest you pass arguments in the Subject line or the Body of the email, or maybe in a BCC field, but not in the To address.  Your mail server will need to be able to route the email on the basis of the To address, and whether you use a PIPE or any other technique, you will need the To address to be consistent.

Of course you can set up several PIPE scripts with several different email addresses.  But the idea of using the email To address to carry variable arguments is not something I would recommend.  Email addresses do not work like a URL GET argument.
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 33914367
I did that to update scores for a baseball league using basically the same script by using the subject.
0
 
LVL 3

Author Comment

by:robofix
ID: 33915170
I basically want my users to be able to set up their own mailing lists and my script to be able to route all mass-mailings automatically, without manual intervention, like having to set up inboxes one by one, for each new mailing list. Further, I don't want to bother my users with specific rules on how to set up their mass mailings (like how they would have to put a specific code in the subject line). They should just be given an individual mail address they send their messages to, and the script at the other hand takes care of dispatching the message to all users in the mailing list.
Maybe the solution is to give them a control panel where they set up their mailing lists, and the PHP script automatically creates a new individual inbox - is that possible? do some webspace providers put limitations on setting up an IMAP inbox from within a php script?
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 33916135
Some do, yes, but you have to realize that unless your host allows unlimited accounts, you will run into problems.
Even with the explanation above, I'm still not clear what it is you really want, how about you, Ray?

Can you give a little clearer scenario?

0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 33917621
I think you want to ask your hosting company about setting up an inbox from within a PHP script.  It can probably be done, but the details are going to be somewhat host-dependent.  From the sound of this new information, it sounds like you are trying to set up a relay for spam.  If that's NOT what you're trying to do, you need to think about how the system might be abused and put very tight controls on it (maybe put it behind HTTPS with authentication).  Are you trying to set up a service like Constant Contact?
0
 
LVL 3

Author Comment

by:robofix
ID: 33917741
Ok, I'll try to give you guys a clearer scenario. Sorry if my statements were confusing.

Think of a college or university, where students select which courses they are about to take the following semester via a webpage. Let's say student A register for a Statistics course by supplying his name, student id number, mail address and so on. All information is saved in a SQL database. The system should give each teacher an individual mailing list address, for example stat101-mailinglist@domain.com, which he or she can use to quickly communicate with all enrolled students of his course. The process of setting up these mailing list addresses should be automated in the sense that the address is created at the same moment a specific course a set up by an admin through a front-end, or the teacher himself.
So the application should be able to create the mailing list email addresses by herself.

As a security measure in order to prevent spam attacks on any of these mailing lists, the script would of course have to check that the mail to be relayed stems from the teacher, an admin or any other legitimate person.

I hope this makes things a bit more tangible.
0
 
LVL 26

Assisted Solution

by:EddieShipman
EddieShipman earned 200 total points
ID: 33918014
Well, what you are describing is EXACTLY what mailing lists are for.
I would check into using mailman: http://www.gnu.org/software/mailman/index.html

This is not trivial and would be better served by an application designed for the job.
0
 
LVL 3

Author Comment

by:robofix
ID: 33918212
I just had a look at the mailman homepage. It looks like a very good and reliable piece of software, but maybe a bit overdimensioned for my needs, requiring to be installed on the webserver, configuring it and so on. I'm looking for an as simple as possible solution. Has anybody another idea?
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 33920061
Meetup and Google Groups implement this kind of functionality.  Here is what I might do.

Each teacher gets one pre-defined email address for each class she teaches.  Each of these addresses is a PIPE that is aware of the class it serves.

The data base coordinates the student's email address and the class matriculation.

The PIPE uses the data base to find all students who are in the class and relays its emails to the students.

There is some risk of tampering, since basic email is completely unauthenticated.  You might want to build into the PIPE some SPF or password authentication.  Checking the originating email address is good, but may not be sufficient to prevent mischief.  Look at all the headers in the email from the teachers and choose carefully.  Maybe consider using a secret password in the BCC field.

Another way to handle this is to give the teacher an HTML form for input of the email.  Then just repeat the contents to each of the student email addresses.  If the action script is password-protected you're probably safe.
0
 
LVL 3

Author Comment

by:robofix
ID: 33920102
@ray: ok, does every pre-defined email address need to be an IMAP inbox, or is it just a PIPE? This is just a general question on how PIPEs work. As I understand it, a PIPE mail address is an email alias being forwarded to a PHP script, so there is no inbox? Please correct me if I'm wrong.
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 33920166
The PIPE needs to have an associated POP inbox (at least on my hosting service) and SMTP is not required.  As I understand the process, the internet MX records route the mail to the local server.   The local MX records route the mail to the mail server.  The mail server knows about the PIPE script, as well as all of the regular inboxes.
0
 
LVL 3

Author Comment

by:robofix
ID: 33920461
thx for the explanation. do you know if it is technically possible to create a pop inbox from within a php script?
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 33920480
That's a good question for the hosting company or the guys who run your servers.  I have always used the cPanel or equivalent.  So I assume it is possible, but I don't have a code example that shows how to do it in PHP.  This link might be helpful:
http://us.php.net/manual/en/function.imap-createmailbox.php
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 33922502
Well, why not have forwarders setup for each class email to each student's inbox, instead?
IMHO PIPES are overkill for what you want to do here.

Also, if you have a cPanel host, they usually have mailman capable of being installed in the extra packages and it is
just a simple little configuration.
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 33923455
Yeah, forwarders might be the most useful answer.  The original question was, "how would I set up a PHP script that RECEIVES an email..." but upon further clarification, it does not look like there is any need to receive the email - it's simply to make sure the email gets routed to several different mail boxes.  Forwarders are fine for that.

Of course there is the question of maintenance and garbage collection, but that is a different matter.
0
 
LVL 3

Assisted Solution

by:robofix
robofix earned 0 total points
ID: 33923525
Each class has between 30 and 200 students, so I don't see myself  (or the admin) writing every student's mail address one by one into the KAS system (similar to cpanel) my webspace provider offers. An incoming mail by a professor should be relayed to all students having opted for his course according to the database .
I have had a chat with the tech support of my provider, and they don't offer PIPEs, but each mailbox can have custom filters written in Procmail language. According to them and some google research, Procmail allows to pipe an incoming message to a script.
My plan of the moment is the following: Put some Procmail commands into my CATCH ALL mailbox that automatically relay incoming messages exhibting some predefined pattern (e.g. list-class_xxx@mydomain.com) to a php script. If this worked out, I would finally have realized what my initial goal was - pass arguments to the script via the To-address.
0
 
LVL 110

Expert Comment

by:Ray Paseur
ID: 33923608
You can pass arguments to a script via the To-address if you want, and you can drive nails with a wrench if you want.  From a computer-science perspective those are roughly equivalent.

Why not tell the students to sign up for the teacher's emails?  They have a very high incentive to do this for themselves.  You could just use a Google Group.  We do this for the Washington, DC PHP Users Group and it works perfectly.  I am sure it would work for your application as well.
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 33923751
You can use the code I provided to build a forwarder, you just need to have some way to get a list of names for each class, whether it be in XML or a database.
You would just have to setup a mailbox for each class and then the forwarder script would
open each email in that box and "forward" it to each student in the list.
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 33923753
Also, at least in cPanel, you can setup forwarders based on a list you can build in a text file.
0
 
LVL 26

Expert Comment

by:EddieShipman
ID: 33965407
BTW, I wasn't aware of this open source mailing list software but it looks promising:
http://www.phplist.com/
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

An enjoyable and seamless user experience can go a long way on an eCommerce site. While a cohesive layout and engaging copy play roles in creating a positive user experience, some sites neglect aspects that seem marginal but in actuality prove very …
Developer portfolios can be a bit of an enigma—how do you present yourself to employers without burying them in lines of code?  A modern portfolio is more than just work samples, it’s also a statement of how you work.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
Learn how to create flexible layouts using relative units in CSS.  New relative units added in CSS3 include vw(viewports width), vh(viewports height), vmin(minimum of viewports height and width), and vmax (maximum of viewports height and width).

726 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question