Solved

Record of all email bounce backs

Posted on 2014-04-24
14
556 Views
Last Modified: 2014-04-24
Hi Guys,
I'm sure you've run into requests like this before so I'm hoping you have an easy answer for me.

I'm trying to create a record of all email bounce backs that occur when I am sending email via PHP & MySQL.

I only need the "to" and "from" fields. Nothing else, because all the data I need will be in those two fields.

So far I have created a catchall email address where each bounce back will be sent.
Then it is piped to a PHP script which (and this is where I'm stumped) will give me the contents of the "to" and "from" fields.

Once I have that data I can do all sorts of things with it, like keep a db record of it or delete the email address from the db.

I've spent days researching this on the Internet. I've found lots of talk and lots of scripts, but each PHP script I've found falls short in that they do not include or demonstrate a way to actually read the email headers to provide contents of the "to" and "from" fields.

So please, help me with a PHP script that will just read & provide the info I need from an email piped to it.
Thanks for the help,
Sas
0
Comment
Question by:sasnaktiv
  • 6
  • 5
  • 2
  • +1
14 Comments
 
LVL 16

Expert Comment

by:Enphyniti
ID: 40020715
Please provide a scrubbed example of the output piped to your PHP script.  (IE: pipe a test to a file, clear out private information, and post it here.)

I don't want to make any assumptions about the header info you are receiving.

Essentially, you need regexps to parse to and from based off that information in the piped input.  For example, your tell you script:
1. when you see "To:" give me everything you find until the first space and throw away the rest of the string. (since smtp addresses cannot have spaces.)
2. when you see "from:" do the same thing.
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 40020779
Try installing this for your pipe script.  Then show us the exact output you get when you send it an email message.  You will need to put your own email address into the mail command at line 60.  The pipe script will send you an email message that contains all of the headers and text that it received.

#!/usr/bin/php -q
<?php
/**
 * /email_pipe/index.php
 * DATED: 2009-05-12
 *
 * THIS IS AN EMAIL PIPE SCRIPT.
 * THIS SCRIPT IS STARTED AUTOMATICALLY FOR EACH MESSAGE.
 * NOTE THAT THIS SCRIPT IS ABOVE THE /public_html/ DIRECTORY TO PREVENT ACCIDENTAL EXECUTION
 *
 * --> HOW DO WE KNOW WHICH EMAIL MESSAGES GET SENT HERE?
 * THIS SCRIPT RECEIVES MESSAGES SENT TO email_pipe@your.org
 * CREATE AN EMAIL MAILBOX EXCLUSIVELY FOR AUTOMATED PROCESSING.
 * SET UP AN EMAIL FORWARD FOR THAT MAILBOX IN cPANEL->EMAIL LIKE THIS:
 * 1...5...10...15...20...25...
 * |/home/{account}/email_pipe/index.php
 *
 * --> WHEN YOU UPLOAD, THIS SCRIPT WILL BE MARKED RW-R-R BUT THAT IS WRONG
 * THIS SCRIPT MUST BE MARKED EXECUTABLE x0755
 * YOU CAN USE FTP SOFTWARE TO CHMOD TO RWX-RX-RX
 *
 * --> NOTE THE FIRST LINE OF THIS SCRIPT MUST SAY #!/usr/bin/php -q STARTING IN COLUMN ONE
 * 1...5...10...15...20...25...
 * #!/usr/bin/php -q
 * <?php ... PROGRAM CODE FOLLOWS
 */
error_reporting(E_ALL);

// USE THE OUTPUT BUFFER - THIS DOES NOT HAVE BROWSER OUTPUT
ob_start();

// COLLECT THE INFORMATION HERE
$raw_email = NULL;

// TRY TO READ THE EMAIL FROM STDIN
if (!$stdin = fopen("php://stdin", "R"))
{
    echo 'ERROR: UNABLE TO OPEN php://stdin' . PHP_EOL;
}

// ABLE TO READ THE MAIL
else
{
    while (!feof($stdin))
    {
        $raw_email .= fread($stdin, 4096);
    }
    fclose($stdin);
}


// REMOVE MULTIPLE BLANKS - AND OTHER PROCESSING AS MIGHT BE NEEDED
$raw_email = preg_replace('/ +/', ' ', $raw_email);

// SPEW WHAT WE GOT, IF ANYTHING, INTO THE OUTPUT BUFFER
var_dump($raw_email);

// CAPTURE THE OUTPUT BUFFER AND SEND IT TO SOMEONE ELSE VIA EMAIL
$buf = ob_get_contents();
mail ('you@your.org', 'EMAIL PIPE DATA', $buf);

// PREVENT ANY BROWSER OUTPUT - MAY CAUSE ERROR RESPONSES AND BOUNCED MESSAGES
ob_end_clean();

Open in new window

0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 40020822
Now on to the nasty part of the problem.
Once I have that data I can do all sorts of things with it, like keep a db record of it or delete the email address from the db.
Yes, about the only thing you cannot do is actually rely on the internet mail system to provide you with bounced messages.  Most mail programs will simply throw bad messages away.  There is no end-user value to bouncing a message.  Think about it... Why would I take a piece of junk mail that I did not want, wrap it up and put a stamp on it, and send it back to the originator with a message that said, "please don't send me this any more?"  It's easy and inexpensive to just throw the junk mail in the recycle bin.  It's also easy and inexpensive to just discard the unwanted email messages.

Why would a mail recipient want to avoid bouncing messages?  One example occurred in about 2006, IIRC.  A hacker sent bogus "unsubscribe" messages to thousands of bulletin-board forums.  The messages had a return address of an email honey-pot at a dumb-as-rocks blacklist like Spamhaus.  The forum software responded to these messages with a "You are not subscribed" message, but since the message was sent to a honey-pot, the honey-pot provider blacklisted the domains of thousands of bulletin-board forums.  Unfortunately, there were a great many anti-spam systems that relied on the blacklist!  It took the industry months to straighten the mess out.  

The clear solution to this cleverly engineered vandalism?  Discard, but do not bounce messages.

If you're still sending broadcast email messages you might want to learn more about these folks.  They are dependable and embarrassingly inexpensive.
http://www.constantcontact.com/index.jsp
0
 
LVL 1

Author Comment

by:sasnaktiv
ID: 40020854
Thanks Ray,
I'm in the process of working with the script you provided.
Also, ConstantContact will actually be a kind of competitor to what we are creating.
More later as I work with your script.
Thanks,
Sas
0
 
LVL 34

Expert Comment

by:gr8gonzo
ID: 40020874
There's a good reason that you're not finding an easy solution. Your request is similar to asking for a car seat that will fit every car model from every manufacturer, of every year. It's an impossible request because all of those different models have different standards.

Mail is one of the least-standards-driven processes out there. There ARE standards but unfortunately there are a LOT of mail servers that do not obey those standards or interpret them differently, and many mail servers try different methods to combat spam and so they handle bounces differently. Here are some of the common scenarios you have to account for in order to detect all failures:

1. Mail server thinks the message is spam, so it:
    a. Quietly deletes the message without informing the sender.
    b. Rejects the message and tells the sender immediately in the same network session/call (no separate bounce message is sent).
    c. Accepts the message and sends a separate bounce message.

2. Mail server accepts the message but initial delivery fails, so it:
    a. Quietly deletes the message.
    b. Tries to keep delivering (in case a mailbox is full), and eventually (usually 1-7 days) will send a separate bounce message.
    c. Rejects the message and tells the sender in the same network call (no separate bounce message)

3. Mail server is down, so a backup receives the message and holds it until the primary mail server comes up (indeterminate amount of time) , and then the above 2 scenarios apply.

Now, when a separate bounce message IS sent, it can be in one of many different types of formats. Again, there ARE standards for bounce back formats, but different mail servers interpret those standards differently. In some cases, you might get a pretty straightforward bounceback with all of the details in the content of that message, and in other cases, the bounceback might have an attachment with the necessary info.

So that said, just doing a simple regex is not going to always capture the information, and I have yet to see an open-source script that has an accurate processing engine for the majority of the formats, but I haven't been looking, either. You might want to check phpclasses.org to see if there's a class out there that might help with the processing of the bounced messages, but you'll have to also make sure you're checking the results of your mail sending attempt, too.
0
 
LVL 1

Author Comment

by:sasnaktiv
ID: 40020997
Okay Ray, thanks.
I got your script to work for me, and it indeed does return the email content to me by email so I know it's working.
But it's giving me EVERYTHING!
I only want to isolate the "to & from" information so that I can either POST or GET the $variableTO and $variableFROM to another PHP script where I can update the db.
How can we do that?
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 40021068
Sas, please go back and read what I posted earlier.  

...show us the exact output you get...

I can't write scripts without test data.  I tried that once about 20 years ago, and now I know better.  So please copy the information that the pipe script sent to you and post it in the code snippet.  Then I can work with it and show you how to reduce it to a manageable data set.
0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 16

Expert Comment

by:Enphyniti
ID: 40021101
But scrub it for sensitive data first.  IE: change your email addresses and IP address information before posting the output.
0
 
LVL 1

Author Comment

by:sasnaktiv
ID: 40021103
Okay Ray below is the email message that I get.
I've altered all the email addresses for security.
I hope this is what you were expecting.
Sas

string(2118) "From Sas@sas.com Thu Apr 24 11:53:38 2014
Received: from smtp.xxxx.com ([xx.sss.dd.rr]:52513)
	by nobo.arvixevps.com with esmtps (TLSv1:DHE-RSA-AES256-SHA:256)
	(Exim 4.82)
	(envelope-from <Sas@sas.com>)
	id 1WdOmA-0004lh-JR
	for bounceback@XXXXXXXXXXXXX.com; Thu, 24 Apr 2014 11:53:38 -0700
X_CMAE_Category: , ,
X-CNFS-Analysis: v=2.0 cv=d493OGfE c=1 sm=1 a=hdgldoAMznI0pJFYVsDwcg==:17 a=fDG7_uJK7BwA:10 a=1xk3CJkCSsIA:10 a=8nJEP1OIZ-IA:10 a=N-TlcrVUAAAA:8 a=ChM806crZ64ElltR0-wA:9 a=wPNLvfGTeEIA:10 a=_W_S_7VecoQA:10 a=AfkkXffo6PgA:10 a=ZwvnQMdMGjAA:10 a=hdgldoAMznI0pJFYVsDwcg==:117
X-CM-Score: 0
X-Scanned-by: Cloudmark Authority Engine
X-Authed-Username: Vml0a2Fuc2FzQFBIQVJNQUNBTEwuaW5mbw==
Authentication-Results: smtp02.xxx.cmh.syneacor.com header.from=Sas@sas.com; sender-id=neutral
Authentication-Results: smtp02.xxx.cmh.syneacor.com smtp.mail=Sas@sas.com; spf=neutral; sender-id=neutral
Authentication-Results: smtp02.xxx.cmh.syneacor.com smtp.user=sas@sas.info; auth=pass (PLAIN)
Received-SPF: neutral (smtp02.rcn.cmh.syneacor.com: 000.00.000.00 is neither permitted nor denied by domain of sas.com)
Received: from [000.00.000.00] ([000.00.000.00:xxx36] helo=EAST-intelMac-746.local)
	by smtp.rcn.com (envelope-from <Sas@sas.com>)
	(ecelerity 3.5.1.37854 r(Momo-dev:3.5.1.0)) with ESMTPA
	id E9/40-64819-1xxxxx535; Thu, 24 Apr 2014 14:53:38 -0400
Message-ID: <53595DB2.3000707@sas.com>
Disposition-Notification-To: Sas <Sas@sas.com>
Date: Thu, 24 Apr 2014 14:53:38 -0400
From: Sas <Sas@sas.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:15.0) Gecko/20120907 Thunderbird/15.0.1
MIME-Version: 1.0
To: bounceback@XXXXXXXXXXXXX.com
Subject: Hi Ray
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<html>
 <head>

 <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
 </head>
 <body bgcolor="#FFFFFF" text="#000000">
 <font face="Arial">Thanks for the help Ray.<br>
 It's much appreciated.<br>
 Sas<br>
 </font>
 </body>
</html>

"

Open in new window

0
 
LVL 108

Accepted Solution

by:
Ray Paseur earned 500 total points
ID: 40021272
Please see http://iconoun.com/demo/temp_sasnaktiv.php where the moving parts start on line 55.

<?php // demo/temp_sasnaktiv.php
echo "<pre>" . PHP_EOL;

/**
 * SEE: http://www.experts-exchange.com/Programming/Languages/Scripting/PHP/Q_28419396.html#a40021103
 */

// INFORMATION FROM THE PIPE SCRIPT
$eml = <<<EOD
From Sas@sasnaktiv.com Thu Apr 24 11:53:38 2014
Received: from smtp.rcn.com ([69.168.97.78]:52513)
	by nobo.arvixevps.com with esmtps (TLSv1:DHE-RSA-AES256-SHA:256)
	(Exim 4.82)
	(envelope-from <Sas@sasnaktiv.com>)
	id 1WdOmA-0004lh-JR
	for bounceback@bizigogo.biz; Thu, 24 Apr 2014 11:53:38 -0700
X_CMAE_Category: , ,
X-CNFS-Analysis: v=2.0 cv=d493OGfE c=1 sm=1 a=hdgldoAMznI0pJFYVsDwcg==:17 a=fDG7_uJK7BwA:10 a=1xk3CJkCSsIA:10 a=8nJEP1OIZ-IA:10 a=N-TlcrVUAAAA:8 a=ChM806crZ64ElltR0-wA:9 a=wPNLvfGTeEIA:10 a=_W_S_7VecoQA:10 a=AfkkXffo6PgA:10 a=ZwvnQMdMGjAA:10 a=hdgldoAMznI0pJFYVsDwcg==:117
X-CM-Score: 0
X-Scanned-by: Cloudmark Authority Engine
X-Authed-Username: Vml0a2Fuc2FzQFBIQVJNQUNBTEwuaW5mbw==
Authentication-Results: smtp02.rcn.cmh.synacor.com header.from=Sas@sasnaktiv.com; sender-id=neutral
Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.mail=Sas@sasnaktiv.com; spf=neutral; sender-id=neutral
Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.user=sas@sasnaktiv.info; auth=pass (PLAIN)
Received-SPF: neutral (smtp02.rcn.cmh.synacor.com: 207.38.179.88 is neither permitted nor denied by domain of sasnaktiv.com)
Received: from [207.38.179.88] ([207.38.179.88:51836] helo=EAST-intelMac-746.local)
	by smtp.rcn.com (envelope-from <Sas@sasnaktiv.com>)
	(ecelerity 3.5.1.37854 r(Momo-dev:3.5.1.0)) with ESMTPA
	id E9/40-64819-1BD59535; Thu, 24 Apr 2014 14:53:38 -0400
Message-ID: <53595DB2.3000707@sasnaktiv.com>
Disposition-Notification-To: Sas <Sas@sasnaktiv.com>
Date: Thu, 24 Apr 2014 14:53:38 -0400
From: Sas <Sas@sasnaktiv.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:15.0) Gecko/20120907 Thunderbird/15.0.1
MIME-Version: 1.0
To: bounceback@XXXXXXXXXXXXX.com
Subject: Hi Ray
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<html>
 <head>

 <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
 </head>
 <body bgcolor="#FFFFFF" text="#000000">
 <font face="Arial">Thanks for the help Ray.<br>
 It's much appreciated.<br>
 Sas<br>
 </font>
 </body>
</html>
EOD;

// SHOW HOW TO ISOLATE THE 'FROM' AND 'TO'
$arr = explode('Date:', $eml);
$eml = $arr[1];

$arr = explode('From: ', $eml);
$arr = explode(PHP_EOL, $arr[1]);
$frm = trim($arr[0]);

$arr = explode('To: ', $eml);
$arr = explode(PHP_EOL, $arr[1]);
$to  = trim($arr[0]);

// SHOW THE ISOLATED INFORMATION
echo PHP_EOL . 'FROM: ' . htmlentities($frm) . ', TO: ' . htmlentities($to);

Open in new window

0
 
LVL 1

Author Comment

by:sasnaktiv
ID: 40021298
Looks good Ray!
How does it apply to your original code?
#!/usr/bin/php -q
<?php
/**
 * /email_pipe/index.php
 * DATED: 2009-05-12
 *
 * THIS IS AN EMAIL PIPE SCRIPT.
 * THIS SCRIPT IS STARTED AUTOMATICALLY FOR EACH MESSAGE.
 * NOTE THAT THIS SCRIPT IS ABOVE THE /public_html/ DIRECTORY TO PREVENT ACCIDENTAL EXECUTION
 *
 * --> HOW DO WE KNOW WHICH EMAIL MESSAGES GET SENT HERE?
 * THIS SCRIPT RECEIVES MESSAGES SENT TO email_pipe@your.org
 * CREATE AN EMAIL MAILBOX EXCLUSIVELY FOR AUTOMATED PROCESSING.
 * SET UP AN EMAIL FORWARD FOR THAT MAILBOX IN cPANEL->EMAIL LIKE THIS:
 * 1...5...10...15...20...25...
 * |/home/{account}/email_pipe/index.php
 *
 * --> WHEN YOU UPLOAD, THIS SCRIPT WILL BE MARKED RW-R-R BUT THAT IS WRONG
 * THIS SCRIPT MUST BE MARKED EXECUTABLE x0755
 * YOU CAN USE FTP SOFTWARE TO CHMOD TO RWX-RX-RX
 *
 * --> NOTE THE FIRST LINE OF THIS SCRIPT MUST SAY #!/usr/bin/php -q STARTING IN COLUMN ONE
 * 1...5...10...15...20...25...
 * #!/usr/bin/php -q
 * <?php ... PROGRAM CODE FOLLOWS
 */
error_reporting(E_ALL);

// USE THE OUTPUT BUFFER - THIS DOES NOT HAVE BROWSER OUTPUT
ob_start();

// COLLECT THE INFORMATION HERE
$raw_email = NULL;

// TRY TO READ THE EMAIL FROM STDIN
if (!$stdin = fopen("php://stdin", "R"))
{
    echo 'ERROR: UNABLE TO OPEN php://stdin' . PHP_EOL;
}

// ABLE TO READ THE MAIL
else
{
    while (!feof($stdin))
    {
        $raw_email .= fread($stdin, 4096);
    }
    fclose($stdin);
}



// REMOVE MULTIPLE BLANKS - AND OTHER PROCESSING AS MIGHT BE NEEDED
$raw_email = preg_replace('/ +/', ' ', $raw_email);

// SPEW WHAT WE GOT, IF ANYTHING, INTO THE OUTPUT BUFFER
var_dump($raw_email);

// CAPTURE THE OUTPUT BUFFER AND SEND IT TO SOMEONE ELSE VIA EMAIL
$buf = ob_get_contents();

mail ('ray@thePowerful.com', 'From /email_pipe/ray.php',$buf);

// PREVENT ANY BROWSER OUTPUT - MAY CAUSE ERROR RESPONSES AND BOUNCED MESSAGES
ob_end_clean();
                                            

Open in new window

0
 
LVL 108

Assisted Solution

by:Ray Paseur
Ray Paseur earned 500 total points
ID: 40021327
The contents of the $eml variable in the new sample script is the information that would be found in the $raw_email variable in the teaching example of the pipe script.  So take the moving parts of the code from the new script and apply that to the pipe.  The data you want to record is in $frm and $to.
0
 
LVL 1

Author Closing Comment

by:sasnaktiv
ID: 40021451
Homage to you oh Most Powerful One!
My utmost thanks & respect,
Sas
0
 
LVL 108

Expert Comment

by:Ray Paseur
ID: 40021533
Ha!  Thanks for the points, Sas.  All the best, ~Ray
0

Featured Post

What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

Join & Write a Comment

Introduction Many web sites contain image galleries; a common design for these galleries includes a page with a collection of thumbnail images.  You can click on each of the thumbnail images to see the larger version of the image.  This is easily i…
I imagine that there are some, like me, who require a way of getting currency exchange rates for implementation in web project from time to time, so I thought I would share a solution that I have developed for this purpose. It turns out that Yaho…
Learn how to match and substitute tagged data using PHP regular expressions. Demonstrated on Windows 7, but also applies to other operating systems. Demonstrated technique applies to PHP (all versions) and Firefox, but very similar techniques will w…
The viewer will learn how to dynamically set the form action using jQuery.

708 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now