Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.
One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.
Messages Not Being Delivered
To whomever might help,
Problem: I'm having many many emails that log as dsn=2.0.0, stat=Sent (Ok: queued as CA29A4FC7D) or dsn=2.0.0, stat=Sent ( <21431207.1095024967457.Ja
Relevant Facts:
- Sendmail 8.12 on Linux. Java creates the messages and sends them to sendmail. The box has 2 new Xeon procs and 2 gigs of RAM.
- A nightly email sends out and is delivered successfully to all legitimate addresses. Sends about 8,000 text only emails. I'm a little Linux ignorant, but I use "dir | wc -l" to get a file count in the queue (var/spool/mqueue) which reads about 220 at the height of the nightly run.
- The problem run occurs once a week on weekends, and sends out approx. 80,000 emails. The messages are multipart, half text and half HTML, and each weigh about 50k. I took a count as above in the queue, and it peeked around 1,590.
- AOL(once I got on their whitelist), Yahoo, Comcast and other fast ISPs get delivered to ok on the weekend.
- I sort by filename, and sort the mail in Java before it's delivered to Sendmail by email address (NOT by host).
- I mention fast ISPs, because I noticed that with at least on smaller domain, the addresses early in the alphabet go through, but the later ones don't. In contrast to that, my comcast address, although beginning with "r", which is after the smaller domains stop delivering, goes through fine.
- Hotmail, as far as I can tell, does not go through even to my Hotmail address that begins with a "d". As stated above, hotmail shows as dsn=2.0.0, stat=Sent ( <21431207.1095024967457.Ja
- On the nightly run, I see my Hotmail address as queued, but I receive the email almost immediately, every time.
I believe it's a queue saturation problem, or maybe even messages getting dropped from the queue. I'm very familiar with both the sendmail.mc and the sendmail.cf files, having crawled through them more than a few times, so I can tell you what various settings are, if you need to know.
Thank You. This one's very important, as it's a final problem in the list of more than a few.
Problem: I'm having many many emails that log as dsn=2.0.0, stat=Sent (Ok: queued as CA29A4FC7D) or dsn=2.0.0, stat=Sent ( <21431207.1095024967457.Ja
Relevant Facts:
- Sendmail 8.12 on Linux. Java creates the messages and sends them to sendmail. The box has 2 new Xeon procs and 2 gigs of RAM.
- A nightly email sends out and is delivered successfully to all legitimate addresses. Sends about 8,000 text only emails. I'm a little Linux ignorant, but I use "dir | wc -l" to get a file count in the queue (var/spool/mqueue) which reads about 220 at the height of the nightly run.
- The problem run occurs once a week on weekends, and sends out approx. 80,000 emails. The messages are multipart, half text and half HTML, and each weigh about 50k. I took a count as above in the queue, and it peeked around 1,590.
- AOL(once I got on their whitelist), Yahoo, Comcast and other fast ISPs get delivered to ok on the weekend.
- I sort by filename, and sort the mail in Java before it's delivered to Sendmail by email address (NOT by host).
- I mention fast ISPs, because I noticed that with at least on smaller domain, the addresses early in the alphabet go through, but the later ones don't. In contrast to that, my comcast address, although beginning with "r", which is after the smaller domains stop delivering, goes through fine.
- Hotmail, as far as I can tell, does not go through even to my Hotmail address that begins with a "d". As stated above, hotmail shows as dsn=2.0.0, stat=Sent ( <21431207.1095024967457.Ja
- On the nightly run, I see my Hotmail address as queued, but I receive the email almost immediately, every time.
I believe it's a queue saturation problem, or maybe even messages getting dropped from the queue. I'm very familiar with both the sendmail.mc and the sendmail.cf files, having crawled through them more than a few times, so I can tell you what various settings are, if you need to know.
Thank You. This one's very important, as it's a final problem in the list of more than a few.
Who is Participating?
To figure out what's happening here I need to see the all records in maillog for the queue ID of one of the messages to Hotmail. I'd expect there to be at least two records in the mail log associated with that message. Grepping for the local queue ID of the message will pull them all out.
To comment in addition to agreeing with PsiCop 100% (I'm also a mail admin =) one of the weights we have assigned heavily on spam scoring is HTML_TEXT_WITHUNBALANCEDTA
I know nothing about hotmail's anti-spam back end, but I can tell you that if you hit any of our mailservers with what you are describing you would probably get a handful of message through and then the rest would be blocked. Consider configuring multiple outbound queues and running persistant background queue runners, throttling your outbound load and delivering fewer recipients per message.
I know nothing about hotmail's anti-spam back end, but I can tell you that if you hit any of our mailservers with what you are describing you would probably get a handful of message through and then the rest would be blocked. Consider configuring multiple outbound queues and running persistant background queue runners, throttling your outbound load and delivering fewer recipients per message.
Oh yeah, I left one small detail out. Make sure your HTML has balanced tags. A hallmark of HTML spam is that the HTML is crap. =)
To all,
I appreciate the general feedback and PsiCop's soapbox preaching; if only the world were that easy, that any one person would have control over such matters. I am but a developer brought in by company x, for the project of converting an existing email service that took over 14 hours to send out the 80,000 plus emails. I can make recommendations about changing the multipart to either text or HTML, but that's as far as I can go.
Here's where I am at this point:
1) I added some information to the header, such as errors-to, and wrapped the HTML in actaul <HTML><BODY></BODY></HTML>
2) I'm recommending that they change the emails to text only for now, for speed and for better clearance through firewalls. I will use some of your quotes to help make my case. Allowing people to opt for text or pure HTML is the best approach, but that will also require changes to their web site, which is out of the scope of this project.
I read in the bat book that one good idea for running mail quickly through sendmail is to instantiate it with two different configurations, one that process the newly outgoing messages, only waiting a few seconds for each SMTP communication, and another configuration that process the queue, with more standard wait times. What do you think about that approach?
I appreciate the general feedback and PsiCop's soapbox preaching; if only the world were that easy, that any one person would have control over such matters. I am but a developer brought in by company x, for the project of converting an existing email service that took over 14 hours to send out the 80,000 plus emails. I can make recommendations about changing the multipart to either text or HTML, but that's as far as I can go.
Here's where I am at this point:
1) I added some information to the header, such as errors-to, and wrapped the HTML in actaul <HTML><BODY></BODY></HTML>
2) I'm recommending that they change the emails to text only for now, for speed and for better clearance through firewalls. I will use some of your quotes to help make my case. Allowing people to opt for text or pure HTML is the best approach, but that will also require changes to their web site, which is out of the scope of this project.
I read in the bat book that one good idea for running mail quickly through sendmail is to instantiate it with two different configurations, one that process the newly outgoing messages, only waiting a few seconds for each SMTP communication, and another configuration that process the queue, with more standard wait times. What do you think about that approach?
When I've set up mail systems for mass mailers I use an aged queueng system to speed up delivery and reduce the load on the MTA. The default delivery queue is processed with a Sendmail instance that has short timeouts on the order of 5 seconds. Each message in that queue is tried at least once and if it can't be immediately delivered the message is moved to a second queue. That queue is processed with normal timeouts and is run at 30 minute intervals. Each message in that queue is tried twice before moving to a third queue that's run at less frequent intervals, and so forth util the message is delivered or 4 days elapse.
I think the multiple-instantiation idea is worth a try. It will give you more granular control over delivery.
To all,
I thank you for your participation. I've yet to solve this one, which is partially why I haven't tended to come back here, and the fact that I only run the software once a week, so testing is a long, long (did I mention long?) process.
I thank you for your participation. I've yet to solve this one, which is partially why I haven't tended to come back here, and the fact that I only run the software once a week, so testing is a long, long (did I mention long?) process.
Question has a verified solution.
Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.
Have a better answer? Share it in a comment.
All Courses
From novice to tech pro — start learning today.
-
Course of the Month8 days, 2 hours left to enrollMonitoring and Operating a Private Cloud with System Center 2012 R2 273 lessons
Next, lotsa HTML is one of the things that SPAM filters look for. If what you have to say is really worth reading, you don't need HTML. A link or two is fine, but much more than that and SPAM filters tend to perk up. Same for multi-part messages that are just repeats (MIMEdefang/SpamAssassin has specific filters for that garbage).
Make sure your mailhost is properly registered in DNS, with a reverse-lookup record, and is listed as a Mail eXchanger.
Since you seem to be able to send out smaller amounts of E-mail fine, consider breaking the 80K honker into smaller jobs spread over several hours. You can pretty much guarantee that Hotmail throttles connections after X number of RCPT TO: requests during the SMTP conversation. And you can also guarantee that Hotmail's servers are configured to keep tabs on who sends them how much E-Mail, and regard lotsa E-Mail from one place in a short time as a spammer.
If Hotmail offers its users a white-list feature (I have no idea, never used Hotmail), encourage your subscribers to white-list you.
And quit sending text part/html part duplicate E-Mails.