We host email elsewhere (VPS-POSTFIX) but going to GMAIL spam. OK Everywhere else

We have some '.co' domains for our company and recently decided to host email ourselves on one of the domains.  We tried sending email to all major email providers like outlook,  yahoo.. etc.  All of these emails go to regular inbox,  except in the case of GMail.
This is not bulk email but rather individual emails sent from our staff to customers or each other.  
Some important things we made sure work 1000%:
1. Our VPS IP is not in blacklist ...and it shows good reputation by every blacklist company we could find.
2. SPF has been set up correctly
3. DKIM has been set up correctly
4. This domain was verified through postmaster.google.com
5. This domain was verified through google.com/webmaster
6 these settings have been in place for over a month,  and we have been trying to get thus work for over half a year to no avail.
7.Bulk guidelines are met despite the fact that we don't send bulk emails.
Our clients are not receiving their invoices or communication from us unless they look in spam folder,  this is insane since 80% of our clients use gmail
can someone advice what will be best possible solution.
LVL 15
Ess KayEntrapenuerAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Dillyn BarberCIOCommented:
2 possibilities I can think of:
1. a few people marked them as spam so google now sees it as spam
2. the emails itself look spammy, try making the invoice email more unique when it comes to the title and text, google will mark it is spam if it is very generic and has a similar body and title to emails it has marked as spam.
Serena HsiMarketing ConsultantCommented:
Ask your Gmail customers to add your email domain to their address book; or setup a custom filter with the setting 'Never send it to Spam'

For Google App for Work users, whitelist by adding IP domains:
Ess KayEntrapenuerAuthor Commented:
@Serena, Thats not feasible. Possibly a solution for a mom and pops store or your blog but  we have a 50k mailing list.  So far we been testing with individual emails so until it works we cannot start sending our newsletters. Currently these mail are being sent (and received into inbox) using a 3rd party,  but we are migrating the mailbox to our local servers.
 The second part of your suggestion is irrelevant as we do not use google apps (see title)

@Dillyn. this is a new email address  we haven't used before.  the ip address was never used either.  We set everything up and its been couple months so I think we are passed the 'wait for it to propegate'  period
IT Pros Agree: AI and Machine Learning Key

We’d all like to think our company’s data is well protected, but when you ask IT professionals they admit the data probably is not as safe as it could be.

Serena HsiMarketing ConsultantCommented:
When I ramped up a new mail server to send to 500k recipients, I didn't dump the whole thing in one go. I ramped it up over time: 1k, 5k, 10k, 50k, etc. Those smaller broadcasts were essentially load tests. Then we dropped a newsletter to our entire subscriber list.

Asking your subscribers to whitelist you isn't a mom'n'pop solution; it's a standard business practice for sending B2B and B2C emails to customers.
Ess KayEntrapenuerAuthor Commented:
The type of customers we have does not bode well with that strategy.  Furthermore,  we are currently sending mail from the old system.  the new emails,  which will come from the new domain will not be transitioned to sent to customers until its certain to work.

At the moment we have a handful of gmail addresses for testing,  and they always end up in spam. if we whitelist an address,  we cannot test that address anymore,  so we set up a new mailbox,  which also goes to spam. I was hoping there is something i missed that may fix this other than calling up our customers to tell them their bill is now in the spam folder
Scott Fell, EE MVEDeveloper & EE ModeratorCommented:
You said you are not sending bulk but sending out invoices.  Are you sending these invoices automatically "from the server" or is a liver person attaching a pdf to an email.

Assuming you are creating the email invoices on the fly and sending them from your own server, you are more likely to get into the spam folders.  

More than likely you are on a shared IP service where you can't control what others do.  This will work against you.  If you have control over your sending ip, you will need time.  It is not the type of thing you can turn on or off.  Email is one of those things that seems easy but that is not the case.

When you are sending out transactional emails like invoices, your best bet is to use a 3rd party smtp service like sendgrid.com or mandrill.com and make sure you use the paid version.  I have used both and when I used mandrill when it was free, I never had a complaint about clients emails going to spam.  Sendgrid has an inexpensive level with a shared ip and a $80/monthly level that has a dedicated ip and whitelisting.

While there are always ways to "do it yourself" and less expensive, your dealing with paying customers and this type of service and others like it are worth it.  

I used to host email and send these types of transactional emails myself and spent time getting whitelisted.  It's not worth the effort.
Ess KayEntrapenuerAuthor Commented:
At the moment we have to-date sent only manually typed emails to our test accounts

No automatic,  no invoices.  just simple test mails

The ip is private not shared.  our server runs on one ip with two domains

The Google message is "similar to messages detected by our spam filters"
Scott Fell, EE MVEDeveloper & EE ModeratorCommented:
What are the messages you are getting in postmaster tools?  If your email is being sent to google spam filter, there must be something there.

If you are sending out one off's by one person where you are at about 1 per minute, that should satisfy sending limits.  If you are sending attachments without boxy text that could be a flag.

More than likely what has happened is you sent an invoice to an angry or ex client and they marked your email as spam.  It only takes a very small percentage of recipients to mark your messages as spam for the rest to show up.  Based on the error you mention, google sees essentially the same mail coming from the same place that has been marked as spam.

When I build email systems, I have opt-in flags for each of newsletters, transactional mail and statements/invoices.  

If you are sending out more than a few thousand dollars of invoices each year, it does pay to spend $10 a month or more on the 3rd party smtp service.   The reason you may get better delivery rates is because of relationships they have with ISP's because of their own sending policy's.  Also, when you send via something like sendgrid or mandril, you can get reports and api call backs as to who wanted off the list or marked your mail as spam.

Look at your postmater tools for any additional clues.  Pick out a couple of good customers and ask them if you can test sending via a different service.  Sign up for sendgrid's free trial and you can set up one of your outlook accounts to send via sendgrid.  See if that helps. https://sendgrid.com/docs/Integrate/Mail_Clients/outlook_2013.html
Ess KayEntrapenuerAuthor Commented:
Postmaster tools on google shows nothing wrong

Here is an example of headers, no attachments

Delivered-To: s***@gmail.com
Received: by with SMTP id g184cs134021ywh;
        Sat, 10 Sep 2016 12:04:02 -0700 (PDT)
X-Received: by with SMTP id l64mr3836606wml.105.1473534242059;
        Sat, 10 Sep 2016 12:04:02 -0700 (PDT)
Return-Path: <s**@h**.co>
Received: from mail.h**.co (mail.h**.co. [**.**.**.**])
        by mx.google.com with ESMTPS id fj4si8392351wjb.181.2016.
        for <s**@gmail.com>
        (version=TLS1_2 cipher=***256 bits=128/128);
        Sat, 10 Sep 2016 12:04:01 -0700 (PDT)
Received-SPF: pass (google.com: domain of s**@h**.co designates **.**.**.** as permitted sender) client-ip=**.**.**.**;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@h**.co;
       spf=pass (google.com: domain of s**@h**.co designates **.**.**.** as permitted sender) smtp.mailfrom=s**@h**.co
Received: from mail.h**.co (localhost [])
	by mail.h**.co (Postfix) with ESMTP id 6**425
	for <s**@gmail.com>; Sat, 10 Sep 2016 19:03:26 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=h**.co;
	s=mail; t=1473334206;
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
Content-Transfer-Encoding: 7bit
Date: Sat, 10 Sep 2016 15:03:26 -0400
From: s**@h**.co
To: s**@gmail.com
Subject: s.  you are late
Message-ID: <29614818e192762bd384a3c58ff54205@h**.co>
X-Sender: s**@h**.co
User-Agent: Roundcube Webmail/1.1.3

please update with time


Open in new window

example of tests
Scott Fell, EE MVEDeveloper & EE ModeratorCommented:
If you send a one off email to these same people is it marked as spam?  

Try adding a semi personal note and switch up the subject line because the message reads that the reason for going to spam is content related.  If enough people marked these same emails as spam, then any future recipients will have also go to spam.
Ess KayEntrapenuerAuthor Commented:
i only tested single emails.  we only  have so many gmails to  play with.

Thats just an example,  trust me,  there are plenty legitimate looking ones

we have NOT sent anything to clients so,  NO ONE marked it as spam
Dillyn BarberCIOCommented:
Try checking the format of your emails. Here is Google's guidelines on format for sending to gmail users:
  • All messages must be formatted according to RFC 5322 and, if using HTML, HTML standards.
  • Messages must have a valid 'Message-ID:' header field.
  • Messages should indicate that they are bulk mail, using the 'Precedence: bulk' header field.
  • Attempts to hide the true sender of the message or the true landing page for any web links in the message may result in non-delivery.
  • The subject of each message should be relevant to the body's content and not be misleading.
  • The authenticating domain, envelope From domain, payload From domain, reply-to domain, and sender domain should not violate the highly-restrictive Unicode Security Profile guidelines for international domain names.
Ess KayEntrapenuerAuthor Commented:
Did that before posting here

Its #7 in the first post
The ip is private not shared.  our server runs on one ip with two domains
My spam filter KILLS mail from third-party mailing services, so my experience is in some ways diametrically opposed to @Scott Fells.

I get annoyed with folks who get conned into using third-party mailers and then wonder why I don't reply to their mail.  Relayed mail dies.  Period.  Every decent spam filter does that, and most mailer company's make the mistake of taking on a spammer for a customer and get blacklisted, even the best of them.  But your mileage may vary.

But I think you may need to check into your DNS and most especially your whitelisting more carefully.
ISPs generally have ranges of IPs that are 'dynamic' and 'static'
There are blacklists out there that kill mail coming from 'dynamic' IP ranges.
SORBS certainly does that
I say 'dynamic' because my ISP hands out 'static' IPs from their dynamic range by setting a MAC address reservation.  The IP has handed out via DHCP but I get the same one every time.

Initially, it was hard to get those blacklists to accept that my IP really was static.
Run it multiple times on multiple days -- because some lists don't respond every thing.
You absolutely have to be clean across the board on that tool.

I did have to do the manual things on their (dynamic ip blocking lists) sites to get a human override.
And it got fouled up by a bot, once.
After that, I set my firewall to ensure that only the Exchange server or printer/scanners could send out on port 25.
Been good since.
Ess KayEntrapenuerAuthor Commented:
its a dedicated ip.  no 3rd party anything.  we lease a clean ip which in my investigation has never been used before us.
Scott Fell, EE MVEDeveloper & EE ModeratorCommented:
> so my experience is in some ways diametrically opposed to @Scott Fells.

When using legit services such as mandrill or sendgrid, you are authenticating the domain with SPF and DKIM.  If you send from mydomain.com via your email client such as outlook it will be sent as if it is from any mail server be it hosted on your web server or a separate mail server.  My own experience may differ, but the chance of mail making it to the inbox via  a service like this is far greater than sending from the webserver.  Deliver-ability may not always be 100%, but it is greater than sending from the server.

Also, these are invoices, not advertising.

After reading this thread again, you said you only have a handful of gmails to test.  Perhaps the issue is you have been doing testing on a few gmails and not responding,opening?

What happens if you send a regular email to somebody else that has gmail?  With a different message?    The issue could very well only be with your testing environment.
And have you run the tests at
several times over a couple day span?
MXToolbox queries 103 separate rbls, so you usually don't get complete responses in any one run.
Your IP may indeed be clean, but as the link I posted indicates, if your ISP got the block of IPs and indicated they intended to use them as dynamic at the time they were assigned, it will make little difference that they have given you a static address in that block now.
Whole blocks get blacklisted.

I'd ask for the IP or domain name, but that's always unwise to post.
You can PM to provide that if you wish.

'Our server runs two domains with one IP'
Are you then encountering the possibility that rDNS is finding the one domain on that IP when the mail is coming from the second domain?

Are the settings for BOTH domains set so that SPF etc for each domain is granted to the other?

And your test emails:
please update with time


was that the entirety of the message.
Overly brief, no salutation and malformed HTML can all get flagged as spam.

To a test account, try

Hi <real name of the account holder>

We've been having a hell of a time testing our new system
Things seem to be going south for some unknown reason.
I figured I'd drop you a note to see how you were

<your real name>
<your outfit>
<your email address>
<Your phone number>

P.S.  Reply when you get this

Message content does matter.
Craft a message that has details inside it that a spammer can't know.
Send it from an address that has already been replied to by the gmail account
(one reply usually frees the spam filter)

Then send a similar message to a gmail account that HASN'T yet replied to that message.
Let us know what the results are.
Ess KayEntrapenuerAuthor Commented:
That was just an example message. It should be assumed that we tried all that already , We've tried all different ways. HTML, plain-text, salutations positions, long body..etc.
Just askin' because you know what they say about assume.
ass u me.

I have clients using Quickbooks that try to send invoices via the Quickbooks mailer
(quickbooks-email @ intuit.com)
Despite being whitelisted in the spam filter, these die.
The message is just too generic, and looks like image spam.
The Cloudmark spam engine rates them 5000 and they die.

Another client tries this message with a PDF starting with tmpXXXX.pdf
See below for your invoice(s) for your recent purchase/ service from us. If you have already received this invoice or it has already been paid please disregard this email.
Please double-click the name of the attachment to review or download.
In case you fail to read a PDF file, please visit the web site below.
After successfully installing Acrobat Reader, you will be able to open the file.
If there are any questions, please direct your inquiries to our Accounts Receivables Department at xxx-xxx-xxxx or toll free at 1-xxx-xxx-xxxx

Thank you.
Attached: Service Invoice 1404013

This looks too much like many of the spear-phishing virus mails bouncing around, and it just dies despite being whitelisted six ways to Sunday.

Was www.mxtoolbox.com clean?
and does your rDNS resolve consistently to the sending domain, or occasionally to the other domain?
Ess KayEntrapenuerAuthor Commented:
MX Toolbox is always clean,
always correct domain

We don't generate emails with 3rd party on new server. We only use our web interface for mail   (roundcube)

Here is the latest one sent


This is the mail system at host mail.h*.co.

Your message was successfully delivered to the destination(s)
listed below. If the message was delivered to mailbox you will
receive no further notifications. Otherwise you may still receive
notifications of mail delivery errors from other systems.

                   The mail system

<m*@gmail.com>: delivery via
    gmail-smtp-in.l.google.com[]:25: 250 2.0.0 OK 1474922924
    yi10si2145645091194wjb.227 - gsmtp
Reporting-MTA: dns; mail.h*.co
X-Postfix-Queue-ID: 14F8660469
X-Postfix-Sender: rfc822; service@h*.co
Arrival-Date: Mon, 26 Sep 2016 20:48:31 +0000 (UTC)

Final-Recipient: rfc822; m*@gmail.com
Original-Recipient: rfc822;m*@gmail.com
Action: relayed
Status: 2.0.0
Remote-MTA: dns; gmail-smtp-in.l.google.com
Diagnostic-Code: smtp; 250 2.0.0 OK 1474922924 yi10si264691194wjb.227 - gsmtp
Return-Path: <service@h*.co>
Received: from mail.h*.co (localhost [])
    by mail.h*.co (Postfix) with ESMTP id 14F64469
    for <m*@gmail.com>; Mon, 26 Sep 2016 20:48:31 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=h*.co;
    s=mail; t=1474922911;
MIME-Version: 1.0
Content-Type: multipart/alternative;
Date: Mon, 26 Sep 2016 16:48:30 -0400
From: H* <service@h*.co>
To: m*@gmail.com
Subject: Murray, Your metric Based Nutrition Account
Organization: H**
Reply-To: service@h.co
Mail-Reply-To: service@h*.co
Return-Receipt-To: H* <service@h*.co>
Disposition-Notification-To: H* <service@h*.co>
Message-ID: <6190c73e2a266d66102e02ef05908@h*.co>
X-Sender: service@h**.co
User-Agent: Roundcube Webmail/1.1.3

Email Original
MIME-Version: 1.0
Content-Type: multipart/alternative;
Date: Mon, 26 Sep 2016 16:48:30 -0400
From: H* <service@h*.co>
To: m*@gmail.com
Subject: Murray, Your metric Based Nutrition Account
Organization: H*
Reply-To: service@h*.co
Mail-Reply-To: service@h*.co
Return-Receipt-To: H* <service@h*.co>
Disposition-Notification-To: H* <service@h*.co>
Message-ID: <619654645666102e02ef05908@h*.co>
X-Sender: service@h*.co
User-Agent: Roundcube Webmail/1.1.3

Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII


Hi Murray

We've been having a hell of a time testing our new system
Things seem to be going south for some unknown reason.
I figured I'd drop you a note to see how you were

John Estafan
H*  Support

CALL US AT (***) ***-**** / ***-***-****

P.S.  Reply when you get this
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset=UTF-8

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset=
=3DUTF-8" /></head><body style=3D'font-size: 10pt; font-family: Verdana,Gen=
<p><em>Hi&nbsp;Murray<br /><br />We've been having a hell of a time testing=
 our new system<br />Things seem to be going south for some unknown reason=
=2E<br />I figured I'd drop you a note to see how you were<br /><br />Later=
,<br />John Estafan<br />H* Support<br />service@h*.co<br =
<h2><span style=3D"font-size: 8pt;">Call us at *** / ***=
<p><em><br /><br />P.S. &nbsp;Reply when you =
get this</em></p>

Looking in my own gmail spam filter, I have stuff from Facebook and stuff from TechConnect.
The excuse in gmail is
Why is this message in Spam? It's similar to messages that were detected by our spam filters.  Learn more

What excuse does your mail generate in GMail?
Mine are passing DKIM and SPF, but clearly getting content-hammered -- which is tough to troubleshoot.
Ess KayEntrapenuerAuthor Commented:
Thats the one we get
Then as I see it you have 5 choices
1.Give up

2. Roll it out and get users whitelisting your mail.
That may eventually get the issue taken care of for everyone
(me, I do little with Facebook.  Gmail sees that and routes that mail to spam, where it would go into the inbox of a frequent facebooker.  It's spam filtering by engagement, and YOU have no influence over that)

3. Try to contact humans at Google and try to get them to be nicer to your emails

4. Do as Scott Fell suggests and outsource it so that it becomes a problem someone else is paid to solve.

5. Temporarily use a mass-mailer service that CAN get through to you customers to warn them of the change, have them add yourmailer@yourdomain.com to their contacts, and to watch their spam folders for your stuff and whitelist it.

Bottom line: when Gmail hates the content of your email, there''s little you can do as the sender to change that situation.  You are at the mercy of the receiving end-user.
Ess KayEntrapenuerAuthor Commented:
Those are not options that we are satisfied with except maybe #3 but they been reluctant to respond.  There should be an easier way
When mail from Facebook winds up in the spam filter because the 'engagement' spam filter has decided the enduser has no use for the mail and FACEBOOK hasn't gottn Google to alter that, then what shot do mortals have?

The engagement filter is Big Data.  Until you start punching out gmails that users view, and reply to and add to Contacts and click on links contained in them, you are unlikely to move the dial.

Spam is a scourge.  All the easy ways have been blocked because they have been exploited. Good luck with the humans.
Ess KayEntrapenuerAuthor Commented:
when we use a 3rd party like mandill
there are no issues
When we use a 3rd party like mandill  there are no issues
That's coming from an IP and DNS source that has been around a long time and has an engagement history with gmail users.

Your brand-new IP address has none of that.
And most brand new IPs that start tossing mail ... are virus infected spam sources.

You could try setting up a bunch (25+) gmail accounts over a couple months and start your mail tosser throwing mail at them daily.  Open that mail, mark it not spam, add the account to contacts.
Build up that big data reputation for your new IP
When a new gmail account you create gets your mail into the inbox by default, you may have turned the corner -- but there's no guarantees.

but  we have a 50k mailing list.
Better alternative is to start sending from your tosser to all NON-gmail accounts right now since they don't have grief.  That starts getting your tosser noticed.  Use Google's DNS servers, so they see traffic to and from your tosser.  Use your third-party tosser to send notices to a subset of your gmail users each time the mass-mailings go out stating that you are cutting over, check the spam filter, whitelist us, yadda yadda -- and then cut those users over.  Phone them!  Yes that's intensive, but it is customer service and may be worthwhile.  Keeping cutting over gmail users until you aren't doing anything third-party anymore.

If Google humans aren't going to smooth the way, the only way to get your tosser on the nice list is to send mail to gmail users that they read.

Alternatively, use your tosser to send something interesting that your customers will engage with that's NOT mission-critical until your gmail testing works well and then cut over.

There will be no magic bullet.
Ess KayEntrapenuerAuthor Commented:
not sure what did it but issue seems to have resolved itself.

Here are some steps that I have taken,  perhaps one or all of them contributed to the solution.

1. set up spf & dkim
2.  marked not spam on several gmails (about 10)
3. gave it time (3 months?)
4. contacted Google using their contact form for delivery issues
5. started a thread in Google groups for emails
6. set up webmasters account and got verified
7. set up postmaster account and got verified
8. used a 3rd party to send a couple of emails (mandrillapp.com)

I don't know the correct combination of the above that gets it done,  but if anyone has a similar issue,  i suggest reproducing all of them.

Thanks for the advice fellas

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Ess KayEntrapenuerAuthor Commented:
Thanks for hrlp.  It solved itself
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Email Servers

From novice to tech pro — start learning today.