Link to home
Start Free TrialLog in
Avatar of SupermanTB
SupermanTB

asked on

Despite MX record change, email still being delivered to old email service days later

I've got a domain registered with GoDaddy...DNS also there.  I deleted the old MX records (Google Apps) and added the new ones (Office365) last Friday at ~2pm.  I noticed today that beginning yesterday and going through this morning that some email is still being delivered to the old Google Apps accounts.  This is happening for multiple users.  For one user, I would say approximately 20 emails were delivered to Google from various sources from yesterday through this morning.  During this same period of time, the user received email to Office365 as well.  

I did a global check to verify the propagation of the MX records.  All the results came back as pointing to Office365.  I also went to the GoDaddy account and everything looks as expected.  There is only one MX entry in the zone file, for Office365.  I called GoDaddy and had them clear out any history or cached zone files.  Hopefully that will help.  I also deleted the existing Office365 MX record and added it back.

I've changed MX records like this more times than I care to remember and I've never seen an issue like this where the MX records magically revert back two days later.

Anyone seen this before?
email-headers.txt
Avatar of scraby
scraby

is the mail that was delivered to the old mx record addressed to a different domain, perhaps an old forgotten account that not too many people used anymore?
Avatar of Frank McCourry
DNS changes at Godaddy take a long time to propegate across Godaddy's servers.  I could never figure out why this is.  It should not take more than 48 hours.

One thing you can do is reduce your TTL time before you make record changes.  If you have a very high TTL other DNS servers that have cached your records will not update untill the TTL expires.  I believe the lowest TTL on Godaddy is 1/2 hour.  

None of this helps you now, only patience can help at this point.
Did you add all the DNS entries that you were given from MS deployement to your DNS configuration in Godaddy? When I was migrating to Office 365, I was given various DNS settings to configure in my control panel.

In my own case I don't have exchange Server? What is your configuration like?

Regards
Avatar of SupermanTB

ASKER

Thank you all for your comments.

scraby:
Unfortunately this is not an issue with a different domain.  There is only one domain being used here.

frankmcc:
The odd thing about this is that email delivered fine over the weekend.  These users receive a lot of email.  Everything delivered to Office365 and nothing to Google.  For some reason on Monday morning, email started going to Google Apps again.

giveandtake638:
I've done Office365 migrations like this many many times and definitely added all the appropriate DNS entries.  I migrated from Google Apps to Microsoft Office365.  There is no on-site Exchange server.
Did the record change at Godaddy?  Is there a second MX record to Google apps?  

If there is a second MX record, even with a higher priority, mail may be delivered to that location if the primary becomes unreachable.

Funny thing about DNS, If you have multiple records and the first on fails, when a machine goes to the next record, it tends to stick with that until it sees a failure.  If the primary comes back, the sending machines will not attempt to use it until the secondary record either times out or the secondary server fails.
There is only one MX record
So it sounds like there are a lot of server that still have your old MX record cached and you will simply have to wait it out.
The problem may not be with GoDaddy but with the remote servers sending mail to you -- their DNS cache may not have flushed the old MX record.

I usually drop the ttl a couple of weeks in advance to something really low (an hour usually) so that by the time I make the change, remote DNS servers that cache my information can pick up the changes relatively quickly.
_jesper_
The TTL was set to 1 hour for quite some time previous to the change.

frankmcc/_jesper
Normally I would agree that it could be an issue with the servers not updating their MX records, but that doesn't seem to fit here.  Zero email was delivered to the Google accounts over the weekend and then magically started yesterday morning.  Also, when I do a Global MX lookup, all the results come back pointing to Office365.  Furthermore, there are many many emails from various sources/domains getting through to Google.  I could understand a few emails getting through, but it would take way too many servers not being updated to cause that many emails to get through to Google.  It's been 96 hours.  The servers should be updated by now.
What is your domain name?  I would like to my own queries and see if I can find something you may be missing.
I don't want to post it here.  You can look at the email headers attachment to my original post.  You'll see  the sender address at the very top line.
I can see that the MX record points to outlook.com from both authoritative DNS servers for your domain.

This still sounds like a remote DNS cache issue where those remote servers are ignoring the ttl (some do).
The only thing that doesn't fit is that everything was fine over the weekend.  These users receive a lot of email, so it would be easy to tell.  It just doesn't make sense to me that the remote servers would get the MX update over the weekend and them revert back yesterday.
The only thing that I can see happening is GoDaddy rolling back the updates in error and fixing it.  That would be improbably, however.

I would recommend contacting GoDaddy to find out if anything happened over the weekend to at least rule that out.
I've already done that and they say nothing happened over the weekend, but you never know with them.

About 1.5 hours ago they say they cleared any zone file caches that may be lingering around in their system.  I still have received email to the Google accounts very recently though.  

Unfortunately I'm going to have to sit and wait this one out I'm afraid.  Hopefully the MX record update I did this morning will cause the problem remote servers to update the MX records for my domain again.
I have always trusted my domain with smaller, more independent providers.

It surely wouldn't hurt to consider your DNS options.
I've used GoDaddy for a very long time and done hundreds of MX record changes like this without any major issues.  Very unusual.
ASKER CERTIFIED SOLUTION
Avatar of Frank McCourry
Frank McCourry
Flag of United States of America image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
I agree completely.  Lucky me!
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Are the domains sending to you Google Apps domains?  

Check a few by trying to go to http://www.google.com/a/<domain name>.   If you can an error message, then no.  Otherwise, the issue may be with Google's internal DNS (sending from/to Google hosted domains).

When you migrated, did you setup dual delivery?  Is there any chance the O365 accounts are forwarding messages?
There are too many different domains that are getting through to the Google Apps accounts to all be hosted with Google.  I've done an MX lookup on a few of them and none were hosted with Google.  I also used the method you mentioned.

When I migrated, I did not setup dual delivery and the O365 accounts are not forwarding messages.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
That's the only thing that makes sense.  Supposedly they cleared the DNS cache yesterday.  I've received email to the Google accounts all day yesterday, although it seems to have lessened a bit last night.  I'll have to keep an eye on things today and tomorrow.

I've arranged for the Google accounts to forward email to the Office365 accounts so the users don't notice the difference.  It will be interesting to see how this behaves in about 2 days.
ISPs are known to cache stuff for a couple of days regarding the TTL. i've seen similar issues dozens of times. sometimes a week is needed to get things right.

if godaddy is reverting back and forth, it will obviously be worse

spammers also tend to use huge caches or possibly deploy next-hop server names in botnets and take a long time to adjust to such changes, so the few mail delivered to the old site will be mostly spam after a reasonable delay. i guess they try to hide the bots by making them send email without doing MX resolutions.

you can monitor dns caches statuses using tools such as this one
http://mega-check.com/index.php?fCached=ubgcharlotte.com&sCached=MX
Thanks for the advise.  The link you provided doesn't work.
SOLUTION
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
The emails to the Gmail account seem to have tapered off.  I receive one email on Friday, Sept. 20th, but none since then.  I'll check back later in the week to be sure, but hopefully this is the last of this problem.  I've done more MX record changes than I care to remember and I've never seen it take more than a day or two, much less an entire week.
Yes, this is very weird.
It ended up taking a week for the emails to finally stop going to Google.  The only thing that makes sense here is GoDaddy having some sort of issue that caused cached or ghost DNS entries.  Thanks for everyone's help.
ISPs are known to cache stuff for a couple of days regarding the TTL. i've seen similar issues dozens of times. sometimes a week is needed to get things right.

spammers also tend to use huge caches or possibly deploy next-hop server names in botnets and take a long time to adjust to such changes, so the few mail delivered to the old site will be mostly spam after a reasonable delay. i guess they try to hide the bots by making them send email without doing MX resolutions.

i've been working in antispam field for years and monitored botnet traffic evolution. you can trust me : this IS normal behavior which i saw dozens of times regardless of the operator. i'm not a godaddy fan but they're not at fault here...

some domain have been harvested by folks that use such botnets, some do not. 2 days for DNS propagation was rather usual a few years ago, nowadays a few hours seem more reasonable whatever the TTL. a week is frequent for many spam flows.