Link to home
Start Free TrialLog in
Avatar of lakegroup
lakegroup

asked on

Connection to Microsoft Exchange has been lost/restored. Event ID 26 occurs daily after work hours at a regular time

Between 12 midnight and 12:30 any PC on my network with Outlook (365/2019) will experience as series of Outlook Event ID 26
Connection to Microsoft Exchange has been lost. Outlook will restore the connection when possible.
Connection to Microsoft Exchange has been restored.

Typically about 4 of each happen in a half hour window.

I turned on Outlook logging and my events then showed up as Event ID’s 19 and

Rpc call (Unknown) on transport (ncacn_http) to server (east.exch030.serverdata.net) failed with error code (6be) after waiting (0) ms; eeInfo (Block (0), Error = 6be, Version = 1, GeneratingComponent = 2, DetectionLocation = 6ae, Flags = 0, Params = 1, [Param (0) Type = eeptLongVal, Value = 1], Block (1), Error = c0021009, Version = 1, GeneratingComponent = 2, DetectionLocation = 6b8, Flags = 0, Params = 0, Block (2), Error = c0021009, Version = 1, GeneratingComponent = d, DetectionLocation = 579, Flags = 0, Params = 0).

Connection to Microsoft Exchange has been lost. Outlook will restore the connection when possible.

Rpc call (EcDoConnectEx) on transport (ncacn_http) to server (east.exch030.serverdata.net) failed with error code (6bf) after waiting (148437) ms; eeInfo (Block (0), Error = 6bf, Version = 1, GeneratingComponent = 2, DetectionLocation = 6ae, Flags = 0, Params = 1, [Param (0) Type = eeptLongVal, Value = 0], Block (1), Error = c0021009, Version = 1, GeneratingComponent = 2, DetectionLocation = 5b8, Flags = 0, Params = 0, Block (2), Error = c0021009, Version = 1, GeneratingComponent = d, DetectionLocation = 596, Flags = 0, Params = 0, Block (3), Error = c0021009, Version = 1, GeneratingComponent = d, DetectionLocation = 579, Flags = 0, Params = 0).

Rpc call (NspiBind) on transport (ncacn_http) to server (east.exch030.serverdata.net) failed with error code (6bf) after waiting (148453) ms; eeInfo (Block (0), Error = 6bf, Version = 1, GeneratingComponent = 2, DetectionLocation = 6ae, Flags = 0, Params = 1, [Param (0) Type = eeptLongVal, Value = 0], Block (1), Error = c0021009, Version = 1, GeneratingComponent = 2, DetectionLocation = 5b8, Flags = 0, Params = 0, Block (2), Error = c0021009, Version = 1, GeneratingComponent = d, DetectionLocation = 596, Flags = 0, Params = 0, Block (3), Error = c0021009, Version = 1, GeneratingComponent = d, DetectionLocation = 579, Flags = 0, Params = 0).

Rpc call (EcDoConnectEx) on transport (ncacn_ip_tcp) to server (east.exch030.serverdata.net) failed with error code (6ba) after waiting (21016) ms; eeInfo (Block (0), Error = 6ba, Version = 1, GeneratingComponent = 2, DetectionLocation = 1f5, Flags = 0, Params = 4, [Param (0) Type = eeptUnicodeString, Value = ncacn_ip_tcp], [Param (1) Type = eeptUnicodeString, Value = east.exch030.serverdata.net], [Param (2) Type = eeptLongVal, Value = a4f1db00], [Param (3) Type = eeptLongVal, Value = 6ba], Block (1), Error = 6ba, Version = 1, GeneratingComponent = 12, DetectionLocation = 5a2, Flags = 0, Params = 1, [Param (0) Type = eeptUnicodeString, Value = east.exch030.serverdata.net], Block (2), Error = 6ba, Version = 1, GeneratingComponent = 12, DetectionLocation = 143, Flags = 0, Params = 0, Block (3), Error = 4d5, Version = 1, GeneratingComponent = 12, DetectionLocation = 139, Flags = 0, Params = 0, Block (4), Error = 274c, Version = 1, GeneratingComponent = 12, DetectionLocation = 137, Flags = 0, Params = 3, [Param (0) Type = eeptLongVal, Value = 87], [Param (1) Type = eeptPointerVal], [Param (2) Type = eeptPointerVal], Block (5), Error = 274c, Version = 1, GeneratingComponent = 12, DetectionLocation = 13e, Flags = 0, Params = 0).

Rpc call (NspiBind) on transport (ncacn_ip_tcp) to server (east.exch030.serverdata.net) failed with error code (6ba) after waiting (21016) ms; eeInfo (Block (0), Error = 6ba, Version = 1, GeneratingComponent = 2, DetectionLocation = 1f5, Flags = 0, Params = 4, [Param (0) Type = eeptUnicodeString, Value = ncacn_ip_tcp], [Param (1) Type = eeptUnicodeString, Value = east.exch030.serverdata.net], [Param (2) Type = eeptLongVal, Value = f5cc5a18], [Param (3) Type = eeptLongVal, Value = 6ba], Block (1), Error = 6ba, Version = 1, GeneratingComponent = 12, DetectionLocation = 5a2, Flags = 0, Params = 1, [Param (0) Type = eeptUnicodeString, Value = east.exch030.serverdata.net], Block (2), Error = 6ba, Version = 1, GeneratingComponent = 12, DetectionLocation = 143, Flags = 0, Params = 0, Block (3), Error = 4d5, Version = 1, GeneratingComponent = 12, DetectionLocation = 139, Flags = 0, Params = 0, Block (4), Error = 274c, Version = 1, GeneratingComponent = 12, DetectionLocation = 137, Flags = 0, Params = 3, [Param (0) Type = eeptLongVal, Value = 87], [Param (1) Type = eeptPointerVal], [Param (2) Type = eeptPointerVal], Block (5), Error = 274c, Version = 1, GeneratingComponent = 12, DetectionLocation = 13e, Flags = 0, Params = 0).

User canceled request against server (east.exch030.serverdata.net) after waiting (184062) ms.

I did some Wireshark captures and I cannot make heads or tails out of them except to say, when the Event ID’s 26 occur I always see a Connection Reset [RST]
User generated imageUser generated image

During the Outlook disconnect/connect bouncing I can open up HTTP web pages, do FTP transfers, access, ping and traceroute to servers locally and across the internet without issue and there is no latency.
The issue is not limited to the Microsoft Office Outlook client, I also see that OWA sessions to my hosted exchange server in a web browser also are connected and disconnected.

I started by looking at our WAN connection and had our ISP take a look at our data circuit. They found nothing.
Next I had our hosted Exchange provider take a look at their logs and they found nothing.
Then I moved onto our firewall provider Untangle and worked with them to look at logs and nothing was found there.
I am at the point of trying OSI layer troubleshooting; Different switches, ports, different Ethernet cables, disabling scheduled tasks,  booting computers to safe mode with Networking  with no Antivirus or Malwarebytes, downing certain servers and network devices overnight to see if any of them are causing it.  But so far, these Outlook disconnects still occur like clockwork regularly and almost always at 12:04am.

Is there anything in the event log items above or the Wireshark screen cap that raises alarms for anyone? I could provide more Wireshark captures if it helps. I’m just not 100% sure what I am looking for. Googling comes up with many similar issues but I never find a resolution that helps me.

Thanks in advance, any guidance will be appreciated!
Avatar of efrimpol
efrimpol
Flag of United States of America image
just a shot in the dark. what time does the backup against the exchange server run (or against the server in general) ...
you may need to ask your provider.
Avatar of lakegroup
lakegroup

ASKER

According to my Hosted Exchange provider, all backup and maintenance occurs between 4-5am EST. They are going to try to replicate the issue tonight so I'm waiting for the results of that..
Thanks!
Avatar of lakegroup
lakegroup

ASKER

Our Hosted Exchange provider could not replicate the issue.

I do have a new finding. I set up two laptops with Outlook and assigned them public facing IP addresses. I used the DNS of our main fiber ISP (Lightpath) on one and on the other laptop I used our backup T1 ISP (GTT Communications) using OpenDNS as the DNS servers.

Both laptops experienced the Outlook disconnect at exactly the same time; 12:05am.

Both the Lightpath and GTT routers are connected to the same switch as the two laptops.

I was thinking I could try connecting one of the laptops directly to the GTT router, bypassing the switch.  Does anyone have any other ideas?
Avatar of efrimpol
efrimpol
Flag of United States of America image
doesn't hurt to try but I think means you may have your answer tomorrow.
Avatar of lakegroup
lakegroup

ASKER

I'm sorry, I don't fully understand your reply.  Are you saying you think the fault is with the switch or am I mis-reading?
ASKER CERTIFIED SOLUTION
Avatar of efrimpol
efrimpol
Flag of United States of America image
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of lakegroup
lakegroup

ASKER

Yes, thanks.
Avatar of lakegroup
lakegroup

ASKER

I just came across this very old open question. The issue was indeed resolved when our Hosted Exchange provider upgraded from Exchange 2007 to Exchange 2016.
Avatar of Christian Bouwer Knudsen
Christian Bouwer Knudsen
Did you ever get this fixed?
Avatar of lakegroup
lakegroup

ASKER

Yes, it was resolved when our Hosted Exchange provider upgraded from Exchange 2007 to Exchange 2016.