Avatar of Jaimy Dixon
Jaimy Dixon
Flag for United States of America asked on

Exchange 2016 Recovery Issue

We have an Exchange 2016 server running on Server 2012. Last night our Exchange server stopped running and we attempted to run a Setup.exe /mode:recoverserver and originally we were receiving a buildtobuildupgrade error that would not allow us to proceed. After removing the BuildtoBuild entries in regedit we are now getting a new error when we try to run the installer. This is the error. I tried to find information on it but no one has seen this error before.

Error:
The following error was generated when "$error.Clear();
      buildToBuildUpgrade-ExsetdataAtom -AtomName MDB -DomainController $RoleDomainController

" was run: "Microsoft.Exchange.Management.Deployment.ExsetdataException: An error occurred with error code '3221685277' and message 'The service did not respond to the start or control request in a timely fashion.'.
   at Microsoft.Exchange.Configuration.Tasks.Task.ThrowTerminatingError(Exception exception, ErrorCategory category, Object target)
   at Microsoft.Exchange.Management.Deployment.ManageExsetdataAtom.HandleExsetdataReturnCode(UInt32 scErr)
   at Microsoft.Exchange.Management.Deployment.ManageExsetdataAtom.BuildToBuildUpgradeAtom(AtomID atomID)
   at Microsoft.Exchange.Management.Deployment.BuildToBuildUpgradeExsetdataAtom.InternalProcessRecord()
   at Microsoft.Exchange.Configuration.Tasks.Task.<ProcessRecord>b__91_1()
   at Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String funcName, Action func, Boolean terminatePipelineIfFailed)".
Exchange

Avatar of undefined
Last Comment
Jaimy Dixon

8/22/2022 - Mon
Amit

Did you reset computer object in AD, before starting the recovery process.
Jaimy Dixon

ASKER
No. We did not.
Amit

You need to reset the Computer account first and then start the recovery process. Check this KB:
https://docs.microsoft.com/en-us/exchange/high-availability/disaster-recovery/recover-exchange-servers?view=exchserver-2019

or check my KB: https://www.experts-exchange.com/articles/18110/How-To-Recover-Failed-Mailbox-Sever-In-Exchange-2010-DAG-Environment.html?headerLink=workspace_article

Though i wrote this to Exchange 2010, however steps are almost same to any Exchange version. You can skip DAG steps.
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy
Jaimy Dixon

ASKER
Trying it now.
Jaimy Dixon

ASKER
I received the same error.

    Mailbox role: Mailbox service                             FAILED

The following error was generated when "$error.Clear();

buildToBuildUpgrade-ExsetdataAtom -AtomName MDB -DomainController
$RoleDomainController

" was run:
"Microsoft.Exchange.Management.Deployment.ExsetdataException: An error occurred
with error code '3221685277' and message 'The service did not respond to the
start or control request in a timely fashion.'.
 at
Microsoft.Exchange.Configuration.Tasks.Task.ThrowTerminatingError(Exception
exception, ErrorCategory category, Object target)
 at
Microsoft.Exchange.Management.Deployment.ManageExsetdataAtom.HandleExsetdataRet
rnCode(UInt32
scErr)
 at
Microsoft.Exchange.Management.Deployment.ManageExsetdataAtom.BuildToBuildUpgrad
Atom(AtomID
atomID)
 at
Microsoft.Exchange.Management.Deployment.BuildToBuildUpgradeExsetdataAtom.Inter
alProcessRecord()

 at Microsoft.Exchange.Configuration.Tasks.Task.<ProcessRecord>b__91_1()
 at
Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String
funcName, Action func, Boolean terminatePipelineIfFailed)".


The Exchange Server setup operation didn't complete. More details can be found
in ExchangeSetup.log located in the <SystemDrive>:\ExchangeSetupLogs folder.
Jaimy Dixon

ASKER
The server is still in production. Does recovery only work on servers that have been removed from production?
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Amit

Can you please answer below questions:

What do you mean by production?
Are you able to access Exchange and all Exchange related services are running.
What was the reason for starting recovery process.
Jaimy Dixon

ASKER
After the initial update to CU10 all services were loading except for Microsoft Exchange Information Store, Microsoft Exchange Diagnostics, Microsoft Exchange DAG Management, and Microsoft Exchange Mailbox Assistants. When we tried to rerun the CU10 update we received an error saying the Submission Service could not be restarted. We looked and saw that was because the Microsoft Exchange Mailbox Transport Submission service was missing after the update as in gone from the service list. We received a recommendation to run the recovery to restore the missing service. Now the service is on the list and is running but the other original services still will not start and if we try to run the CU update we get the error that I submitted above.
Amit

I advise you to open case with Microsoft. As this might need in-depth analysis.
Experts Exchange has (a) saved my job multiple times, (b) saved me hours, days, and even weeks of work, and often (c) makes me look like a superhero! This place is MAGIC!
Walt Forbes
Jaimy Dixon

ASKER
Thank you for your suggestions.
Amit

Keep me posted, how things goes with MS. If nothing works, extreme step, would be to re-install OS and then install Exchange server with recovery switch and restoring the db from backup. However, i want to avoid this last option.
Jaimy Dixon

ASKER
We have a VM we are trying to restore as a last resort. Will the installation changes that have been made effect that machine with AD?
Get an unlimited membership to EE for less than $4 a week.
Unlimited question asking, solutions, articles and more.
Amit

Don't restore VM Snapshot, as that would cause more issues. Refer:
https://thoughtsofanidlemind.com/2011/03/18/be-careful-with-vm-snapshots-of-exchange-2010-servers/

Also, note, if your schema already extended, then you cannot revert back.
Jaimy Dixon

ASKER
That was why I was asking. Thank you again.
ASKER CERTIFIED SOLUTION
Jaimy Dixon

Log in or sign up to see answer
Become an EE member today7-DAY FREE TRIAL
Members can start a 7-Day Free trial then enjoy unlimited access to the platform
Sign up - Free for 7 days
or
Learn why we charge membership fees
We get it - no one likes a content blocker. Take one extra minute and find out why we block content.
Not exactly the question you had in mind?
Sign up for an EE membership and get your own personalized solution. With an EE membership, you can ask unlimited troubleshooting, research, or opinion questions.
ask a question
Jaimy Dixon

ASKER
Thank you very much for your assistance with this. Even after all of this we still had issues connecting due to missing files so we were forced to update to CU11 afterward and even with direct support from Microsoft it took 6 attempts to get the CU update to install  and completely fix the server.
All of life is about relationships, and EE has made a viirtual community a real community. It lifts everyone's boat
William Peck