Caveats and precautions when applying windows update to Domain Controller ?

People,

Can anyone here please suggest me what do I need to do as pre cautions and mitigation plan when applying Windows Update to the following servers:

1. Windows Server 2008 R2 VM - Primary DNS servers (Production Data Center) , Domain Controller / Global Catalog Server (Schema Master) - (220 Updates to be applied)

2. Windows Server 2008 R2 VM - Secondary DNS servers (Production Data Center) , Domain Controller / Global Catalog Server (Domain naming master) - (215 Updates to be applied)

3. Windows Server 2012 R2 Physical box - Primary DNS & DHCP servers (HQ Office) , Domain Controller / Global Catalog Server (RID & Infrastructure master, PDC Emulator) - (110 Updates to be applied)


For the VM in number #1 and #2 does updating the WIndows during the working hours can have some impact in the Name resolution and Exchange email flow ? Can I take snapshot on both VMs at the same time and roll it back when it failed or hangs when applying some updates ?

For the physical server in #3, how can I prevent any outage during the working hours patching ?

Sometimes when I apply the updates to the WIndows Servers, there are some things that caused the updates to be failed thus taking too long not responding back to remote desktop.

Any help would be greatly appreciated.

Thanks.,
LVL 9
Senior IT System EngineerIT ProfessionalAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
For the VM in number #1 and #2 does updating the WIndows during the working hours can have some impact in the Name resolution and Exchange email flow ?

Only if you restart the server, we would never attempt to update servers in core hours, schedule the work for out or cores.

Can I take snapshot on both VMs at the same time and roll it back when it failed or hangs when applying some updates ?

Do not use snapshots, and attempt to roll them back, you will get into snapshot rollback USN issues.

For the physical server in #3, how can I prevent any outage during the working hours patching ?

Complete the udpates out of hours.

We backup and clone all servers before any update.
0
Mark BillExchange, AD, SQL, VMware, HPE, 3PAR, FUD, Anti MS Tekhnet, Pro EE, #1Commented:
Andrew you are the boss. ;)
0
McKnifeCommented:
The most important thing is to do them at all and, for security's sake, in a timely fashion. Apart from that, not much special applies, have a backup/emergency strategy, don't do all domain controllers at the same time.
About vm snapshots/USN Rollbacks: the game has changed a bit with server 2012, please read http://blogs.technet.com/b/reference_point/archive/2012/12/10/usn-rollback-virtualized-dcs-and-improvements-on-windows-server-2012.aspx (site down at the time of writing)
0
Ultimate Tool Kit for Technology Solution Provider

Broken down into practical pointers and step-by-step instructions, the IT Service Excellence Tool Kit delivers expert advice for technology solution providers. Get your free copy now.

Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
What we find useful, is we have replica Labs of all our clients, so we can test, test, test and re-test Windows Updates before they are applied to live production servers, and this is even easier now, we have virtual environments. This takes the guess work out of how long it's going to take, and also what the back out plan, and change control plans are if updates go wrong.

Testing is provided on isolated networks, so as not to conflict with production devices and services.
0
Will SzymkowskiSenior Solution ArchitectCommented:
My Answers are below...

For the VM in number #1 and #2 does updating the WIndows during the working hours can have some impact in the Name resolution and Exchange email flow ?

Performing updates should not have any impact on your DNS resolution or Exchange as you have multiple DC's in the same site where Exchange is hosted. Exchange will query a DC that is online. I would not update this during production hours.

Can I take snapshot on both VMs at the same time and roll it back when it failed or hangs when applying some updates ?

As stated already, it is not a good idea to snapshot DC's as there is ALWAYS processes that are running and things are always chaning. USN roll back is likely as stated already if you restore from a snapshot.

For the physical server in #3, how can I prevent any outage during the working hours patching ?

Do not do the patches during production hours. Even though you have redundancy with your DC's it is still recommended to perform patches outside of business hours.
Another thing i suggest is make sure that you TEST these patches in a lab environment. Only install Security and Critical patches on DC. Also apply the patches in small Groups like 10-15 at a time, testing after every reboot.

I know this might be a lot of work while doing this but if something goes wrong and you have applied 200 patches it will be almost impossible to figure out which patch cased the issue.

Sometimes when I apply the updates to the WIndows Servers, there are some things that caused the updates to be failed thus taking too long not responding back to remote desktop.

As stated already do these patches in groups of 10-15 or even smaller and reboot after every group of patches have been applied and ensure that they havbe completed successfully and also make sure that you CHECK your Services and Event logs to ensure everything started properly.

Based on the servers that you have I would leave DC3 until the end as it is the most critical server, holding DHCP and all of the domain FSMO roles.

Will.
1

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Senior IT System EngineerIT ProfessionalAuthor Commented:
Hi All,

Unfortunately I do not have test environment here in my company, so there is noway of testing it safely.

So therefore in the event of Windows is stuck in "Applying Updates..." screen how and what can I do to resume to the desktop ? can I just hard reset the server ?
0
Will SzymkowskiSenior Solution ArchitectCommented:
If you do not have a test environment then follow that I have stated and install the updates in groups of 10-15 each time.

Sometimes if updates hang there is no other solution by to do a hard reset, at which point, if your luck windows will continue to apply the patches upon bootup.

Make sure that you are only applying critical and security patches to the DC's, nothing else. These are usually the safest and should not create any issues. It is usually drivers or .NET updates that usually mess things up, not saying security or critical CAN'T but just apply them in Groups.

Will.
0
Senior IT System EngineerIT ProfessionalAuthor Commented:
@Will, ok that does makesense. so I guess this rule also applies to the Exchange, SQL and SharePoint Servers as well.
0
Andrew Hancock (VMware vExpert / EE MVE^2)VMware and Virtualization ConsultantCommented:
Very dangerous game, you are playing, "Russian Roulette with Windows Updates!"

No excuse, if you have a virtual environment, it's so easy to create. As I posted we have test environments for all our Clients.

Good Working IT Practice, if you follow ITIL Service Framework.
0
Will SzymkowskiSenior Solution ArchitectCommented:
Exactly.

"ounce of prevention is a POUND of cure"

Will.
0
Senior IT System EngineerIT ProfessionalAuthor Commented:
@Will and @Andrew: Yeah I know and I wish that it is the case here.

It is a scary stuff when you have to apply 200+ updates for all of your Exchange Servers and Domain Controllers.

This new company where Iam working didn't know the importance of Windows Update hence I'll have to pick it up to make it up to date.

The only thing that I can see or utilize is Veeam 8.0 Surebackup but running very slow. (I'll create another thread for this).
0
Will SzymkowskiSenior Solution ArchitectCommented:
Ultimately we can all provide suggestions as the best way to accomplish these tasks. But in the real world we do not always have the appropriate resources required so there is going to be more risk in this case.

If the company you work for is aware of this and they still do not provide the funds for a lab environment then they know the risks upfront and you can use that against them if something goes wrong.

Will.
1
Senior IT System EngineerIT ProfessionalAuthor Commented:
Many thanks guys !
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Active Directory

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.