Tool to test Windows Updates

We have a client with a requirment to have all Windows updates thoroughly tested before deployment.  The problem that arises is that it is a cumbersome task to go through each Security update, etc.  Is there a site devoted to Windows Updates where issues are posted as they arise?  What is the most efficient way to test Windows updates and patches?  The client in question is a life critical organization where they can only afford 2 hours of downtime a month so cannot have their systems down longer than the patches take to install.
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

I would create a small lab (virtual or physical with a few servers and workstations). Try to get the lab machines as close to a production state as possible.  If using WSUS - create a distribution container with these lab machines.  It might be helpful to create a TestOU in AD and add them to it as well for future testing with other Windows components like GPO's.

Load Update Compatibility Evaluator (UCE) onto all the test machines. For more info.

Ensure that all event logs are set to verbose for the test machines so that you can review errors from the updates should they occur.

Once the lab is built, manually or automatically distribute via WSUS the patch you want to test on the test machines.  UCE and the event logs are going to your tell-tells for problems.

I would recommend reading this patch management process by SANS to get a general idea of what you want to drive out.

Once you get a plan of how you want to process patches in your business, start picking off pieces to automate it to reduce time.  Microsoft TechNET is a decent places to watch for issues along with just googling the KB or update itself to see what pops up.


Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Your situation would be well served with a virualized environment.... VMWare and/or citrix both have a lot of tools/products that would allow you to basically have an environment where you could apply the patches to the system, and should they cause problems, roll back to the pre patch time with little effort.  Other than some set up like that, you would be forced to have a Production system and a test system where you could apply the patches to the test system prior to doing them on the prod system.  That is really the best any only way to truly test out patches and updates to see how they affect your environment.  No two systems are the same.
I agree with both suggestions but if budget allows, it would be great to get some physical machines for this testing lab that are the same model as what you have. I suggest this becuase some updates that are pushed out may have an effect on hardware specific drivers. For example, one time I saw a server with network cards teamed together. After a windows update, the network connection 'broke'. There was also a time where an update caused the network connection to simply drop until the server was restarted.

Of course this option isnt always possible since it is an expensive investment. It sounds like the biggest thing you might want to test for is application compatibility. As another example, an organization I worked for had custom web applications that looked for the users log on name. After windows updates, none of the workstations could log onto the web application becuase it couldnt read the username anymore. Of course this was later repaired by microsoft but caused a good few hours of downtime before a workaround could be created.

Using WSUS is a good way to track specific updates and push them out as they are approved. For testing, I would suggest making a check list of all functions and internal applications that should be tested after an update. You can also sign up for advanced notification of new updates

You may also considering pushing updates in waves. So instead of all clients or servers updating at the same time, you can push updates to certain clients Wednesday, some other clients Thursday, etc. This might help avoid everyone being effected by unforseen errors at once and let you stop the update from being pushed out before the scope grows and everyone else has a problem.
Has Powershell sent you back into the Stone Age?

If managing Active Directory using Windows Powershell® is making you feel like you stepped back in time, you are not alone.  For nearly 20 years, AD admins around the world have used one tool for day-to-day AD management: Hyena. Discover why.

advserverAuthor Commented:
The problem that we run into is that it is impossible to have replica virtual or physical environments of all the client networks.  With the client in question an explanation is required for the installation of every update along with the assurance that nothing will break upon completion.  Even with a lab environment which is similar to the client's the chance still is there that one of the updates will not play nice once it's installed even if there were no ill effects in the lab.

Thank you very much for your responses.  I guess it is safe to say that when it comes to Microsoft Updates there is only so far you can go with testing and researching prior to installing before you are left to crossing your fingers during installation.  

All three posts are acceptable solutions. Thank you!

One last thing to consider is that you can do trend set patch management for another non-critical client  that is similar to your critical client if that is possible.  Put the standard client safely near bleeding edge on patches and trend/document changes and challenges as needed.  This is basically making one of your clients into a patch lab of course with their blessing first.

By doing this, you prove that the patches are working appropriately in a production environment similar to your critical client.  You can also scale back the deployment time of the patches for the critical client from the standard client mentioned above to establish a greater stability time to show stronger success of the patches being deployed.

Clients have a tendency to swing very heavy on the pendulum of patching.  Either I want it all now or you better make sure I need it and it will work perfect before you put them on our systems.  Aside from this, I would still recommend creating a mini lab environment (virtual or physical) to at least perform alpha testing on machines before pushing out to any of your clients.  There are many cost benefits that can be incurred from this but that is another thread.


PS ~ Please don't forget to divide and post points for our answers.
advserverAuthor Commented:

Thank you for your response.  I have been relying on clients who do not operate 24hrs and have the flexibility to have their servers rebooted and down for extended periods of time after hours if necessary.  I wish it didn't have to be a game of Russian Roulette when it comes to loading patches, especially those deemed critical.  Thank you to everyone once again for your suggestions!
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Windows Server 2008

From novice to tech pro — start learning today.