Remediating windows servers

Hello all,

What is the ideal way for patch management. How to remediate windows servers. Where should we start?

A SAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Here is the approach I would take :

Reference  :

Step 1: Develop an up-to-date inventory of all production systems, including OS types (and versions), IP addresses, physical location, custodian and function. Commercial tools ranging from general network scanners to automated discovery products can expedite the process (see Resources, below). You should inventory your network periodically.

Step 2: Devise a plan for standardizing production systems to the same version of OS and application software. The smaller the number of versions you have running, the easier your job will be later.

Step 3: Make a list of all the security controls you have in place--routers, firewalls, IDSes, AV, etc.--as well as their configurations. Don't forget to include system hardening or nonstandard configurations in your list of controls. This list will help you decide how to respond to a vulnerability alert (if at all). For example, let's say you learn that OpenSSH has a vulnerability that may allow a buffer-overflow attack, but from your list of controls you know you don't allow the SecSH protocol through your firewall. If nothing else, that knowledge gives you more time to react.

Step 4: Compare reported vulnerabilities against your inventory/control list. There are two key components to this. First, you need a reliable system for collecting vulnerability alerts. And second, you need to separate the vulnerabilities that affect your systems from those that don't. Some companies have staff dedicated to managing this process; others use vulnerability reporting services.

Step 5: Classify the risk. Assess the vulnerability and likelihood of an attack in your environment. Perhaps some of your servers are vulnerable, but none of them is mission-critical. Perhaps your firewall already blocks the service exploited by the vulnerability. In general, to classify and prioritize the risk, consider three factors: the severity of the threat (the likelihood of it impacting your environment, given its global distribution and your inventory/control list); the level of vulnerability (e.g., is the affected system inside or outside perimeter firewalls?); and the cost of mitigation and/or recovery.

Step 6: Apply the patch! OK, so now you have an updated inventory of systems, a list of controls, a system for collecting and analyzing vulnerability alerts and a risk classification system. You've determined which patches you need to install. Now comes the hard part: deploying them without disrupting uptime or production. Fear not, there are several tools that can help you with the actual patch process (see Resources, below). Evaluate these tools in terms of how well they fit your environment and budget. In some cases, manual patch maintenance may be more cost-effective. But in most cases--particularly for multiple servers or server farms distributed across multiple locations--some type of automated patch system will more than pay for itself.

Also Make sure you are using a system such as WSUS in order to automatically approve and deploy patches on a scheduled basis. SO you are not applying blanket updates to your servers.

Install and configure WSUS:

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
A SAuthor Commented:
thanks a lot,
When I deploy patches to windows servers, how can I check the difference before installing patch reboot and then restart.
How to check if there is something happening after installing a patch, I mean how to validate the patch is good and there are no issues with server.
Also when to involve application team to validate.
what is the best rollback scenario?
This site will probably assist with this :

These are the sections you will want to review :

Plan your WSUS deployment
Step 1: Install the WSUS Server Role
Step 2: Configure WSUS
Step 3: Approve and Deploy Updates in WSUS
Step 4: Configure Group Policy Settings for Automatic Updates

Protecting & Securing Your Critical Data

Considering 93 percent of companies file for bankruptcy within 12 months of a disaster that blocked access to their data for 10 days or more, planning for the worst is just smart business. Learn how Acronis Backup integrates security at every stage

A SAuthor Commented:
We WSUS installed.
and group policies are made
When I deploy patches to windows servers, how can I check the difference before installing patch reboot and then restart.
 How to check if there is something happening after installing a patch, I mean how to validate the patch is good and there are no issues with server.
 Also when to involve application team to validate.
 what is the best rollback scenario?
The only way you can is to do a search on TechNet for the KB number and see what others are saying about the patch. There isn't really a sure way to be able to get that information..

All servers can potentially react differently to patches because they are "ALL" running different configurations with different software.. One server could be fine with it and another could crash 5 different applications because a security protocol or DLL had been changed. You ALWAYS want to set default time every month to deploy patches and then monitor your event logs and servers closely after you deploy them..

If there is a magic ticket or formula past that I don't know it.

Roll back, you would role back the patches individually per server usually as not all servers are going to react to patches the same.  

You can also remove by WSUS but that would be a blanket removal for all servers.
aravind ancheWindows/Vmware Commented:
Best practise is to have a target group from WSUS that these patches go to first, before going company wide – but either way, you’ll want to remove the patch from the affected PCs.

How do you do this? This is my recommended safe approach:

Step 1. Disable the patch in WSUS.
Just do this now, before anyone else gets it. You’re not going to break anything by choosing the ‘Decline’ option on a patch in WSUS. Make sure you do it to each OS version or product you manage (e.g. Windows 7 32 bit, Windows 7 64 bit, Windows 8 32 bit etc).

Step 2. Test uninstalling the patch manually
Before you go nuts and try to fix all the things at once, do a quick test or two. If you manually uninstall the patch, does it successfully uninstall? Reboot and make sure the PC seems happy (check event viewer!). Reboots may take a while doing system state backups and rolling back the patch.

Step 3. – Set WSUS to Uninstall the patch.
It’s a bit counter intuitive to approve a patch to then set it to remove, but that’s how WSUS works. Find the patch by searching for the KB, and once you right click ‘Approve’, you’ll get the option to choose ‘Approved for Removal’. Make sure you’re targeting the correct Computer Group. If you can’t use WSUS, work out how to get your PCs to run a command like this: “wusa /uninstall /kb:3097877 /quiet /norestart” – without the /norestart, they’ll restart :)

Step 4 – Test Windows Update uninstall
Test another PC’s ability to use Windows Updates to uninstall the patch. ‘Checking for updates’ either through the Windows Update GUI or the good old ‘wuauclt /detectnow’ command will do the trick. Similar to Step 2, check it uninstalls and reboot. You can also check C:\Windows\WindowsUpdate.log to make sure it’s happy (this doesn’t apply to Windows 10 as that log doesn’t exist).

Step 5 – Trigger your PCs to check for Windows Updates
Depending on your group policies, Windows Updates will check at certain intervals and may auto download or auto patch. Easiest thing to do is trigger all your PCs to check Windows Updates now. There’s an easy PowerShell way of doing this here, but requires WinRM to be enabled – you should have this on if you want to be able to do a bunch of cool stuff to your PCs. Otherwise, try psexec which will have the same result. This can take a long time to do! Optional component – WOL your PCs first.

Step 6 – Reboot
Now that you’re ready to clean up, test reboot a PC or two and make sure the patch goes away. If that happens, then schedule all your PCs to reboot. You should have a way of doing this already – SCCM can do it well, you can create a once off scheduled task and push that out to PCs, or a bunch of other ways.

Step 7 – Report in WSUS
WSUS has some nice client reporting options. Search for the KB again, right click and choose ‘Status Report’. This is usually not too lagged in it’s information, and you can check to make sure none of your PCs have the update any more. If there’s only a few, it may be easier to manually fix the remainder.
A SAuthor Commented:
thanks for the comments
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.