SCCM 2012- windows patching

I have deployed the windows patches for this month and it went successfully except for few servers. Some server had to be manually installed by service desk team.Now , i 've got a ticket to investigate the issue why my servers had to be manually patched. I have gone through SCCM logs and client server logs and found nothing. Please help me out as i have tried every where and done almost everything.
Jimmy SanAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Jimmy SanAuthor Commented:
I have deployed the windows patches for this month and it went successfully except for few servers. Some server had to be manually installed by service desk team.Now , i 've got a ticket to investigate the issue why my servers had to be manually patched. I have gone through SCCM logs and client server logs and found nothing. Please help me out as i have tried every where and done almost everything.
0
MaheshArchitectCommented:
There is no concrete solution for this problem

Because virtually you don't have much control over sccm agent installed on servers

Sometimes sccm agent unable to respond sccm server advertisement, in that case it might skip the patches

Just ensure below
SCCM client status is approved in sccm console for those servers
try to fetch all sccm policies including life and machine cycles from sccm client
You might try to flush wmi repository
U might reinstall sccm agent

however this will not give you RCA essentially

If your server count is less, better option could be do patch management manually through some kind of .bat file
Because I have face this problem some times with my clients for which I have to either cancel package advertisement or some times those server woke up very late for patches or they never realize that there are patches out for them, and then I need to pay attention, this behavior causes increase in overheads instead of automation.

Some of my clients use bigfix / LANDesk which seems to have better results
0
Dan McFaddenSystems EngineerCommented:
On the servers that failed to patch, was there a specific error message for the patches that failed to get installed?  If you open the SCCM client on the server, is the status of the patch... "Failed"?

If a server does seem to get the advertisements, you can force a rescan from the client side so the server checks it patch state against the sccm server's list.

Dan
0
How do you know if your security is working?

Protecting your business doesn’t have to mean sifting through endless alerts and notifications. With WatchGuard Total Security Suite, you can feel confident that your business is secure, meaning you can get back to the things that have been sitting on your to-do list.

Jimmy SanAuthor Commented:
Yes they have failed, i checked the software distrubtion event log and found that they are failed on some servers.
0
Nagendra Pratap SinghDesktop Applications SpecialistCommented:
Select a single server and work on it.
0
Dan McFaddenSystems EngineerCommented:
Normally there is an error code that can be found in the CfgMgr client.  Looks something like:  0x080040AC

These are specific to the failure and can really be a big help.

In the last 2 or 3 months, my SCCM pushed patches have been hitting an installation timeout issue.  This is was quite evident with May's patches.

As an example of info that would help, here is a shot of a failed patch on 1 of my servers:

SCCM-Failure-Message.JPG
If you could post a failure message or 2, we might be able to find a resolution.

Dan
0
Jimmy SanAuthor Commented:
No matter which month we apply May or june , we always get this problem. The way it works its that we deploy the patches  and schedule it for Saturday and have our service team to follow up. In case , some servers fail to install the updates, service team manually install the updates.  I am attaching some screen shoots to illustrate.
software-event-log.png
Log-event.png
software-distribution.png
0
Dan McFaddenSystems EngineerCommented:
So, the error that is being reported in the event log is 2149842967... in decimal.  Converting it to Hex you'll get:  0x80240017

Looking up the error code on Microsoft's Windows Update Agent Result Codes page... you get:


0x80240017,  WU_E_NOT_APPLICABLE,  Operation was not performed because there are no applicable updates.

In other words, the patches that are failing are being reported as not applicable to the version of the Microsoft OS you are attempting to install them on.

It may be worth uninstalling and reinstalling the SCCM client on an effected server.  Then test if the patches are pushed after a scan cycle.

Are you bundling all the updates into a single Software upgrade group or do you have separate update groups based on system OS?

Dan
0
Jimmy SanAuthor Commented:
We have separate collection for all  the different OS. For instance, w2k3 32 bit , w2k364 bit and so on. The screen shot I have provided you , its the time when service team use the tool to deploy the patches manually. I am so tired with this investigation as I have to report it to my manager.
0
Dan McFaddenSystems EngineerCommented:
Do you use dynamic or statically built groups?  Are the servers that weren't patched, in the proper device collections in SCCM?

Dan
0
Jimmy SanAuthor Commented:
This is that I don't know but I guess they use dynamically when the server is build  and then put it in to the right collection afterwards.
0
Dan McFaddenSystems EngineerCommented:
I think you Need to find this Info out. You can't completely troubleshoot SCCM patch deployment  issues without having a view into how things are setup in Configuration Manager.

The OS Deployment groups are irrelevant to patch deployment.

The best you can do without access to the SCCM server is follow the article below and gonthru all the mentioned logs to see if you can ID the source of the failure.

Link: http://blogs.technet.com/b/sudheesn/archive/2010/05/31/troubleshooting-sccm-part-i-client-push-installation.aspx

Dan
0
Jimmy SanAuthor Commented:
Ok so i have gone through the article and looked for the logs on client side as i do not have access to sccm server(only see the citrix sccm console). Agent has been scanning fine and there are tons of logs in ccm folder which hardly make any sense to me.Do you think this could be cus of Ncsi.dll file?
0
Nagendra Pratap SinghDesktop Applications SpecialistCommented:
You will not like this answer but here it goes.

For SCCM troubleshooting you need.

SCCM Admin access
Client remote/admin access
Some SCCM training.
0
Dan McFaddenSystems EngineerCommented:
Zemran, no I do not believe that the patching issues you are experiencing are due to the ncsi.dll.  It has to do with Network Awareness:

Link:  https://technet.microsoft.com/en-us/library/cc766017(v=ws.10).aspx

One thing you could do, on the servers where the patching issues exist, make sure that they have a properly identified the active network as a Domain Network.  You may also want to test turning off the firewall (if its not already off) on an effected server to see if has been blocking the SCCM client/server communication.

Again, as I mentioned earlier, and Nagendra as well, troubleshooting this issue without knowing how the SCCM Application is setup to work, will make this situation quite hard to figure out.

Can you please post the following log files, so I can get a look at what happened on a failed installation.

Please post the following files from "C:\Windows\CCM\Logs" :

1. LocationServices.log
2. WUAHandler.log
3. Updatedeployment.log
4. Execmgr.log
5. RebootCoordinator.log

Dan
0
Jimmy SanAuthor Commented:
0
Dan McFaddenSystems EngineerCommented:
Sorry for the delay.  So here's what I find in the logs posted>

It appears that updates were advertised but the pending reboots were not setup as required.

You see these log entries throughout the RebootCoordinator.log"

<![LOG[Entered ScheduleRebootImpl - requested from 'UpdatesDeploymentAgent'. set Rebootby = 0. set NotifyUI = True. set PreferredRebootWindowType = 4]LOG]!>

Open in new window



This "set Rebootby = 0" means the reboot is not required and will not automatically reboot the server.

In the WUAHandler.log, I see that updates were found on these dates:

1. 04-18-2015
1a. 9 updates were installed
1b. a pending reboot was indicated but it is configured as non-mandatory
2. 04-25-2015
2a. 1 update was installed
2b. a pending reboot was indicated but it is configured as non-mandatory
3. 05-23-2015
3a. 10 updates were installed
3b. a pending reboot was indicated but it is configured as non-mandatory
4. 06-14-2015
4a. 4 updates were installed
4b. a pending reboot was indicated but it is configured as non-mandatory

So to me it would appear that the patches did deploy but the required reboot was not executed because it was defined as non-mandatory.

If this is a server that had the patch installation issues mentioned in the OP, then I would investigate the Update Deployment Package.

Dan
0
Jimmy SanAuthor Commented:
Dan,

Updates were scheduled for 13 june 3.pm. The log you were referring it to 14 june , It was the day that service team use the tool to manually deploy the patches. I was initially heading it to the same direction but I see no updates being installed on 13 th june. I hope it makes sense to you
0
Dan McFaddenSystems EngineerCommented:
Then here is proof that no updates were being advertised or available on 13-JUN-2015:

<![LOG[Attempting to install 0 updates]LOG]!><time="22:06:00.302+240" date="06-12-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="6056" file="updatesmanager.cpp:1581">
<![LOG[No actionable updates for install task. No attempt required.]LOG]!><time="22:06:00.302+240" date="06-12-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="6056" file="updatesmanager.cpp:2914">
<![LOG[Updates could not be installed at this time. Waiting for the next maintenance window.]LOG]!><time="22:06:00.302+240" date="06-12-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="6056" file="updatesmanager.cpp:1600">

<![LOG[CUpdateAssignmentsManager received a SERVICEWINDOWEVENT END Event]LOG]!><time="00:00:00.249+240" date="06-13-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="7852" file="assignmentsmanager.cpp:277">
<![LOG[CUpdateAssignmentsManager received a SERVICEWINDOWEVENT START Event]LOG]!><time="00:00:00.264+240" date="06-13-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="2708" file="assignmentsmanager.cpp:277">

<![LOG[Failed to ConnectSettings for ICcmPolicyAgent in CSoftDistPolicyNamespace::ConnectToNamespace]LOG]!><time="00:04:00.317+240" date="06-13-2015" component="UpdatesDeploymentAgent" context="" type="3" thread="2708" file="cliuisettings.h:95">
<![LOG[Failed to ConnectSettings for ICcmPolicyAgent in CSoftDistPolicyNamespace::ConnectToNamespace]LOG]!><time="00:06:00.336+240" date="06-13-2015" component="UpdatesDeploymentAgent" context="" type="3" thread="2708" file="cliuisettings.h:95">
<![LOG[Auto install during non-business hours is disabled or never set, selecting only scheduled updates.]LOG]!><time="00:06:00.336+240" date="06-13-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="2708" file="assignmentsmanager.cpp:500">
<![LOG[A user-defined service window(non-business hours) is available. We will attempt to install any scheduled updates.]LOG]!><time="00:06:00.336+240" date="06-13-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="2708" file="updatesmanager.cpp:1558">
<![LOG[Attempting to install 0 updates]LOG]!><time="00:06:00.336+240" date="06-13-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="2708" file="updatesmanager.cpp:1581">
<![LOG[No actionable updates for install task. No attempt required.]LOG]!><time="00:06:00.336+240" date="06-13-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="2708" file="updatesmanager.cpp:2914">
<![LOG[Updates could not be installed at this time. Waiting for the next maintenance window.]LOG]!><time="00:06:00.336+240" date="06-13-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="2708" file="updatesmanager.cpp:1600">


<![LOG[CUpdateAssignmentsManager received a SERVICEWINDOWEVENT END Event]LOG]!><time="00:00:00.244+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="3928" file="assignmentsmanager.cpp:277">
<![LOG[CUpdateAssignmentsManager received a SERVICEWINDOWEVENT START Event]LOG]!><time="00:00:00.244+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="7732" file="assignmentsmanager.cpp:277">
<![LOG[Failed to ConnectSettings for ICcmPolicyAgent in CSoftDistPolicyNamespace::ConnectToNamespace]LOG]!><time="00:04:01.919+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="3" thread="7732" file="cliuisettings.h:95">
<![LOG[Failed to ConnectSettings for ICcmPolicyAgent in CSoftDistPolicyNamespace::ConnectToNamespace]LOG]!><time="00:06:01.937+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="3" thread="7732" file="cliuisettings.h:95">
<![LOG[Auto install during non-business hours is disabled or never set, selecting only scheduled updates.]LOG]!><time="00:06:01.937+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="7732" file="assignmentsmanager.cpp:500">
<![LOG[A user-defined service window(non-business hours) is available. We will attempt to install any scheduled updates.]LOG]!><time="00:06:01.937+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="7732" file="updatesmanager.cpp:1558">
<![LOG[Attempting to install 0 updates]LOG]!><time="00:06:01.937+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="7732" file="updatesmanager.cpp:1581">
<![LOG[No actionable updates for install task. No attempt required.]LOG]!><time="00:06:01.937+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="7732" file="updatesmanager.cpp:2914">
<![LOG[Updates could not be installed at this time. Waiting for the next maintenance window.]LOG]!><time="00:06:01.937+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="7732" file="updatesmanager.cpp:1600">
<![LOG[Service startup system task]LOG]!><time="04:41:50.905+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="4584" file="systemtasks.cpp:30">
<![LOG[Software Updates feature is enabled]LOG]!><time="04:41:50.936+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="4584" file="cdeploymentagent.cpp:56">
<![LOG[CTargetedUpdatesManager::DetectRebootPendingUpdates - Total Pending reboot updates = 0]LOG]!><time="04:41:52.653+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="4584" file="updatesmanager.cpp:2520">
<![LOG[Suspend activity in presentation mode is selected]LOG]!><time="04:41:52.684+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="4584" file="cliuisettings.h:163">
<![LOG[At least one user has elected to suspend non-business hours activity when in presentation mode. Checking for presentation mode.]LOG]!><time="04:41:52.684+240" date="06-14-2015" component="UpdatesDeploymentAgent" context="" type="1" thread="4584" file="deploymentutils.cpp:711">

Open in new window


In the first few lines you'll see nothing available for patching.

In the next 2 line, the request for a Service Window Event

The next 7 lines are on the 13th and reports a policy error, looking for scheduled updates and 0 were found.

The last of the section shows a patch cycle scan on the 14th still finding no updates available for installation.

Dan
0
Jimmy SanAuthor Commented:
what it means , its that there is some policy error on server causing it not install?
0
Dan McFaddenSystems EngineerCommented:
I would show the above log sections to your SCCM admins.

You could ask them what they think this means:

Failed to ConnectSettings for ICcmPolicyAgent in CSoftDistPolicyNamespace::ConnectToNamespace

Open in new window


It may mean that you should uninstall and reinstall the SCCM agent on the affected servers.  Here is reference forum post that describes similar issues as your servers:

Link:  https://social.technet.microsoft.com/forums/systemcenter/en-US/471d1211-5dbe-481f-aa85-2690b8089070/sccm-client-problem

On the server end, I would have you check with the admins and ask about the 06/13/2015 patch advertisement.  As stated from the above logs, there were no patches found to be available on the 13th.  They appeared on the 14th.  It could be that a software inventory scan and patch scan happened before the patches were released.  That would explain why the clients didn't see the update package.

As mentioned previously, you need to have cooperation from the SCCM Admins to try to identify the source of the issue.  Without their cooperation, your troubleshooting scope is limited to the clients.  Even though there is a possibility that there may have been issues elsewhere.

Dan
0
Jimmy SanAuthor Commented:
Ok I am currently working on it , let me get back to you once I have something
0
Jimmy SanAuthor Commented:
Ok Dan
I have this logs for another where deployment was advertised on june 10 and I see no logs on anywhere?
WUAHandler.log
UpdatesDeployment-20150619-135704.log
0
Jimmy SanAuthor Commented:
Looks like i found the problem for the 50 % of computers. Three things to look ,Bit services,windows update and sms host services, all these services have to be runing Quite often or not , sms host services get corrupted and cause the sccm agent not to communicate with wsus.in an event when you start the sms host service and gets an error. Follow this path and run this file and it will start the sms host services.c\windows\ccmsetup\ccmsetup.exe.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Dan McFaddenSystems EngineerCommented:
Have you disabled the automatic remediation on the SCCM clients?

By default, the SCCM client installation creates a scheduled task that periodically checks the health of the SCCM Client installation.   If it detects any issue, it attempts to reinstall the client software.

Here is a link that describes how to disable the Automatic Remediation of the client health.

link:  http://blogs.technet.com/b/seanm/archive/2012/12/20/configuration-manager-client-health-disable-automatic-remediation-for-selected-machines.aspx

link:  http://blog.coretech.dk/kea/configure-client-remediation-in-configmgr-2012-to-monitor-only-using-settings-management/

It talks about how to disable it, but you could look at the clients that you are having problems with and see if these configuration items are enabled or disabled.

In the WUAHanlder.log above, I see updates being installed on the 10th at 15:40... 3 updates were installed and 2 required a reboot.

<![LOG[Going to search using WSUS update source.]LOG]!><time="15:38:21.281+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1256">
<![LOG[Synchronous searching of all updates started...]LOG]!><time="15:38:21.281+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:591">
<![LOG[Successfully completed synchronous searching of updates.]LOG]!><time="15:39:03.625+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:643">
<![LOG[1. Update: 2b2d8c36-5b91-408e-b969-bb9337107329, 201   BundledUpdates: 1]LOG]!><time="15:39:03.640+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1361">
<![LOG[       Update: a1dbda97-323a-4b9f-a465-2c1f211ab76a, 201   BundledUpdates: 0]LOG]!><time="15:39:03.640+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1361">
<![LOG[2. Update: 3326d83c-c716-48f8-a194-fe8cae70a8dd, 201   BundledUpdates: 1]LOG]!><time="15:39:03.640+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1361">
<![LOG[       Update: cfcb7c81-5415-4403-b6de-92def95f07fe, 201   BundledUpdates: 0]LOG]!><time="15:39:03.640+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1361">
<![LOG[3. Update: 9648fd02-1304-4b34-aff2-666fa7c1ef1a, 206   BundledUpdates: 1]LOG]!><time="15:39:03.640+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1361">
<![LOG[       Update: 7b4e66c9-a915-48b9-a0d5-65b54d54e883, 203   BundledUpdates: 0]LOG]!><time="15:39:03.640+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1361">
<![LOG[1. Update (Missing): Cumulative Security Update for Internet Explorer 6 for Windows Server 2003 x64 Edition (KB3058515) (2b2d8c36-5b91-408e-b969-bb9337107329, 201)]LOG]!><time="15:39:03.640+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1483">
<![LOG[2. Update (Missing): Security Update for Windows Server 2003 x64 Edition (KB3033890) (3326d83c-c716-48f8-a194-fe8cae70a8dd, 201)]LOG]!><time="15:39:03.890+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1483">
<![LOG[3. Update (Missing): Security Update for Windows Server 2003 x64 Edition (KB3057839) (9648fd02-1304-4b34-aff2-666fa7c1ef1a, 206)]LOG]!><time="15:39:03.890+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1483">
<![LOG[Async installation of updates started.]LOG]!><time="15:39:07.875+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:1844">
<![LOG[Update 1 (2b2d8c36-5b91-408e-b969-bb9337107329) finished installing (0x00000000), Reboot Required? Yes]LOG]!><time="15:40:00.703+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="35912" file="cwuahandler.cpp:1997">
<![LOG[Update 2 (3326d83c-c716-48f8-a194-fe8cae70a8dd) finished installing (0x00000000), Reboot Required? No]LOG]!><time="15:40:23.093+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="36388" file="cwuahandler.cpp:1997">
<![LOG[Update 3 (9648fd02-1304-4b34-aff2-666fa7c1ef1a) finished installing (0x00000000), Reboot Required? Yes]LOG]!><time="15:40:42.187+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="35312" file="cwuahandler.cpp:1997">
<![LOG[Async install completed.]LOG]!><time="15:40:42.203+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="38324" file="cwuahandler.cpp:2024">
<![LOG[Installation of updates completed.]LOG]!><time="15:40:42.203+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="33896" file="cwuahandler.cpp:3898">
<![LOG[Scan results will include superseded updates only when they are superseded by service packs and definition updates.]LOG]!><time="15:40:42.343+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:2909">
<![LOG[Search Criteria is ((DeploymentAction=* AND Type='Software' AND CategoryIDs contains '0FA1201D-4330-4FA8-8AE9-B877473B6441'))]LOG]!><time="15:40:42.343+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:2916">
<![LOG[Async searching of updates using WUAgent started.]LOG]!><time="15:40:42.359+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37216" file="cwuahandler.cpp:579">
<![LOG[Async searching completed.]LOG]!><time="15:40:54.750+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="37264" file="cwuahandler.cpp:2068">
<![LOG[Successfully completed scan.]LOG]!><time="15:40:57.453+240" date="06-10-2015" component="WUAHandler" context="" type="1" thread="38144" file="cwuahandler.cpp:3557">

Open in new window


Dan
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Microsoft Applications

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.