Situation:
All servers in our a newly-formed environment required a first set of ‘baseline’ patches to bring them up to speed, after which we would set up a monthly patch update schedule.
Following this ‘inaugural’ deployment of around 380 patches. everything appeared on the face of things, to go smoothly. Machines all received updates and rebooted as expected. I could see from reports however that one or two patches had failed to install for some reason in this deployment. I figured that we could schedule in another time to try these again, maybe even the following month as none were actually deemed critical. In any case this was another conversation I needed to have with the application owners.
Two days later I got a call saying that a sizeable percentage of the server collection I had patched ‘randomly’ rebooted again at various times that morning. A significant amount of arm waving ensued and I was asked to look into it, especially as it was clear from the event logs that the reboot had been instigated by the CCMExec:
The process C:\Windows\CCM\CcmExec.exe (COMPNAMEXXX) has initiated the restart of computer COMPNAMEXXX on behalf of user NT AUTHORITY\SYSTEM for the following reason: No title for this reason could be found Reason Code: 0x80020001 Shutdown Type: restart Comment: Your computer will restart at 08/14/2015 09:58:37 AM to complete the installation of applications and software updates
I frantically checked all deployments and confirmed there was absolutely no deployments due and everything had basically gone out as expected. In fact nobody had even touched the console that morning. Further digging revealed the following in the logs of affected machines:
Log Name: System Source: Microsoft-Windows-WindowsUpdateClient Date: 12/08/2015 17:24:00 Event ID: 20 Task Category: Windows Update Agent Level: Error Keywords: Failure,Installation User: SYSTEM Computer: COMPNAMEXXX.domain.local Description: Installation Failure: Windows failed to install the following update with error 0x80092004: Update for Windows Server 2008 R2 x64 Edition (KB3045645).
Then, later on:
Log Name: System Source: Microsoft-Windows-WindowsUpdateClient Date: 14/08/2015 10:02:43 Event ID: 19 Task Category: Windows Update Agent Level: Information Keywords: Success,Installation User: SYSTEM Computer: COMPNAMEXXX.domain.local Description: Installation Successful: Windows successfully installed the following update: Update for Windows Server 2008 R2 x64 Edition (KB3045645)
Similar entries appeared for several of the failed updates. Clearly what had happened is that another software update scan cycle had taken place and detected that the machine was missing several important updates, specifically those that had failed previously. The updates were then automatically re-tried and successfully installed the second time round. Presumably they needed a reboot before they would install correctly although I have read plenty to suggest this shouldn’t happen. Regardless, the second successful installation instigated our phantom reboots.
My recommendation here would be to ensure you have a dedicated Maintenance Window to alleviate this headache. There were one or two that picked these updates up and installed them after my discovery and the Maintenance Window prevented the reboot from occurring. Obviously it won’t be properly patched until the client has been restarted but that can at least be left to the application owner to decide on a suitable time to do this.
Very short and (not so) sweet post with a word of warning on ADRs.
CM2012 SP1 R2
I wanted to set up and Automatic Deployment Rule to select a bunch of updates and automatically download and set up my Software Update Group. In my client’s environment the only box with internet access belongs to the SUP. Unfortunately, if you want your ADR to carry out the tasks above then you will need to ensure your Primary Site Server has access to the internet as well.
An ADR will always fire from the Primary Site Server and without internet access, the ADR will fail to download any updates.
You’ve been warned!