Category Archives: Software Updates

Failed to download file- “http://officecdn.microsoft.com/xxx/xxx.cab” from internet. Using proxyserver – 0. Error = 12029

This one caught me out recently. Scenario:

All http://officecdn.microsoft.com is whitelisted and available, definitely no issue there.

  • All other updates seem to come down fine.
  • There are no ADR rules at all in place.
  • Running a full sync produces the error below:

My client downloads updates through a proxy server and naturally the correct details for the proxy were in place on the Site System Settings under Site System Roles. Also the checkbox for Use a proxy server when synchronizing software updates was checked under the Proxy And Account Settings tab under the Software Update Point Properties role.

SOLUTION:

Despite the fact that ADRs were NOT created or used in any way, it transpired that the checkbox Use a proxy server when downloading content by using automatic deployment rules also had to be checked. Once this was selected, O365 updates started syncing.

 

Adding a Software Update Point to a Secondary Site (for those who already know how to add it to the primary site!)

This is something I’ve meant to get round to writing about for months as the first time I did this I couldn’t find any direct answers in the multitude of blogs I read about it.

On the face of things, installing WSUS/SUP on a secondary site sounds pretty straightforward if you’re used to adding them to a primary site but when you start it soon becomes clear there are a few unanswered questions which start to materialise. It’s important to first consider whether a SUP is even required for the secondary site: remember, the update packages will be present anyway. The secondary site SUP is really only there for scanning purposes and to relieve associated traffic which is really quite minimal. Assuming you are over this and need it anyway then here are a few other things to consider, some of which have changed since CM2007.

  1.  Since CM2012, there is now a DB on the secondary site so should we be installing to that?
  2.  WSUS Content?
  3. How does it interact with the Primary SUP?

This blog is only intended to answer one or two questions you might have as regards specifics of installing on a secondary site and there are plenty of answers out there for general installation. In short, the SUP on the secondary site is a secondary SUP and when you install it ConfigMgr will actually take care of marrying it up to the Primary SUP.

So the high level tasks are as follows:

  1. Install the pre-req’s for a Software Update Point. I thoroughly recommend Nickalaj A’s PreRequisite Tool  for this job, see https://gallery.technet.microsoft.com/ConfigMgr-2012-R2-e52919cd. I must confess this has made me lazy and I barely remember what the pre-reqs are these days but to be honest I don’t need to with this.
  2. Install the database on the secondary site SQL instance. Best practice is to have separate instances but I’ll leave that to you. Personally I don’t find there is any real performance issues using the same instance. Also, as an aside, if you have full SQL on the primary, (and you will) then I don’t really see why you wouldn’t use it on the secondary. Why use SQL Express when the licence is free for the full version? Just my opinion though.
  3. WSUS Content should always be local. It’s a requirement to specify this during installation but it’s not really used when part of a SUP in ConfigMgr.
  4. Finally, after WSUS installation, add the SUP role to the site. ConfigMgr will take care of the rest. Click on the secondary site under  the Sites node, and click the SUP under the site component properties to check the sync status. It should be greyed out because SCCM has recognised it as a child of the primary SUP.
  5. A good, detailed explanation of the whole process can be found here

Happy Deployments…

WSUS / SUP : System.Net.WebException: The request failed with HTTP status 401: unauthorized.

I noticed recently that after an extended period of being switched off, the Software Update Point in SCCM lab looked extremely poorly. I’m not sure why this was or if it had anything to do with being switched off for several days but in any case here is the scenario:

WSUS lives on a separate server to my site server and SQL is on another separate box (I know, better to install it on the same box as site server but I find few customers these days that’ll let me do this so I keep it this way to replicate their environments as far as possible). Anyway I digress; the setup is as follows:

Comms: HTTPS / SSL throughout  for SCCM and for WSUS.
Version: Current Branch 1606
OS: Server 2012 R2  (WSUS 6.2, commonly referred to as WSUS 4.0)

After noticing some errors in my component status messages with regard to WSUS, I checked the WSUSCtrl.log and saw the following message appearing every minute or so:

System.Net.WebException: The request failed with HTTP status 401: Unauthorized.~~ at Microsoft.UpdateServices.Administration.AdminProxy.CreateUpdateServer(Object[] args)~~ at Microsoft.UpdateServices.Administration.AdminProxy.GetUpdateServer()~~ at Microsoft.SystemsManagementServer.WSUS.WSUSServer.ConnectToWSUSServer(String ServerName, Boolean UseSSL, Int32 PortNumber)
Failures reported during periodic health check by the WSUS Server UT1.BC.LOCAL. Will retry check in 1 minutes

Furthermore, if I ran WSUSUtil checkhealth on the SUP, my Application Log read as follows:

The Reporting Web Service is not working.
 The API Remoting Web Service is not working.
 The Server Synchronization Web Service is not working.
 The Client Web Service is not working.
 The SimpleAuth Web Service is not working.
 The DSS Authentication Web Service is not working.
 On 13/10/2016 19:56:06, component SMS_WSUS_CONTROL_MANAGER on computer UT1.TEST.LOCAL reported: WSUS Control Manager failed to configure proxy settings on WSUS Server "UT1.TEST.LOCAL".

Possible cause: WSUS Server version 3.0 SP2 or above is not installed or cannot be contacted.
 Solution: Verify that the WSUS Server version 3.0 SP2 or greater is installed. Verify that the IIS ports configured in the site are same as those configured on the WSUS IIS website.You can receive failure because proxy is set but proxy name is not specified or proxy server port is invalid.

Not good. Fortunately the fix was straightforward:

I ran c:\Program Files\Update Services\Tools\wsusutil.exe configuressl ut1.test.local
and then I saw URL: https://ut1.test.local:8531 appear on the screen.
Then restarted the IIS services (IISAdmin, WWW) and all sprang to life. An IISReset would probably have done the same thing. After this the log should start to look like that below.

wsuslog

(Open image in a new tab to see more clearly)

Assuming you are configured for SSL and for some reason you see something like URL: http://ut1.test.local:8530 instead, then most likely the SSL settings for WSUS are probably incorrect. Ensure you have the settings below in place in IIS:

1. WSUS Administration. SSL Settings should be unchecked / ignore.
2. ApiRemoting30. SSL settings should be checked / ignore.
3. aspnet_client. SSL settings should be unchecked / ignore.
4. ClientWebService. SSL settings should be checked /ignore
5. Content. SSL settings should be unchecked / ignore.
6. DSSAuthWebService. SSL settings should be checked/ ignore.
7. Inventory. SSL settings should be unchecked / ignore.
8. ReportingWebService. SSL settings should be unchecked / ignore.
9. SelfUpdate. SSL settings should be unchecked / ignore.
10. ServerSyncWebService. SSL settings should be checked / ignore.
11. SimpleAuthWebService. SSL settings should be checked / ignore.

‘The software change returned error code 0xc1800118 (-1048575720)

So you’re testing out deployment of the new 1607 feature upgrade for Windows 10 in your shiny new 1606 SCCM console. The upgrade appears in your client’s Software Center and starts downloading* and then installing…and then disaster. You are hit with the dreaded message The software change returned error code 0xC1800118(-1048575720). Some Googling reveals a lot of the same responses, ie that you’ve allegedly forgotten to apply KB3159706 before synchronising. (Just in case this happens to be the case, check out the MS blog with appropriate links) but of course you we all know you applied that correctly, so what else?

It goes without saying that if you’re up and running with the official advice then leave as is and don’t fiddle further unnecessarily. However, if you’re still reading then the chances are that that isn’t the case. Well there are a number of other solutions describing how you must add the MIME type for the WSUS server as follows:

extension: .esd   MIME Type: Application/octet-stream
or
extension: .esd   MIME Type: application/vnd.ms-cab-compressed

mimetype

Try these for sure. Ensure you restart the IIS web services after adding the entry.

Again, however, this didn’t fix my issue. Maybe I was looking in the wrong places but I could find no information on the internet which fitted my particular set of circumstances, although there appear to be many in the same boat with unanswered forum posts.

RESOLUTION

After some head-scratching I did find a solution that worked. I simply reinstalled the the SCCM client agent from the client folder on the site server. Huh? This didn’t make too much sense to me until I took a closer look at the client version number before and after.
My freshly-built Windows 10 1511 builds were installing with client version 5.00.8355.1307:

8355

After re-installation of the new client, the version number was 5.00.8412.1307:

8412

Following this I was able to make the feature upgrade run without error. Subsequent SCCM servicing upgrades also upgraded the client files in this directory and the new client version has clearly fixed some compatibility issues with the 1607 feature upgrade.

I updated the default SCCM client version in the console and ensured all new 1511 builds were running the newer client version. Sure enough, feature updates worked perfectly following every new build. I then rolled out the new client agent version to my existing estate. This will obviously be a pre-requisite to ensure all machines upgrade as expected when the 1607 feature upgrade is rolled out wholesale.

Hope these tips help someone else suffering with upgrade frustrations.

* Another issue I have seen is where the feature upgrade gets stuck downloading at 0%. I solved this problem by simply creating a new package and re-downloading the feature upgrade and re-deploying.

Software Updates My Way :-) (plus a useful tool to help out)

There are a 101 ways of deploying software updates out there and you’ll undoubtedly get a different answer on the best way to deploy updates from every ConfigMgr admin you ask. Indeed it obviously depends on the environment and company politics how best the solution is implemented. This is my tuppence worth.

As an admin, you have enough to do in setting up the required updates in the first place. In many environments you’ll then have to raise the necessary change requests and all the headaches that entails. Then there is the deployment itself:
“Why did it install that?”
“Why didn’t it install that?”
My solution to this is to move the onus to the application or server owner to set up when their computers are patched.

1, STRATEGY:

Select you maintenance window(s). Ensure the window is of sufficient length to take into account all the updates that need deploying. If a ‘hard’ maintenance window is in force, bear in mind you might not finish installing and rebooting before it closes if it is set too short. For my example I will select three per day for different deployment types. This doesn’t mean there will necessarily be computers populated in all three collections, but the option is there. Agree these with all interested parties first. For example:

1800-2000 (Special applications)
2300-0100 (Workstations)
0300-0500 (Servers)

2. DURATION:

By duration I’m talking about how long to complete the deployment to the enterprise. For me, there are two scenarios here. I will typically roll out patches to the servers and/or workstations in my test domain before rolling out to live. The duration for the full rollout is 14 days for each environment (test/non prod and production). This (usually) leaves a few days in between patch Tuesday to download the new updates and deliver the update list for the coming month to interested parties to approve, before rollout on the following Sunday (‘Patch Sunday’). In many situations you may not have the advantages of a test environment to hand. In these situations you may potentially argue that a longer rollout period was necessary. However there are few situations I have ever seen where a 14 day cycle isn’t sufficient if proper testing is put in place.

3. DEFER OWNERSHIP:

As alluded to earlier, IMO the onus should be on the SCCM admin to set up the job but after that it should be up to the application owner or server team to administer when their servers are patched. Ideally the patch window for the server should be set at the server build stage and should be specified in the original request.
My approach to this is to ensure all the software update collections have a query rule assigned which defines a particular AD group that can be updated by the interested party. Responsibility then falls squarely on the owner for any problems related to when the machine is updated.

4. AD GROUPS, QUERIES AND COLLECTIONS
This is how I set up my collections in SCCM along with the queries and groups.
EXAMPLE AD GROUP:
Name: PR1-SCCM PRD-07-2300-0100 (Site PR1, Prod environment, day 7, 2300-0100)
EXAMPLE COLLECTION:
Software Updates – WK1 – Saturday – 2300-0100 or Software Updates – WK2 – Wednesday- 2300-0100
EXAMPLE QUERY RULE:
select SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client from SMS_R_System where SMS_R_System.SystemGroupName = ‘DOMAIN\\PR1-SCCM PRD-07-2300-0100’ and SMS_R_System.Name not in (select distinct SMS_R_System.Name from SMS_R_System where SMS_R_System.SystemGroupName = ‘DOMAIN\\MW Exception Group’)
Note:
DOMAIN\\MW Exception Group represents a group that a computer can be dropped into if for some reason it needs removing from its usual schedule temporarily.

5. NO REBOOT GROUP
In addition to the three daily maintenance window collections, another daily collection I have set up is one with no reboot. Indeed this is of particular interest to certain application owners who only have a narrow window where servers can be down. They take the responsibility on themselves to manually reboot these machines at a time convenient to them.

6. ADMINISTRATION
So…if should you decide the above all sounds like a great idea (that’s a big if, but go with me here) we’re talking about over 60 collections to deploy updates to. That’s going to take all week to set up isn’t it, particularly if there are a lot of update to deploy? Well yes, it is. If done manually. Powershell would traditionally be your friend here and when I first started this, that is exactly what I was using. What I found in practice though was even with this I wanted to see what was going to happen first and again you could write scripts to do this too. What I decided on though was to write a tool that would do everything for me. When I say everything, I am including creating collections, hard maintenance windows (if required), AD groups, queries and of course the selection, deployment schedule and deployment of the updates themselves. This tool is adaptable for different environments and should make the whole process very simple. Whilst I take no responsibility for how you use it in your environment, I have used it in live and prod environments at clients I have consulted and all sorts of different test environments. I include the download and source code at the end of this blog.

7. TO WRAP UP

This blog has detailed my specific approach but the high-level is really what it’s about. As I mentioned in my opening paragraph, one size does not fit all but I do believe some of the pressure (and work) can and should take a shift sideways. Some computers you’ll most likely always have to deal with, eg workstations/laptops. The systems that cause the most headaches though are the servers and I firmly believe ownership can and should be shifted here. It won’t work everywhere but it’s all about changing culture.

Toolio_SoftwareUpdateTool_Instructions

ToolioSoftwareUpdateTool_Source_v2.9.1 (C# source code)

Toolio_Installer_v2.9.1 (exe)

Let me know any issues you have and hopefully I’ll be able to help you out. Please make sure you test before putting into production and I take no responsibility for use!

 

Patching workgroup computers and template VMs in CM2012

I recently needed to update the templates of our VMWare servers to reduce the amount of patching required after each deployment. IMO it’s good practice to do this every six months or so and reduces the time it takes for the server to be in a state that is ready for business.

Our template is essentially a non-domain-connected workgroup with no licence key as you’d expect (this all gets added during the orchestration process). As such this presents additional challenges as regards deployment from SCCM which would typically be to a domain-joined machine. The process below can be used for any deployment  to a workgroup-based machine of any type from SCCM, it just so happens I need to do this for software updates to a server.

1.) Ensure you have a local administrative account set up on the workgroup machine, eg %computername%\SBond in my example below.
2.) RDP onto the machine using account above. Copy the CM2012 client install folder from a site server (eg Primary or MP) and paste it via an RDP session on the desktop.
TIP: Copy over CMTrace.exe too, you’ll more than likely use it for something.
3.) Add the local account to ConfigMgr:
UserAccount

4.) Bring up an administrative cmd prompt and CD to C:\Users\<localuser>\Desktop\Client. Run the following command (update as appropriate):

ccmsetup.exe SMSSITECODE=WPR SMSMP=<APPROPRIATE MANAGEMENT POINT>.domain.com DNSSUFFIX=Domain.com FSP=<FSP.domain.com>  CCMLOGMAXSIZE=10240 CCMENABLELOGGING=true SMSCACHESIZE=10240

5.) Recommended: Depending on your setup, you may wish to add the following registry entry for preferred MPs. Select at least one MP on the same subnet as your machine:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CCM\AllowedMPs
REG_Multi_SZ
MP.domain.com (eg)
This should stop much of the noise you’ll get trying to contact MPs you’re not interested in. This might be the case if you have cross forest relationships, etc. This step can safely be ignored if your configuration is reasonably simple.
6.)  Leave for 10 mins or so and then find machine in ConfigMgr. Right-click and select Approve.
7.) Open the CM Client applet and if you don’t already have all the tabs you’d expect usually ~7) then click Machine Policy Refresh under the Actions tab and Find Site under the Site Tab. Give it a few more minutes and the tabs should all appear.
8.) Finally, Add the computer to the appropriate patch collection and wait for patches to download. Ensure there are no further patches to run after reboot.
9.) Remember to remove the registry entries from earlier before handing back the machine!
 
NOTE
1.) In my case, because the template machine hasn’t been deployed, it doesn’t have a licence. If you select to run the software update scan cycle you’ll see a popup like that below. You can safely ignore this, it will still install the updates. You will also see this popup from time to time just before they start installing. Just ignore it.
error

 “You may be a victim of software counterfeiting.”

2.) I noticed all the machines had the same name. Because they’re in a workgroup, this is acceptable to CM and had no noticeable detrimental impact on deployment.

Software Updates: error 0xc80003f3 or error 0x8007000e in ScanAgent.log or WUAHandler.log

PROBLEM:
You may also see the following message in the ScanAgent.log:
Sources are current, but Invalid. TTL is also invalid.
SCANAGENT.log
Invalid
WUAHANDLER.log
0xc80003f3

1. You’ve tried reinstalling the client.

2. You’ve installed Windows6.1-KB3050265-x64
3. Maybe even https://support.microsoft.com/en-gb/kb/2887535
Errors just won’t disappear…well help is at hand.
These errors translate to the following:
0x8007000E -2147024882 E_OUTOFMEMORY
0x8007000E -2147024882 [DCOM belly up] CAgent::CreateJob failed Not enough storage is available to complete this operation.
and
0xC80003F3 -939523085 hrOutOfMemory The computer is out of memory
However, you look at the available memory and can clearly see there is plenty to be playing with.
 
SOLUTION:
 
Turns out you need to up the pagefile on the affected box – this shouldn’t even require a reboot and immediately the errors will disappear from the logs and lovely software updates will all appear in Software Center.
One slight oddity I did notice was that while the software updates appeared, they wouldn’t start downloading/installing. I waited quite while and they just seemed to sit there. I restarted the SMSAgent host and this cleared them but wouldn’t pick them up again despite manually instigating the Scan or Software Updates Deployment actions from the SMS agent applet. I needed to restart the box, and then run these actions again manually after that and all started to work as expected and start downloading/installing.
Simon

Software Updates Failure – 0x800F0902

0x800F0902

Symptom

You try to deploy software updates to a system and no updates will install. The system picks up the updates, tries to install but ultimately fails after hanging for ages. Eventually you are presented with the error above.

Resolution

BITS is most likely not running. It is unlikely it isn’t installed as you would expect it to be installed if the ConfigMgr agent is installed but it may not be running. Start the service and try again.

 

Phantom reboots following new software update installation

A word of caution….

Situation:

All servers in our a newly-formed environment required a first set of ‘baseline’ patches to bring them up to speed, after which we would set up a monthly patch update schedule.

Following this ‘inaugural’  deployment of around 380 patches. everything appeared on the face of things, to go smoothly. Machines all received updates and rebooted as expected. I could see from reports however that one or two patches had failed to install for some reason in this deployment. I figured that we could schedule in another time to try these again, maybe even the following month as none were actually deemed critical. In any case this was another conversation I needed to have with the application owners.

Two days later I got a call saying that a sizeable percentage of the server collection I had patched ‘randomly’ rebooted again at various times that morning. A significant amount of arm waving ensued and I was asked to look into it, especially as it was clear from the event logs that the reboot had been instigated by the CCMExec:

The process C:\Windows\CCM\CcmExec.exe (COMPNAMEXXX) has initiated the restart of computer COMPNAMEXXX on behalf of user NT AUTHORITY\SYSTEM for the following reason: No title for this reason could be found
Reason Code: 0x80020001
Shutdown Type: restart
Comment: Your computer will restart at 08/14/2015 09:58:37 AM to complete the installation of applications and software updates

I frantically checked all deployments and confirmed there was absolutely no deployments due and everything had basically gone out as expected. In fact nobody had even touched the console that morning.  Further digging revealed the following in the logs of affected machines:

Log Name: System
Source: Microsoft-Windows-WindowsUpdateClient
Date: 12/08/2015 17:24:00
Event ID: 20
Task Category: Windows Update Agent
Level: Error
Keywords: Failure,Installation
User: SYSTEM
Computer: COMPNAMEXXX.domain.local
Description:
Installation Failure: Windows failed to install the following update with error 0x80092004: Update for Windows Server 2008 R2 x64 Edition (KB3045645).

Then, later on:

Log Name: System
Source: Microsoft-Windows-WindowsUpdateClient
Date: 14/08/2015 10:02:43
Event ID: 19
Task Category: Windows Update Agent
Level: Information
Keywords: Success,Installation
User: SYSTEM
Computer: COMPNAMEXXX.domain.local
Description:
Installation Successful: Windows successfully installed the following update: Update for Windows Server 2008 R2 x64 Edition (KB3045645)

Similar entries appeared for several of the failed updates. Clearly what had happened is that another software update scan cycle had taken place and detected that the machine was missing several important updates, specifically those that had failed previously. The updates were then automatically re-tried and successfully installed the second time round. Presumably they needed a reboot before they would install correctly although I have read plenty to suggest this shouldn’t happen. Regardless, the second successful installation instigated our phantom reboots.

My recommendation here would be to ensure you have a dedicated Maintenance Window to alleviate this headache. There were one or two that picked these updates up and installed them after my discovery and the Maintenance Window prevented the reboot from occurring. Obviously it won’t be properly patched until the client has been restarted but that can at least be left to the application owner to decide on a suitable time to do this.

ADR Gotcha – ADR only fires from the site server

Very short and (not so) sweet post with a word of warning on ADRs.

CM2012 SP1 R2

I wanted to set up and Automatic Deployment Rule to select a bunch of updates and automatically download and set up my Software Update Group. In my client’s environment the only box with internet access belongs to the SUP. Unfortunately, if you want your ADR to carry out the tasks above then you will need to ensure your Primary Site Server has access to the internet as well.

An ADR will always fire from the Primary Site Server and without internet access, the ADR will fail to download any updates.

You’ve been warned!