One of my major tasks when I started my new job was to automate our Windows Server patching so I wouldn’t have to be up two nights every month to deal with patching.
All of our Windows Servers have SCCM clients on them, and we manage the patching software push and the maintenance windows for the servers to reboot with MECM. These servers are members of AD groups that are used to correlate to Device Collection in MECM. This is important because these same AD groups are used for custom scripts below.
This blog is not going to go over approving and downloading patches, setting maintenance windows, or deployment settings.
Things Done Before Automated Patch Run:
- We set the Installation Deadline of patches to 2 hours before the Maintenace window (reboot).
- We run a custom script on all boxes through SCCM before patching to clear the CCM Cache folder. This helps prevent running out of storage space in Windows. Our OS disks are not very big.
- Set the Maintenance Schedule in Operations Manager (SCOM), so the boxes will not report issues during the patch window.
- Set the custom scripts on our scripting server to force patch installs and check services.
- Set time aside to manually patch the boxes that cannot be automated (i.e. Domain Controllers, etc.)
Custom Scripts For Patching:
These two scripts are pinned in Task Scheduler and uses and an AD Service account that has local admin on the servers. (This is not meant for Domain Controllers, DA would be needed).
A template for the script we use to force servers to install patches is located here:
This script will basically make sure that the servers start installing patches at the time we have it run in Task Scheduler from our scripting server (Typically a few minutes after the 2 hours prior that we schedule). This requires WinRM ports open from the scripting server to the server endpoints that are being updated.
A template for the script we use to monitor the patch deployment, force MECM check-in and auto start services is here:
This script we schedule to run about 20 minutes after the maintenance window starts and have it continue to run every 20 minutes until the maintenance window completes. This is a LOT of e-mails and I have not edited the script yet to just write to a log file but that would be easy enough to do. This script will report back last boot, and any attempts to either force SCCM check-in, or restart any services that failed to start.
By default if the box restarted over 25 minutes ago but not over 4 hours ago it will force an MECM check-in so the patch compliance report the next morning is accurate.
By default if the box restarted over 25 minutes ago but not over 4 hours ago, any Automatic services that are not started will be attempted to be restarted. Services can be whitelisted, so the script does not act on them.
Screenshot of Last Boot Report:
Screenshot of Services Not Started Report: