Microsoft has released an out of band update to fix the memory leak issues with the lsass process on Domain Controllers after applying the March 2024 cumulative update. Microsoft strongly recommends that the latest servicing stack updates are installed before applying the out of band patch.
This out of band patch is a cumulative update and can be installed in place of the normal March cumulative update that is published on Windows Update / local WSUS server. If you want this patch in WSUS you will have to side load it yourself.
We have confirmed in our testing environment that the lsass memory process was starting to creep up. We removed the March cumulative update and applied the out of band patch in our test environment and so far all seems good.
Microsoft has identified an issue that affects Windows Server domain controllers (DCs), and has expedited a resolution that can be applied to affected devices. Out-of-band (OOB) updates have been released for some versions of Windows today, March 22, 2024, to addresses this issue related to a memory leak in the Local Security Authority Subsystem Service (LSASS). This occurs when on-premises and cloud-based Active Directory domain controllers service Kerberos authentication requests.
This issue is not expected to impact Home users, as it is only observed in some versions of Windows Server. Domain controllers are not commonly used in personal and home devices.
Updates are available on the Microsoft Update Catalog only. These are cumulative updates, so you do not need to apply any previous update before installing them, and they supersede all previous updates for affected versions. If your organization uses the affected server platforms as DCs and you haven’t deployed the March 2024 security updated yet, we recommend you apply this OOB update instead. For more information and instructions on how to install this update on your device, consult the below resources for your version of Windows:
This might be an unpopular take but I very much dislike the way Microsoft uses its Authenticator app for personal accounts. Seriously, it sucks.
If you have a personal account with Microsoft and turn off passwordless authentication in your account but enable two factor with the Microsoft Authenticator, it turns on passwordless authentication in the app for you and you can’t disable it. (Queue up the MFA fatigue).
The way to fix this is to setup another authenticator that’s not the Microsoft one. Save your sanity and stop being MFA bombed by threat actors trying to break into your accounts.
This blog is not going to go through the details get the initial ADFS setup. I’m using ADFS on Server 2019 for this demo though. Bring up a Windows Server 2019 and install the ADFS role. This will require a valid SSL certificate. I did not bring up a web proxy as this ADFS server’s only function is to serve authentication for vCenter so we can put DUO behind ADFS.
Add ADFS Certificate(s) Chain to vCenter:
Add the Root & Intermediate CA certificates to vcenter. This can be done under Administration in vCenter at the bottom under Certificates click “Certificate Management”.
Then next to Trusted Root Certificates, click Add and Add Root and/or Intermediate CA certificates here. I added just regular .CRT files that you could open with notepad and show the certificate text.
While here in vCenter Administration, navigate to Single Sign On -> Configuration
Over on the right hand side click on the “i” next to “Change Identity Provider”
Make note of these Redirect URIs you will need them when you configure ADFS.
Configure ADFS for vCenter:
You should now have a brand new install of ADFS with the most basic things configured to make it happy including your SSL certificate. Launch the ADFS Management tool and right click “Application Groups” and click “Add Application Group”.
In the wizard that appears, give your Application a Name and Description that means something to you and select “Server application accessing a web API” and click “Next”.
Make note of the Client Identifier. This will be needed later in vCenter configuration. Also input the two redirect URLs you noted earlier in vCenter. Click “Next”.
Check the box to Generate a shared secret and copy this down as it will be needed later in vCenter configuration. Click “Next”.
Input the Client Identifier from earlier in this wizard in the Identifier box and click “Add” and then Click “Next”.
On the next screen for now click “Permit everyone” we can revisit this later after everything is setup. Click “Next”.
On the next screen make sure “openid” and “allatclaims” are both checked, you will probably have to check “allatclaims”. Click “Next”.
Review and make sure everything looks good. Your Server Application Identifier, and Web API Identifiers should be the same GUID. Click “Next”, and “Close”.
We now need to configure the claims before we can attempt to configure vCenter and attempt a login.
Right-Click the newly created Application Group and select “Properties”
Select the item under Web API and click “Edit”
Click on Issuance Transform Rules. Here we will create three rules. Click Add Rule three times and make your rules look like the following: (Note: All Rules will use the default “Send LDAP Attributes as Claims”)
Group Rule:
LDAP Attribute: Token-Groups Qualified by Long Domain Name
Outgoing Claim Type: Group
Subject Rule:
LDAP Attribute: User-Principal-Name
Outgoing Claim Type: Name ID
UPN Rule:
LDAP Attribute: User-Principal-Name
Outgoing Claim Type: UPN
Click “OK” to close the dialog box after all three rules have been configured. Click “OK” again to close the Authentication Properties box.
Before we switch over to vCenter configuration we need to snag one more thing from our ADFS server, it is our OpenID Address. To obtain this launch a PowerShell window as Administrator on your ADFS server and type in the following:
You should receive something like the following (save this URL you will need it for vCenter configuration):
vCenter ADFS Configuration:
We are now ready to setup the Identity Provider in vCenter. Navigate back to Administration -> Single Sign On -> Configuration.
This time click the “Change Identity Provider” link instead of the “i” next to it.
Click “Microsoft ADFS” and click “Next”.
On the next screen, you want to copy the Client Identifier and Shared Secret from ADFS that you set aside during setup. The OpenID address also that we just snagged from ADFS.
On the next page fill out the base DN for Users and Groups, I’m using the root because this is just a lab setup. I’m using a service account (only requires read access to AD), and I’m only using ldap:// again because this is a home lab. If you are setting this up in Production make sure you are using ldaps://.
Click “Next”, review and click “Finish”.
You should now be able to add an Admin group in vCenter from your Active Directory source. At this point you should be able to logout and log back in to test out authentication to ADFS.
———–YOU SHOULD NOW HAVE vCENTER & ADFS SETUP————
Proceed forward for the DUO configuration and setup.
Login to your DUO Administration Portal and Click “Applications” and then “Protect and Application” and then locate the entry for “Microsoft ADFS” and click “Protect”
Take note of your “Client ID”, “Client Secret” and “API Hostname”. You will need this information to complete setup on your ADFS server.
If you have an AD FS farm install Duo on the Primary Server first, if you have a SQL farm you may begin with any node. Since this is just a lab I only have the one server.
Launch the Installer. Click “Next”
Enter the Information from the DUO Admin page, or what your Information Security Team will need to provide to you.
Bypass Duo authentication when offline (unchecked) will make sure it “fails closed” and nobody will be able to access resources if Duo is down. This can be changed later in the registry.
By default DUO sends the sAMAccountName, if you want UPN sent, check the box that says “Use UPN username format”.
Click “Next” and Click “Install”
You may then be prompted by DUO to enable the DUO MFA Authentication.
To do this in ADFS Management Open up Service -> Right-Click “Authentication Methods” and click “Edit Multi-factor Authentication Methods”.
Check the box for Duo and click “OK”
At this point you should still be able to login to ADFS and MFA will NOT be working.
Configure ADFS for MFA:
Go back to your Application Groups, and Right-Click your Application Group and click “Properties”
Click Web API and click “Edit”
Go to Access Control Policy and click “Permit Everyone and require MFA”
After messing with using .htaccess files to lock down my blog I decided to try Cloudflare’s free offering for a simple WAF to protect my blog as well as get some more detail as far as who is talking to it.
I’m still working on configuring Cloudflare and I may have more to say after I get more time to get used to the interface and make some more tweaks.
One of the most important things I wanted to do was be able to lockdown /wp-login.php, /wp-admin, and /xmlrpc.php like I was doing with .htaccess.
I was able to get the same lock down functionality done relatively quickly, using the WAF security rules.
Over on the left-hand side if you navigate to Security -> WAF
I used their template for Zone lockdown, and blocked access to those things from everyone except my IP, which can obviously be edited later if it ever changes or I’m out traveling and want to update my blog.
Of course another lesson learned is you can’t use your domain anymore to SSH to your server. Cloudflare will not allow it. So I created a SSH sub-domain on my domainname in Cloudflare and set it to DNS only.
You might say this defeats the whole purpose of cloudflare because now somebody knows your IP address, and to a certain extent that is true. However, I have apache configured so if someone tries to browse by IP address they get a 403 from .htaccess in a different web directory.
I attended a talk last night regarding web application security and the speaker was presenting about OWASP JuiceShop which I had played with very little in the past. I have since started to bring this application back up to play with locally.
I will be writing up tutorials as I go through them.
To “install” OWASP JuiceShop on Kali Linux using docker do the following:
Install Docker: sudo apt install docker.io
Run: sudo docker pull bkimminich/juice-shop
Run: sudo docker run -d -p 3000:3000 bkimminich/juice-shop
—
Quick Docker Commands To View/Stop Containers:
To view what docker containers are running: sudo docker container ls
To stop a docker container: sudo docker stop 5698ef8694e5 (Using the ContainerID from above, use your own Container ID here)
Performing another sudo docker container ls shows that there are now no containers running.
If you have an application running on your Windows Server OS that is not using IIS as a front-end, it may be relying on the built-in Certificate Binding in Windows.
In order to check the certificate bindings you can run:
netsh http show sslcert
If you need to update one of the certificates listed here you can run:
Note: ipport, appid will be given to you with the first command to look at the certs, use the values of the current cert that is/was working. Make sure the certhash is the new certificate you want to be used here.
One of my major tasks when I started my new job was to automate our Windows Server patching so I wouldn’t have to be up two nights every month to deal with patching.
All of our Windows Servers have SCCM clients on them, and we manage the patching software push and the maintenance windows for the servers to reboot with MECM. These servers are members of AD groups that are used to correlate to Device Collection in MECM. This is important because these same AD groups are used for custom scripts below.
This blog is not going to go over approving and downloading patches, setting maintenance windows, or deployment settings.
Things Done Before Automated Patch Run:
We set the Installation Deadline of patches to 2 hours before the Maintenace window (reboot).
We run a custom script on all boxes through SCCM before patching to clear the CCM Cache folder. This helps prevent running out of storage space in Windows. Our OS disks are not very big.
Set the Maintenance Schedule in Operations Manager (SCOM), so the boxes will not report issues during the patch window.
Set the custom scripts on our scripting server to force patch installs and check services.
Set time aside to manually patch the boxes that cannot be automated (i.e. Domain Controllers, etc.)
Custom Scripts For Patching:
These two scripts are pinned in Task Scheduler and uses and an AD Service account that has local admin on the servers. (This is not meant for Domain Controllers, DA would be needed).
A template for the script we use to force servers to install patches is located here:
This script will basically make sure that the servers start installing patches at the time we have it run in Task Scheduler from our scripting server (Typically a few minutes after the 2 hours prior that we schedule). This requires WinRM ports open from the scripting server to the server endpoints that are being updated.
A template for the script we use to monitor the patch deployment, force MECM check-in and auto start services is here:
This script we schedule to run about 20 minutes after the maintenance window starts and have it continue to run every 20 minutes until the maintenance window completes. This is a LOT of e-mails and I have not edited the script yet to just write to a log file but that would be easy enough to do. This script will report back last boot, and any attempts to either force SCCM check-in, or restart any services that failed to start.
By default if the box restarted over 25 minutes ago but not over 4 hours ago it will force an MECM check-in so the patch compliance report the next morning is accurate.
By default if the box restarted over 25 minutes ago but not over 4 hours ago, any Automatic services that are not started will be attempted to be restarted. Services can be whitelisted, so the script does not act on them.
There are many security risks with running Active Directory. In the Year 2022, one of these is still running your Active Directory with unsigned/simple ldap binds.
If you don’t already have a PKI environment setup you should probably work to set one up so you can get certificates on your Domain Controllers that are trusted by your businesses devices so unsigned/simple binds can be retired.
A script I’ve found very helpful for monitoring the Domain Controller firewall logs for these events is located here:
As I’m migrating Domain Controllers over to Server Core one of the major issues I’ve run into is managing the Windows Firewall Rules. On a GUI version of Windows Server it is very easy to see what firewall rules are applied, in core… not so much, especially if you are pushing them with GPO (Group Policy).
All of the PowerShell cmdlets and netsh advfirewall commands all seem to return the local firewall rules and not any of the Group Policy pushed firewall rules. Moreover I could not find an easy way to see what the current firewall rules are that are applied via GPO.
As I have disabled all of the built-in firewall rules as to lock down the Domain Controller Firewall Rules as tightly as possible, even with RPC open and the dynamic RPC range locked to specific ports but open the Windows Firewall MMC would not open. I was seeing no blocked traffic in the Windows Firewall Logs.
I received the following error message:
“There was an error opening the Windows Firewall with Advanced Security snap-in”
“The specified computer could not be remotely managed. Ensure that you are not trying to connect to a remote computer with an earlier version of Windows…..”
My solution to this problem was to enable the built-in Firewall Rules for Remote Firewall Management so you can use MMC console:
Windows Firewall Remote Management (RPC) Windows Firewall Remote Management (RPC-EPMAP)
These firewall rules seem to have some special magic to them that I haven’t put my finger on yet that allow the Remote MMC Firewall snap-in to work. You can of course lock these rules down to remote IPs as well.
