Exchange HAFNIUM+ Mitigations

On-Premises Exchange Compromises

We have a new one folks.

2021-04-15: Released: April 2021 Exchange Server Security Updates

It's important to note that our Exchange 2016 CU19 instance would not apply the security update. We had to update to CU20 before the KB5001779 would run.

You may get a prompt about services not being stopped, we had three. Be patient and wait for them to disappear from the Task Manager Processes list! They will. Then hit the RETRY button.

As always, make sure to run the update file from an elevated CMD!

TROUBLESHOOTING NOTE: Microsoft Docs: Exchange Issues: Upgrade Patch Cannot be Installed

HAFNIUM and Compromise Related

2021-04-15: Just an FYI for all y'all.

• Justice Department Announces Court-Authorized Effort to Disrupt Exploitation of Microsoft Exchange Server Vulnerabilities.
• Schneier on Security: The FBI Is Now Securing Networks Without Their Owners’ Permission

The FBI is logging in to compromised Exchange servers to "clean-up web shells" left by the various folks running the now crypto-malware deployment via them.

Recently (as of the original article date), Microsoft Threat Intelligence Center (MSTIC) has attributed previously unknown exploits targeting on-premises Exchange Servers to HAFNIIUM. 

From the MS blog CyberAttacks, it is being recognised as a sophisticated three step attack :
First, it would gain access to an Exchange Server either with stolen passwords or by using the previously undiscovered vulnerabilities to disguise itself as someone who should have access. 
Second, it would create what’s called a web shell to control the compromised server remotely. 
Third, it would use that remote access – run from the U.S.-based private servers – to steal data from an organization’s network 

There are steps to test and check your vulnerability. Most importantly, make sure you are applying all the security patches. Your first step, is to read this Article, and the various articles listed below. Then follow the suggestions presented below.

Backup Setup

The key to any disaster is knowing that the backup repository is okay. Is it? Has it been tested?

Immutability built-in to the backup setup is critical to having a place to go to if compromised and encrypted. Why? Because the folks running the ransomware processes get in, scope, compromise, and wait until their intelligence gathering is done before _deleting_ the backups and the backup server.

Make sure to set aside the last two weeks of February and the first two weeks of March backups as soon as possible. Get them air-gapped stat!

Verify Edge/Router

Check with the edge/router vendor's security suite to see if and when they updated their signatures to block any Exchange exploit related scans.

SUGGESTION 1: Geo-Limit incoming packets to your geographical region. At least until things have been cleared up.

SUGGESTION 2: Use a cloud based sanitation and continuity provider. With Exchange offline users would still have access to a mailbox. Plus, SMTP inbound would only be from the provider thus eliminating SMTP authorization attacks and most of the bad stuff would never make it inside the network.

Mitigations

If scans show that there were issues, then it's time to delve into the indicated logs.

Create a new Domain Admin account with Schema Admin and Enterprise Admin membership then disable all other domain admin accounts _after_ logging on with the new account.

Have all passwords for user accounts reset.

The PowerShell steps to scan AD for new objects below is key to figuring out if there are any rogue accounts set up after the fact.

Scan Exchange Server

NOTE 2021-03-15 at 1930Hrs MST: Microsoft has released a new one-click utility: Exchange On-premises Mitigation Tool (EOMT)

This utility replaces the steps below. It's just released so we've not had a chance to work with it yet!

Use the following tool in an elevated PowerShell to scan the Exchange Server:
https://github.com/microsoft/CSS-Exchange/blob/main/Security/Test-ProxyLogon.ps1
NOTE: Microsoft keeps changing the destination link. The repository is here with the PoSh called Test-ProxyLogon.PS1 as of 2020-03-12 at 0948HRs MST.
https://github.com/microsoft/CSS-Exchange/tree/main/Security

1. Copy the Test-ProxyLogon code into Notepad
2. Save As "Test-ProxyLogon.ps1" with the quotes in your C:\Temp folder
3. Run in Exchange Management Shell: .\Test-ProxyLogon.ps1 -OutPath C:\Temp

This is what a good result looks like:

NOTE: Make sure to back up the Exchange server _before_ making any changes!
That will allow for forensic examination at a later time.

What are the create dates on the .ASPX files that would have been in:

• C:\inetpub\wwwroot\aspnet_client\
• C:\inetpub\wwwroot\aspnet_client\system_web\

That would tell you when the initial breach happened. That also gives you the restore point.

Task Scheduler

Verify if there's any rogue tasks in Task Scheduler. Check the usual Registry Run locations for any possible rogue software persistence.

Active Directory Check

Here is some PowerShell to poll your AD for recently added accounts:

# Get user accounts in the Domain Admins Security Group
Get-ADGroupMember "domain admins" -recursive | Select -Expand Name

# Check New User Creation 
# C:\Temp needs to exist or change that
$ExportName = "C:\Temp\2021-03-08-UserADCreate.CSV"
Get-ADUser -Filter {Enabled -eq $True} -Property Created,LastLogonDate |
  Select-Object -Property Name,SAMAccountName,Created,LastLogonDate |
  export-csv "$($ExportName)"

# List all members of local admin group in AD
$ComputerName = Get-ADComputer -Filter * | Select-Object -ExpandProperty Name
foreach ($Computer in $ComputerName) {
  Invoke-Command -ComputerName $Computer -ScriptBlock{ Get-LocalGroupMember -Name 'Administrators' }
}

You can tweak the group names for the various local and domain groups to run the above reports. 

Kerberos Ticket

An important step is to reset the krbtgt  setup (HT: EE Expert Michael B. Smith).

Exchange Server

The Exchange server should be restored to a point in time before the compromise.

If there are partitions for each Exchange component, which is a best practice, then you can use the existing databases/logs partitions when the time comes to "overwrite".

Make sure HTTPS is not forwarded from the Internet to the Exchange server while running this process.

Make sure to run the following in an elevated CMD!

• Update the restored Exchange server to the latest Cumulative Update
• Run the Exchange Security Update

Forklift Restore

The process we’ve done many times in the past and would be doing if we had to back date restore keeping in mind that our Exchange partition layout is as follows:

• C: OS
• D: Exchange Program Files
• E: Exchange Dbs
• F: Exchange Logs

 
We would: 

1. Close HTTPS and SMTP to Exchange
2. Restore C: and D: to a date prior to the compromise.
3. Forklift Recovery (Allow overwrite on Dbs)
4. Update CU
5. Update Security
6. Verify all services
7. Open SMTP inbound
8. Test mail in and out
9. Open HTTPS

Use the following to perform a "Forklift Restore" of the up to date databases: 

# Exchange Forklift Restore 
# Enable overwrite 
Set-MailboxDatabase "Mailbox Database YOURS" –AllowFileRestore $true 
# Dismount Database 
Dismount-Database "Mailbox Database YOURS" -Confirm:$False 
# Copy Files (Restore most recent)

# Mount the database 
Mount-Database "Mailbox Database YOURS" 
# Disable overwrite 
Set-MailboxDatabase "Mailbox Database YOURS" –AllowFileRestore $False

An Exchange Dial Tone Recovery is an option to get mail flowing after a restore if the mailbox databases are large and will take some time to bring online.

The following comes via EE Expert Michael B. Smith:

• Remove the old HealthMailboxes and System Mailboxes
  • They are tied to user accounts that could be compromised
  • HOW: You remove the health mailboxes and reboot. They will be re-created.
  • HOW: Recreate the Arbitration Mailboxes (Microsoft)

Domain Trust Renewal

If a step-back restore is required, there's a possibility trust with the domain may break due to machine password difference.

# Option 1 
Test-ComputerSecureChannel # false = #! Broken 
# TODO Fix it! 
$cred = Get-Credential 
Test-ComputerSecureChannel -Credential $cred -Repair # True = #? Fixed Test-ComputerSecureChannel # True 

# Option 2
nltest /sc_reset
netdom resetpwd 

# Option 3 # Sign in as local admin on Win10/Server 
$Domain = "DOMAIN.Com" 
$DomainAdmin = "MyAdmin" 
Reset-ComputerMachinePassword -Credential "$($Domain)\$($DomainAdmin)"

Use the above to reset the machine password and restore trust with the domain.

Various Articles

The principle Microsoft article is here:

• HAFNIUM Targeting Exchange Servers
• The Microsoft Update Catalog with the security updates
• TEMPORARY: Mitigations until Exchange can be updated and patched
• UTILITY: Microsoft Safety Scanner

Some additional articles:

• CISA: Updates on Microsoft Exchange Server Vulnerabilities
• CISA: Alert (AA21-062A): Mitigate Microsoft Exchange Server Vulnerabilities
• FireEye: Detection and Response to Exploitation of Microsoft Exchange Zero-Day Vulnerabilities
• Palo Alto Networks: Analyzing Attacks Against Microsoft Exchange Server with China Chopper Webshells
• Brian Krebs: A Basic Timeline of the Exchange Mass-Hack 
• PRETORIAN: Reproducing the Microsoft Exchange Proxylogon Exploit Chain
  • Bold because this is an _excellent_ article

Video

Philip Elder discusses this article with EE Community Manager, Rob Jurd. They discuss how small businesses can mitigate risk during the MS Exchange vulnerability.
https://www.experts-exchange.com/videos/78932/EE-and-Philip-Elder-Discuss-MS-Exchange-HAFNIUM-Mitigations.html 

Conclusion

As new resources come to my attention, they would be added here.
Please message me if you have something that should be added!

Happy Hunting!

UPDATED: 2021-04-15 @ 0747Hrs MST