Simple powershell script

I have a script right now that looks for all files certain day old and certain file extension and it deletes all of the files. This works fine and it counts fine

Then I have to delete all folders that correspond to being empty and that includes all sub folders too. I also have to output this into a file and display each file deleted. The output would show 30 folders deleted but actually 48 were really deleted.

Now my question is i am trying to do a count of all the folders deleted. I have this script but it just counts the deepest folders not all the ones deleted. Here is the part of the script i can not get to count

$TargetFolder = "C:\Users\user\Desktop\temp"
$LogFile = "C:\Summary.txt"
$Count = 0

Date | Out-File -filepath $LogFile

get-childitem $TargetFolder -recurse -force | Where-Object {$_.psIsContainer}| sort fullName -des |
Where-Object {!(get-childitem $_.fullName -force)} | ForEach-Object{$Count++; $_.fullName} | remove-item -whatif | Out-File -filepath $LogFile -append

$Count = "Total Folders = " + $Count
$Count | Out-File -filepath $LogFile -append