Home firewall recommendations

You need to secure it with VPN for secure access. That is what I do for my home office systems. So you need a VPN router to do this. More than 40 quid when you are done.

1) Avoid having any public facing server lack firewall protections if you don't have to do this. (I know this is exactly what you're looking to do, but putting in a reminder)
2) Even with a firewall, people are still going to try the exact same things. However, without a firewall, the attack surface area is much larger.
3) Make sure to disable unneeded bits, and to make passwords complex.
4) If you're trying to keep it low budget, you could try Untangle's free version. You would need a computer that has at least 2 NIC cards (no need for an OS as Untangle will address that). You'd get NG Firewall.

Creating a VPN would be a far more secure approach than what you have now as John has mentioned. However, you still should get an actual firewall into play first. Untangle also includes OpenVPN. Going the VPN route would prevent you from having to open firewall ports beyond what is necessary for VPN.

TZ-210 is a good firewall, maybe TZ-215 is better with similar price
You don;t need to buy any licenses to use firewall if you don't want to use additional features like virus and spam screening

I second Masnrock suggestion of using Untangle for use as Firewall and VPN option which also included in Untangle. You can check the intrusion prevention module in Untangle for added security, just make sure the machine you're using as the appliance has more than 2GB of memory.

you can try sophos UTM home edition too.
License is free for home use (and limited to 50 IP's)
nearly all UTM features are available.
You can use RAS-VPN and restrict RDP access to your own IP.

Please post you Windows Firewall settings.

You can also use RDPGuard
https://rdpguard.com/

Going to try this on an old PC

Just would like to add that Untangle has a home license that contains all of the paid applications. You can pay $5 a month or $50 for the year.

There are two reason I didn't recommend untangle maybe three
1.Home user has to pay for it, if for business then it a right off and that's ok
2.Untangle has become a resource hog, and you must use better to best hardware to harness its power so having your lowly computer hanging around with sub-par hardware will not cut it.
3.I'm pissed because they used an opensource platform, let us do the beta testing in the early phase, then turn around and sell us free software

I understand your pain, but you have the option to use it for free, but with limits. Depending on the environment, usage and connected devices the free solution can work. Concerning the hog on resources, you can configure it to suit your hardware and use what's only needed. I heard that the current version contains a mix of open source and commercial technology. For a product or service to become better, money is needed, free things also come with a cost often times not apparent.