User GPOs not working on RDP server

I have an issue with an remote desktop services server. The issue i'm having is with the screen timeout.

We want the user's local computer to have a timeout of 5 minutes for the screen lock. But we want the server they are remoting into to not have a screen lock.  Currenty, the remote desktop session locks after 5 minutes of inactivity. So when the user goes pee they have login to their local PC as well as re-type their password in remote desktop.

Is this because I'm putting the server (computer object)  in an OU and the screen lock settings are user policies so the policy is being passed to the remote desktop session from the local PC because the user account is already logged in on the local PC?

There is a GPO at the domain level that set the screen lock for 5 minutes for all users.
These settings are in "User config > Policies > Admin templates > Control Panel/Personalization"
Enable Screen saver = Enabled
Password protect screen saver = Enabled
Screen saver timeout = Enableed & 300 seconds

I've setup an OU that has a GPO screenlock policy with disabled screen lock and placed the server in it and have disable GPO inheritance on that OU.
Here are the settings for this GPO
 "User config > Policies > Admin templates > Control Panel/Personalization"
Enable Screen saver = Disabled
Password protect screen saver = Disabled
Screen saver timeout = Disabled