Num Lock gets disabled in Terminal session

We have a strange case with a customers Azure server  (Windows 2012 R2). It is setup as a Terminal server for a customer. They log in from their PC's wich are Windows 7. Num Lock works for a while if they use programs like Word, but sometimes using Outlook or Access program, it stops working. And when it happens pressing Num Lock does nothing, it is still disabled. If they log out and then back into Terminal Server - it works until Customer have used Access or Outlook. It varies when it happens, but it always happens in some cases. The problem are likely not related to these program, its just those Customer use when it happens.

We checked online for similar problems. We found the article below about activating keyboard as a mouse. But i cant see that it is activated when the Num Lock problem occurs.
https://mcmw.abilitynet.org.uk/using-your-keyboard-to-control-the-mouse-windows-8-1/

Also found this case - but that is not the exact same scenario.
https://social.technet.microsoft.com/Forums/windows/en-US/c002126a-0f50-49e0-977c-d66ec051c880/num-lockcap-lock-are-working-in-reverse-on-terminal-server-2012-r2?forum=winserver8gen

I did some testing and found out this:  

If i log on to Terminal Server from my own PC, i don't get same problem. Even when using same functions and features in Outlook and Access. So this tells me there is nothing wrong with the server.

If i log remotely into the customers PC using Teamviewer, and from there connect to Terminal Server (in same manners as customers do), then i can't reproduce the error. So both these 2 tests point to some problem related to using the local keyboard at customer.

Any PC at customer (3 in total) all have the same problem with Numlock.

We have tried swapping the keyboard driver from Generic MS one to Spesific driver, but no change.

Any ideas on what is going on?  Anything else i can try to pinpoint the issue?