Domain Admin Group Level Access

I would first begin by reviewing what permissions are required to perform the functions needed by your scripts.
To enumerate objects in AD, one (normally) merely needs to be a member of Domain Users and certainly not Domain Admins.  The service account may need to be granted the "Run As A Service" permission on the server it runs on.  You could create a group, Domain Service Accounts, and create a group policy to allow this group to run as a service on all servers, but this opens up (probably) far more access than is needed.
Also, ensure you document everything you find and everything you do so that, should you need to change something 6 months from now, you don't have to go on the great trek of discovery again.

My question. the four people only need access to this group for access to servers  and network shares (I can get rid of this).

This leaves my service accounts, whats the best way to go about removing these accounts from domain admin group while still allowing them permission to run the actions they run?

You can configure global admins
https://www.experts-exchange.com/articles/29596/Securing-Active-Directory-Administrators-Groups.html

I have never found a service account that cannot function without DA rights, even when the vendor claims so.

Hi,

First I would point you to read the MS Best Practise documentation here: http://aka.ms/bpsad.

Microsoft have realised, after years of people logging into laptops as domain admins, to surf, read phishing emails and "help" remoting onto other laptops that none of those behaviours are good. Worse, it's become habit so many IT staff do not know any better, partly because MS haven't explained.

I've made a point of reading up and educating myself and found the following:

1) Domain Admins (and Enterprise Admins and a few others) are solely meant for occasional or rare use, in emergencies. They are "break glass accounts": i.e. only for using for emergency situations, for a limited time.
2) The above accounts are privileged accounts. Privileged accounts are not normal accounts and behave differently and the OS protects them with specific engineering.
3) Domain Admins are meant for one thing: making changes to domain controllers.
4) Domain Admins are meant to only logon from a secure admin server or workstation to make those changes to the DC. Ideally, never logon directly to your DC.

There's a long PDF from MS here:
https://www.microsoft.com/en-us/download/confirmation.aspx?id=38815

From MS
DAs are all-powerful within their domains, while EAs have forest-wide privilege. In a properly designed and implemented delegation model, DA membership should be required only in "break glass" scenarios, which are situations in which an account with high levels of privilege on every computer in the domain is needed, or when certain domain wide changes must be made.

https://docs.microsoft.com/en-gb/windows-server/identity/ad-ds/plan/security-best-practices/appendix-b--privileged-accounts-and-groups-in-active-directory

https://activedirectorypro.com/active-directory-security-best-practices/
https://blogs.msmvps.com/richardsiddaway/2016/09/14/how-many-domain-admins-do-you-need/ : (Richard Siddaway is a PowerShell author & MVP but writes on security too)
https://www.csoonline.com/article/2627737/authentication/how-many-enterprise-admins-is-too-many-.html?page=2 (second paragraph) (Roger Grimes is a leading security consultant).

Your Environment
we have four actual people in the domain admin security group,
Remove 3, leaving the most trusted person in

the administrator account
Fine :)

handful of service accounts that primarily read AD...
Remove them ALL

For pure authentication you only need AD read permissions. Any standard user account will do. However, if you are using 2012R2 then I highly recommend using Group Managed Service Accounts. They provide a "set it and forget it" way to create an account with a password that the OS manages.
In the event one of the apps using a service does not work, go to the vendor's website and check. My favourite bogey man is BMC agents that get service accounts with passwords that never expire and worse, get Domain Admin. If you check BMC, it only needs a few specific permissions. Quite often services do NOT even need logon permissions.

Q: My question. the four people only need access to this group for access to servers  and network shares (I can get rid of this).
A: No, they don't. Really. They need a logon account and maybe a few other limited permissions. You don't need Domain Admin to RDP to servers. You don't need Domain Admin to use SCCM or indeed BMC Patrol either. The most I would go to is local admin, but even that I would limit if possible.

I will end with another link by Orin Thomas;
https://channel9.msdn.com/Events/Ignite/Microsoft-Ignite-New-Zealand-2015/M321
It's old but good. The very first point covers service accounts. The first 10 points are most relevant to everything in your question, but if you have the time watch them all. It is quite entertaining.

Finally, with regards policies, it is worth doing two things:

1) set harder passwords for the privileged accounts - 25 minimum length, stronger lockout than normal. Note this is a separate password policy to every other user account.
2) Deny logon to workstations and standard servers to Domain Admins!

The two things will block people from doing things and stop pass-the-hash dead and make your business safer.
I do realise this is both alien and potentially time consuming but the IT world is getting more threats as time goes by. Doing the above will help attackers choose someone else!

Mike