Exchange Server to SQL Server/Access VBA Connection

We are currently trying to create a contact list from our Exchange Server.  I have been researching methods of connecting to an Exchange Server.  The only way that I have found that I can gather contacts from each "user mailbox" is create a loop that will open each box, and download contacts.  Much like you do an Access by creating a link to an Exchange Server, but Access will only allow me to connect to my personal mailbox.

We currently have an SQL Server, and Exchange server, both on different machines.  The SQL Server does not have Outlook, or any Exchange drivers loaded.  I have developed linked servers, but they cannot connect without the Exchange drivers, or so I have read.  

I would appreciate any thoughts on what I need to do to get the SQL Server talking to the Exchange Server.  Also if there is a more efficient method of extracting, and updating contacts.  

Thank you