I have designed the skeleton code for a custom SharePoint 2010 service application.
The service follows strictly Microsoft's Service Application Framework infrastructure and was built after two service application examples available online as tutorials.
The service itself deploys globally no problem and I have verified already, on my own and with the help of Microsoft support, that the feature is indeed in the assembly and hive folders of both servers. The environment is a two-member farm, with server 1 being the central administration host, as well as where the config database currently resides. Server 2 has an identical configuration (except for SQL Server) however a lot of the Microsoft OOB services are off as I intend for this server to be specifically an application server for my custom services.
From Visual Studio 2010, the solution deploys fine and I am able to activate it successfully via the "Manage Farm Features" under System Settings in the Central Administration portal. Moreover, I can create the service application and its corresponding proxy under Application Management.
The issue is that the service will only show up on one of the servers (server 1 in this case, where Visual Studio runs) under "Manage Services on Server", but not the other, which means that, despite having activated the application, created the service application and proxy, I am unable to provision it on my application server (server 2).
I have troubleshot this issue to no end and Microsoft is currently trying to figure out what the problem is, however I thought I would ask about this here to see if anybody had any experience with custom service applications and farm-wide provisioning.
Of note, built-in Microsoft SharePoint services like "PerformancePoint Service" show up on both servers and I can provision them at the same time, or selectively on one server only, however this is not happening for my custom service application.
I look forward to reading your responses, do let me know if you require additional information.
Thanks in advance!