Okay, here's an overview of what we want to accomplish:
Application Server program running on machine A.
Web (HTTP) server running on machine B.
Client applet running on machine C, on page served by web server at B.
Client on C uses RMI to talk to app server on A, without any change to the security policies on machine C.
I know this sounds impossible, because connecting to a machine other than the web server machine is a security violation. Here's how we get around it:
We have a special client socket factory that takes the request to connect to machine A, port X, and instead opens a socket to a special port on B where we have another program listening. It can ask that program, the "link", to connect to A:X. The "link" can do so, then just hook the socket streams together so that all I/O for the socket gets re-routed.
This works, believe it or not. The socket transparently communicates through this link program, with no special knowledge at either the client or the server end. But alternating calls from the client result in security exceptions. By watching when our factory gets called and when we get exceptions, it looks like what is happening is:
1 - First RMI call - no connection; therefore, factory called. Factory creates (legal) socket connected to B, and tells the "link" to hook it up to A.
2 - The RMI call completes, everything is fine.
3 - Second RMI call - there's a connection already; the factory is not called, but the RMI code checks security AGAINST THE ORIGINAL INTENDED CONNECTION (to machine A); security check fails and the socket is closed.
4 - Third RMI call - once again, there's no connection, so the factory is called to create one; the RMI call works.
So, it seems the RMI code does not check security itself when it creates a socket - it assumes security is checked when the socket is created. But if it already has a socket, it does check security, and looks at where it "wanted" a socket rather than where the socket actually goes.
We can work around the problem by re-trying after we get the security violation, or maybe we could force the socket to close after each transaction, but obviously, these are less efficient than leaving the socket open.
Can anyone suggest a way to either get RMI to check against the socket's real connection, or to not check security when the socket already exists?