I have a situation where I am using Perl and SOAP::Lite to build a system of multiple machines and multiple layers of communication. Now I am running into a scalability issue that I did not forsee. In certian cases of this communication I want to send a request and I dont want to wait for a response. Now I know that is a limitation at several levels http, soap, and perl. But what I am thinking of doing is on certian requests I want to spawn a thread to respond with a status of 0 but have my service continue processing the request. Right now it looks something like this
client (sends request to process log) -> server (processes log)
client <- server (sends response when completed)
What I want is
client (sends request to process log) -> server
client <- server (responds with return value right away)
server (processes log and doesnt send any further notice to anyone)
My server is actually a soap service as a cgi script on apache. Now I have never used fork or threads and I am wondering how I should design my service to deal with that. I know there are a bunch of issues with sharing objects and memory. But I think I should be able to avoid most of those because I only need to spawn 1 thread or fork to either return a value to the soap client or to process the log and let the main script return a value.
I am thinking a fork might not be the best thing because it looks like its a copy of the whole script and I dont know if I want to do that. And I dont think it would be able to know about that session or variables passed into that would it?