adrianward
asked on
Effeciency of IPC, suggestions please!
Hi.
Scenario: Several different processes written in either C or Perl, all needing to communicate with each other (on the same machine).
I need a central data store, which all processes can read/write to, which I was going to implement as a server/client system, using Unix Domain sockets. But one process will need to be reading/writing to this store up to 20,000+ times a second. Is this wise?
What on earth is the best way to go about doing this? Can I use shmem between both Perl and C (I have _no_ experience of shmem, I only have a vague idea what it does)?
Any suggestions or thoughts on effeciency, suitability, and thoughts on whether this project is doable or not are warmly appreciated.
Many thanks in advance.
Scenario: Several different processes written in either C or Perl, all needing to communicate with each other (on the same machine).
I need a central data store, which all processes can read/write to, which I was going to implement as a server/client system, using Unix Domain sockets. But one process will need to be reading/writing to this store up to 20,000+ times a second. Is this wise?
What on earth is the best way to go about doing this? Can I use shmem between both Perl and C (I have _no_ experience of shmem, I only have a vague idea what it does)?
Any suggestions or thoughts on effeciency, suitability, and thoughts on whether this project is doable or not are warmly appreciated.
Many thanks in advance.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Wow, your answer is very complete. Thank you for your effort.
I'm not sure I'm up to implementing all the ideas you have suggested, but your thoughts about multithreaded server was one that I'd previously considered.
However, initially I have rethought the problem through and have tried using shared files - I managed to rejig it so that one process only needs to -receive- data from other processes (this wasn't clear before). So now I have this one process monitoring some files and then whenever another process updates one of these files, the first notices this change (by monitoring the modification date) and re-reads the file.
I know this isn't efficient (at all, infact - it hogs the CPU) but I have found it suitable for my problem, and I'm sure I can improve it's performance a bit more.
Thanks for your effort, though. It's always nice to know what better ways I could achieve things.
I'm not sure I'm up to implementing all the ideas you have suggested, but your thoughts about multithreaded server was one that I'd previously considered.
However, initially I have rethought the problem through and have tried using shared files - I managed to rejig it so that one process only needs to -receive- data from other processes (this wasn't clear before). So now I have this one process monitoring some files and then whenever another process updates one of these files, the first notices this change (by monitoring the modification date) and re-reads the file.
I know this isn't efficient (at all, infact - it hogs the CPU) but I have found it suitable for my problem, and I'm sure I can improve it's performance a bit more.
Thanks for your effort, though. It's always nice to know what better ways I could achieve things.
If you use shared memory you will have the contention problem, for which you will need to use something like a semaphore. This is going to kill your performance.
A better approach might be to create a conversation server which would work as outlined before.