We’re in the process of designing a server-side application.
The first "filter" component of the server should receive requests from multiple concurrent clients over TCP/IP, each request containing an entity ID number. The filter should then process each ID and decide whether to pass it on to the next component (database) according to a given set of rules. The first version of the system should deal with around 20 concurrent clients producing ~1,000 requests per second, but future versions should be much more scalable (up to hundreds of concurrent clients and ~100,000 requests per second). The development environment is Microsoft .NET on Windows 2000 servers.
We came up with two possible architectures for the filter component:
1. Maintain a different communication thread for every client. Every such communication thread receives ID’s from its corresponding client and writes them into a common queue. A different worker thread reads data from the queue and processes the requests.
2. Maintain one communication thread for all clients. The communication thread reads ID’s and writes them into a queue. Several worker threads, managed in a thread pool, read data from the queue and process requests.
The first solution is obviously less scalable, since when the number of clients increases the overhead of a large number of threads starts affecting performance. Still, it is easier to code and it fits the first version/versions of the server.
Our question regarding the second solution, which seems more "server-oriented", is this: when taking into consideration the large number of requests per second and the fact that the processing time of each request is relatively short, isn’t the overhead of managing a thread pool, allocating a worker thread for every new request and returning it back to the pool when finished, etc. becoming too expensive ? Won't this affect the overall performance ?
1. What should be the limit (roughly speaking) of the number of concurrent communication threads we can expect to run on a standard Windows 2000 machine ?
2. Is there any other alternative for the design of the filter component ? Are we missing something basic ?
Thanks in advance,