General design question - A data server and various clients

stormist
stormist used Ask the Experts™
on
I am looking for some suggestions on how to design my server and clients. The server in this case will deliver price data on stocks. There are basically two types of data requested for each stock the user is watching
1) Historical data:an initial stream of data, roughly 50-100kb in size giving past price history on that stock
2) live data updated every second or so.

What would be the most efficient way to deliver this data to the clients? What are some ways the .NET helps me to deal with having multiple clients connected to the server at a time?

I'm assuming this type of bidirectional communication is best done over TCP sockets? How are sockets generally assigned in this case, when the number of clients is unknown at any given time?

When say 100+ clients are connected to the server, and a particular price is updated say every half second, is say a simple for loop sufficient to distribute the data over the various outgoing sockets; or is there some advanced technique that might be utilized to speed this up? (multi-threading ?)

Any thoughts and sharing of your network expertise would be most appreciated.
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
DarrenSenior Software Engineer

Commented:
Hi,

Firstly is the information critical meaning: Does it have to be delivered to the client or could you miss one or  two packets and still keep going. If you missed info for a couple of seconds and you got the information on the third would it matter.... I only asking this as you could use multicasting which would mean that you would have one server that sends out a stream to all of the clients connected to the multicast address.

The problem with multicast are that messages can get lost. So it depends on how mission critical your systems is.

Another solution is to use TCP and have your clients connect directly to your server and your server just sends out messages to the client as they are requested.

Hope this helps a little. Sure see what some others come back with

Darren
Software Architect / Team Leader
Commented:
This is a very tricky situation.
The solution depends on how critical the data is, how much bandwith you have, how many connections your server can serve simultaniously, ...

When you have 10 connections you can use something as simple as a for loop to push the data and if the amount of connections rise further you can use a seperate thread for each 10 connections for example. But if the amount of connections increase further you will have to switch to another scenario like multicasting. This will reduce the strain on the server and the network greatly.

An other possibility is to reverse the logic and use something like a webservice to handle the request. Means that you let the clients poll for the data and no longer push the data. The advantage of this is that you can use some very convienent IIS and server load balancing. The disadvantage is that the stress on the network remains. The stress on the server stays too but can be managed. If you don't like the webservices idea (it has alot of overhead) you can also use the polling with tcp sockets (connect, ask, disconnect); this will allow to queue requests if the stress gets to high.

A last possibility is using "good old COM+". It has methods on which you can subscribe to get updates, so from server point of view you fire an event every half a second and COM+ does the rest. The only thing you have to do is have you client connect to the object and subscribe to the events. This solution is also HIGHLY scalable through the numberous options that are available in COM+.

Hope this helps a little.
Peter

Author

Commented:
Thanks for everyones comments so far. The live data can have lost packets, because once a few seconds past a live quote is no longer valid.

On multicasting and COM+, any good recommendations/resources to learn more? (I've already started googling away)

The most useful thing I think would be sample projects built using these methods.
Exploring SharePoint 2016

Explore SharePoint 2016, the web-based, collaborative platform that integrates with Microsoft Office to provide intranets, secure document management, and collaboration so you can develop your online and offline capabilities.

DarrenSenior Software Engineer

Commented:
I've got a really simple and I mean simple project done using multicasting which you can have if you want.

I think I can upload it on this site somhow but I'm not sure. I'll have a look on this site and see how its done.

I can't even remember what the project does but basically it sends a message and all members of the multicast group receive the message or something like this.

This is a good article I think!! I bookmarked it so I must have wanted it
http://www.osix.net/modules/article/?id=409
illusioSoftware Architect / Team Leader

Commented:
A COM+ application can be programmed very quickly. The only trick is in correct deployment and configuration of the COM+ system.
The most extended information I have found so far can be read at:
http://www.codeproject.com/dotnet/complusnetpracticalapp01.asp
(don't forget the part 2 and part 3 articles)

Kind regards,
Peter

Author

Commented:
Peter,
Do you have an example of a working cross internet client/server COM program? That article was hard to follow.

Author

Commented:
From what I understand, I still have to open up a socket between the client and server? This is not handled by the methods above? A question I have is can multiple clients connect to a single IP/port combination.. How does that work?

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial