Solved

High performance TCP server  (Linux C)

Posted on 2010-09-18
9
1,058 Views
Last Modified: 2012-05-10
Hi guys,

I have an app that generates content that needs to be shipped out to interested parties. Originally I thought about using UDP but I want the guarantee of delivery that TCP provides and I'm not convinced (by a long way) that I could do a better job. So now I'm thinking that I would be better off creating a simple TCP server. Clients would connect to the server and whenever a snippet of data is ready, I would fire it off to all interested parties.

Does anyone have any examples or feedback on this implementation? I'm not looking for complete examples ( though finding any on the web would be great), but I really just want to make sure I'm doing this the right way. I want to make sure that each interested party gets the data as quickly as possible...

Thanks in advance!
0
Comment
Question by:PMembrey
  • 5
  • 4
9 Comments
 
LVL 53

Expert Comment

by:Infinity08
Comment Utility
What you describe sounds a lot like an RSS feed :

        http://en.wikipedia.org/wiki/RSS

Would that suit your needs ? If not, why not ?
0
 

Author Comment

by:PMembrey
Comment Utility
Because I need to push out updates in real time. Rss is effectively a polling technology. The payload is also in a binary format.

:-)
0
 
LVL 53

Expert Comment

by:Infinity08
Comment Utility
>> Because I need to push out updates in real time.

Well, you said :

>> Clients would connect to the server

But I assume I misinterpreted that, and you meant that they would connect once and stay connected "forever" ?

>> Rss is effectively a polling technology.

True, but it depends on how you define real-time. The clients can poll every few seconds if they want - or they can poll less frequently if they are less interested in getting the latest information (in other words : you let the clients choose).


>> The payload is also in a binary format.

Ok. So, maybe RSS is not perfectly suited, but would the same idea suit your needs (given a relaxed definition of real-time) ?

I'm just trying to get a better understanding of what your exact needs are, so I can give better advice.

Is it possible to give a bit of background information about what kind of data we're talking about here ? And how fast the clients would need to get it ?
0
 

Author Comment

by:PMembrey
Comment Utility
Sorry, I should have giving more information. I trade forex and have several applications that need the same data feed. At present they all connect directly to the brokers server ( right now over the internet ) and when the market gets busy, I start to see slippage on limit orders because basically the market moved before I could place my order.

 I'm writing a simple proxy app so that only one connection is needed for my apps, but I need to push out the data in as close to real time as possible.

Hope that helps :)
 
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 
LVL 53

Accepted Solution

by:
Infinity08 earned 500 total points
Comment Utility
Right, that makes sense then :)

So it'll probably be best to roll your own according to your specific requirements. Fortunately, creating a TCP server is not very complicated. I can recommend Beej's guide as an excellent resource to get you started :

        http://beej.us/guide/bgnet/

You'll have to allow for incoming connections, and keep track of all current connections in some data structure (a list eg.). Every time new data becomes available, you can then iterate over that list, and send it to each of the connected clients.
If the number of clients is reasonably small, you can also opt for having a thread for each connection.

Is the protocol used for the data transfer an open protocol ? Is it difficult to implement yourself ?
0
 

Author Comment

by:PMembrey
Comment Utility
The protocol is simple, I can literally pipe it to any number of apps. Its effrctively a one way stream. I have five apps at the moment so a thread each should be fine I hope :)
0
 
LVL 53

Expert Comment

by:Infinity08
Comment Utility
5 threads is no problem at all - especially given that they're just passing on data.

So, accept an incoming connection. Create a new thread for it. And have the thread be signalled when new data is available, which the thread can then get from a central location, and send it on to the client.
That should do it.

Of course, if you consider threads to be overkill, you can still go for the list approach I mentioned earlier. For 5 clients, there should be no noticeable delay between the first and last in the list.
0
 

Author Comment

by:PMembrey
Comment Utility
Just reading through the guide you gave me - I'll be awarding the points for that :-)

If you could give more detail on the implementation you suggested (accept a connection, then multiple threads) I will start to build it and see what I end up with. I think multi-threaded would be best and the machine has plenty of cpu power to spare (it's a core i7).

Any other handy hints or hard won expertise also gratefully welcomed :)
0
 
LVL 53

Expert Comment

by:Infinity08
Comment Utility
>> and the machine has plenty of cpu power to spare (it's a core i7).

Since it's just acting as a gateway for forwarding data, this will not be CPU intensive. So, you should have a lot of room to spare ;)


>> If you could give more detail on the implementation you suggested

I could write a book about it ;) It would be best if you tell me what part you need more information about, and I'll be happy to provide it.

I'll keep monitoring this question for the foreseeable future, and I'll be happy to answer all your follow-up questions. So, take your time :)


>> Any other handy hints or hard won expertise also gratefully welcomed :)

I would recommend to start with a very simple server. Have just one client, no threads, and get things working that way. That will give you the confidence to extend it with threads and multiple clients.

Beej's guide contains some server sample code you can use as a starting point. Get it running, and then get it to correctly forward the data to one client. Once you're happy with how it works (it's stable, and does the job you want it to do) - only then start extending the functionality to multiple clients.
0

Featured Post

Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

Join & Write a Comment

Suggested Solutions

This tutorial is posted by Aaron Wojnowski, administrator at SDKExpert.net.  To view more iPhone tutorials, visit www.sdkexpert.net. This is a very simple tutorial on finding the user's current location easily. In this tutorial, you will learn ho…
Summary: This tutorial covers some basics of pointer, pointer arithmetic and function pointer. What is a pointer: A pointer is a variable which holds an address. This address might be address of another variable/address of devices/address of fu…
The goal of this video is to provide viewers with basic examples to understand and use pointers in the C programming language.
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use while-loops in the C programming language.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now