Solved

UDP or TCP Streams

Posted on 2001-06-03
57
1,385 Views
Last Modified: 2008-03-03
I am currently studying Sockets in Delphi and I'm having some trouble. Can anyone post source code that implements either TCP Streams or Reliable UDP Streams. The code must use raw Winsock functions. I would really appreciate it. Thanks!
0
Comment
Question by:DelFreak
  • 31
  • 23
  • 2
  • +1
57 Comments
 

Expert Comment

by:reuveni
Comment Utility
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
DelPhreak, what exactoly is your problem? Include WINSOCK in your "uses" list, declare a socket handle of the type TSOCKET. Use SOCKET to allocate a socket handle and define what kind of socket you want (TCP is the stream protocol). Then connect to another host using CONNECT, send and receive data using SEND and RECV, and when you're done close the socket and free the handle using CLOSESOCKET.

That's pretty much it.
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss, I want to create a TCP packet stream and also a reliable UDP stream. Can you help me out?
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
DelFreak, there is no such thing as a reliable UDP stream. UDP is packet oriented and unreliable; if you need a stream, you have to use TCP which is reliable and streamed.

I don't have enough details about your project to give more help than what I have given above. Look up the functions I wrote doen in uppercase in the help file (In the start menu, Borland Delphi -> Help -> MS-SDK help files -> Windows Sockets 2, Index)
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss, yes I know that UDP is an unreliable protocol. What I want to do create a TCP or UDP Packet Stream using raw Winsock functions. Can you help me out?
0
 

Author Comment

by:DelFreak
Comment Utility
I read about this in http://www.xiph.org/archives/vorbis/200007/0093.html

Maybe you can help me out. Thanks!
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
DelFreak, I read the URL you gave me. However, I don't see anything specific there; I still don't know what protocol you want to implement nor the exact purpose of your code.
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss, I would like to implement a program that uses TCP or UDP packet streams. I hear it is very reliable especially when transferring large amounts of data. Any code that will give me an idea on how to implement packet streams will be fine. Thanks!
0
 

Author Comment

by:DelFreak
Comment Utility
Or any code that will allow me to check all outgoing UDP datagrams thus making the connection more reliable. Thanks!
0
 

Author Comment

by:DelFreak
Comment Utility
Please! I need help!
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
DelFreak, packet streams are NOT reliable. That's not what they are designed for. However, they are more flexible (in regards of adaptive data rate or multicasting), and they can scale better.

The calls to winsock functions above is all you need to send or receive data using both UDP or TCP. That's all you'll get from me for the points offered.
0
 

Author Comment

by:DelFreak
Comment Utility
Okay! I'll add more points. How many do you want for a complete source that implements UDP and TCP Packet Streams?
0
 

Author Comment

by:DelFreak
Comment Utility
Will 150 suffice? Just tell me if you want more and I'll add more points.
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
Do you need both a client and a server, or do you want to connect to an existing server? If you need a server, do you want it multithreaded? Do you have any special requirement for the protocol to be used? Give some more info, so that I'll be able to estimate the works to implement it.
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss, Thank you for helping out. I am applying for a new job and it requires that I am skilled with sockets programming and both TCP and UDP protocols. The code you post will help me a lot. After this, I will post a question on tunneling. Would you know about that?

Anyway, I need it in C/S form and it must use raw Winsock functions. It is very important that it is fully commented as I need to learn how it works. A multi-threaded server will be great. No special protocol is required although I would appreciate it if you could make a different example for each of the following protocols:

1. UDP
2. TCP
3. RAW (If possible)

If you need more points, just tell me. I just really need your help. Thank you!
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
DelFreak, I think I get the idea. Note that RAW data exchange is not working on all stacks (actually, only Windows 2000 has full RAW support; the other Socket 2 implementations allow RAW sockets to be used for ICMP but not much more since not all header fields can be defined manually).

I'm gonna write some simple code for you, to start with a TCP echo server (that is, he sands back everything you send him). I take TCP in a first step because it easily allows testing using the TELNET application. When do you need that stuff to be done?
0
 

Author Comment

by:DelFreak
Comment Utility
Anytime soon. Yes, I read that only Win2K has full Raw Socket support. However, it will come in handy. Please don't forget that your example must use the Packet Streaming technique. I would appreciate an example for all 3 protocols but you may start with TCP if you wish. I'll add more points if you need them. Thank you.
0
 

Expert Comment

by:reuveni
Comment Utility
AvonWyss
can you also post me the same example I am also
interested in this issue,
if you will post me I will grant you with more 100 points.
Thanks Eli
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
Ok, here comes the first part. A simple, multi-threaded echo server listening on port 1234. You can test it using telnet to connect to the machine running the service on port 1234.

I tried to keep it straightforward and clear to understand. However, by this, I left some things away. For instance, the code does assume that noe error ossurs in most socket operations. In a true server, this assumption must not be made; return codes of the WINSOCK functions have to be checked for error conditions.

I use both blocking and nonblocking modes, depending on what suits best for the given task. Note that, by defaulkt, sockets are created in blocking mode; however, sockets inherit their settings from their "parent" when they are created by the means of Accept. That's why I change the mode explicitly to blocking in the DataThread.

The app will run on the command line and has very little overhead.


program TCPServer;

{$APPTYPE CONSOLE}

uses
      Windows,  // basic OS stuff like sleep
      WinSock,  // socket calls
      SysUtils, // exception handling
      Classes;  // basic classes, threads

type
      TDataThread=class(TThread)
      private
            FSocket: TSocket;
      public
            constructor Create(ASocket: TSocket);
            procedure Execute; override;
      end;
      TListeningThread=class(TThread)
      private
            FSocket: TSocket;
            FListeningPort: Word;
      public
            constructor Create(ListeningPort: Word);
            procedure Execute; override;
      end;

{ TListeningThread }

constructor TListeningThread.Create(ListeningPort: Word);
begin
      FListeningPort:=ListeningPort;
      inherited Create(False);
end;

procedure TListeningThread.Execute;
var
      BindAddr: TSockAddrIn;
      Sock: TSocket;
      WSAData: TWSAData;
      Arg: Longint;
begin
      WSAStartup($101,WSAData); // initialize sockets library for this application
      FSocket:=Socket(AF_INET,SOCK_STREAM,IPPROTO_TCP); // create new TCP socket, unbound and unconnected
      Arg:=1;
      IOCtlSocket(FSocket,FIONBIO,Arg); // set socket to nonblocking mode. this allows to check for Terminated
      FillChar(BindAddr,SizeOf(BindAddr),0);
      BindAddr.sin_family:=AF_INET;
      BindAddr.sin_port:=HToNS(FListeningPort);
      BindAddr.sin_addr.S_addr:=INADDR_ANY; // fill up TSockAddrIn structure with address to bind to
      Bind(FSocket,BindAddr,SizeOf(BindAddr)); // bind the socket to the specified address and port
      Listen(FSocket,5); // start listening on this socket
      WriteLn('Listening start on port ',FListeningPort);
      repeat
            Sock:=Accept(FSocket,nil,nil); // try to accept a new connection. will return immediately (non-blocking mode)
            if Sock<>SOCKET_ERROR then // did we get an incoming connection?
                  TDataThread.Create(Sock) // yes: create thread to handle it
            else
                  Sleep(20); // no: we are idle, wait a little (-> will keep CPU usage low)
      until Terminated; // loop only if the thread was not marked as being terminated
      CloseSocket(FSocket); // release socket handle
      WriteLn('Listening ended');
      WSACleanup; // close socket library
end;

{ TDataThread }

constructor TDataThread.Create(ASocket: TSocket);
begin
      FSocket:=ASocket;
      WriteLn('Accepted socket ',FSocket);
      inherited Create(False);
end;

procedure TDataThread.Execute;
var
      Len,Arg: Integer;
      Buf: packed array[0..1023] of Byte;
begin
      Arg:=0;
      IOCtlSocket(FSocket,FIONBIO,Arg); // set socket to blocking mode (calls will not return immediately)
      repeat
            Len:=Recv(FSocket,Buf,SizeOf(Buf),0); // try to receive data. returns on data received or connection reset
            if Len>0 then begin // did we get data?
                  Send(FSocket,Buf,Len,0); // yes: send it out (echo server)
                  WriteLn('Data transfer %d');
            end else
                  Terminate; // no: mark thread to terminate
      until Terminated; // loop only if the thread was not marked as being terminated
      CloseSocket(FSocket); // release socket handle
      WriteLn('Disconnected socket ',FSocket);
end;

var
      Listening: TThread;

begin
      Listening:=TListeningThread.Create(1234);
      WriteLn('Press RETURN to abort');
      ReadLn;
      Listening.Terminate;
      Listening.WaitFor;
end.
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss, Thank you! A few Q's though. Can you specify which part of the code you posted describes Packet Streaming? Why does DataThread use Synchronus mode? Also, maybe you could post a full server with error checking and if possible in Asynchronus mode. Thanks again!
0
 

Author Comment

by:DelFreak
Comment Utility
I realize that I asked for three examples and 150 points is too small for my request. I have raised the points to 300. I hope that this will suffice. If more is needed, please tell me. Thank you!
0
 

Author Comment

by:DelFreak
Comment Utility
Also, maybe you could create a client so I don't have to use Telnet. It will help me understand things better. Thanks!
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
OK, here are the answers to your Q's:

* As said, this is NOT packet streaming, but a small echo server. It's purpose is NOT to stream data, but to show you how socket calls work without having many thousands of lines for code handling packets, error conditions, timeouts, file I/O etc. This code will listen on a TCP port, when a connection comes in it will echo the data received, and it will correctly shutdown. That's it.

* You asked for a multithreaded server. The classigc multithread server uses blocking calls and opens a thread for each data connection. Async mode in mutlithreading environment gives a huge management overhead (thread pooling, event and task sheduling, data synchronizing), so that it cannot be implemented in such ways that you can learn from it without the necessary knowledge of the single parts. So this would not have helped you. Look, I do have the code for that, I use it in server applications I write; however, this code belongs to the firm I work for and therefore I'm not allowed to post it here, and it does have thousands of lines which would not help you understand the issues at all because you wouldn't know what part of the code does what. Read this for more information on blocking/nonblocking operation:
http://www.cyberport.com/~tangent/programming/winsock/articles/io-strategies.html

* Why would it help better to use another client than telnet? For the server provided, telnet is the optimal testing suite; you can set breakpoints in your server code and step through the code, and you can test this echo server completely. Of coursem when it comes to UDP, another test program will be needed. And it may be a good excercise to make a client yourself, it's even easier than the server (no multithreading necessary, only one socket to manage).
0
 

Author Comment

by:DelFreak
Comment Utility
Oh! But I wanted an example of Packet Streaming. that was the main purpose of this question. TCP/UDP sockets, Asynchronus, Synchronus, no problem because i am quite familiar with it. Packet Streaming is what I want to learn about. Will you be able to make an example for me?
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
Well, initially you asked: "Can anyone post source code that implements either TCP Streams or Reliable UDP Streams. The code must use raw Winsock functions." I posted code that implements a TCP stream echo server, using raw Winsock functions. Because of this, I concluded that you were not familiar with the use of raw Winsock calls, and that's why I made a basic yet working Winsock server app.

What do you want to go into details? I can write you some packet transmission code, but this doens't have much to do with your first question.
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss, I'm sorry for the misunderstanding. When I posted this question I assumed that TCP/UDP Streams reffered to Packet Streaming in general. My mistake. Anyway, I would like to learn about TCP and/or UDP Packet Streaming. If you can, please post an example project that iplements this technique. The implementation must use raw Winsock functions since I don't like using VCL's. I have already raised the points to 300. If more is needed then please tell me.
0
 

Author Comment

by:DelFreak
Comment Utility
Can anyone please help me.
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
DelFreak, "Packet Streaming" is quite a loose expression. If you like, I'll implement a small broadcast client and server which sends packets (with any binary data) to clients connected. But give me some days, I have lots of other stuff going on currently.
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 

Author Comment

by:DelFreak
Comment Utility
Hi! AvonWyss, Streaming meaning the client app reads the data coming in, and discards that data as newer data arrives. Something like Audio Streaming or any other data. Can you be able to give me an example? TCP and/or UDP will be fine depending on which protocol you're more familiar with. Please reply ASAP. Thanks!
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
Yes, something like this was in my mind. Since you asked for a reliable transport, I'm going to use TCP. If lossless transfer was not necessary (for Audio and Video there are methods to deal with losses, but that's definetely out of scope here). Coming soon.
0
 

Author Comment

by:DelFreak
Comment Utility
Thanks!
0
 

Author Comment

by:DelFreak
Comment Utility
Can you do it for UDP as well? It will really help me out. Also, please make the example in client/server architecture. Thanks!
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
Ok, here's a packet transmission server and client for the TCP protocol. If you would want to use UDP, it would be pretty much the same idea but the client would have to send acknowledgments to the server (for reliable transmission), making the whole thing larger and less easy to understand. I tried to make the source code such that you can easily focus on the main aspects of WinSock and packets.

If a client is too slow reading data, the server will skip data, but because of TCP and the serial number included in each packet it's easy to detect this by the client. The client I wrote does not provide a feedback other than the TCP ack packets to the server.

Here you go:



program TCPPacketServer;

{$APPTYPE CONSOLE}

uses
      Windows,  // basic OS stuff like sleep
      WinSock,  // socket calls
      SysUtils, // exception handling
      Classes;  // basic classes, threads

type
      TPacket=packed record
            Serial,Length: Longint;
            Data: packed record end;
      end;
      PPacket=^TPacket;
      TPacketSender=class(TThread)
      private
            FQueue,FSockets: TList;
            FLock: TRTLCriticalSection;
            FListeningPort: Word;
            FListeningSocket: TSocket;
            FSerial,FPacketsSent,FPacketsDropped: Integer;
      protected
            procedure Execute; override;
      public
            constructor Create(ListeningPort: Word);
            procedure Send(var Data; Length: Cardinal);
            destructor Destroy; override;
      end;

{ TPacketSender }

constructor TPacketSender.Create(ListeningPort: Word);
var
      WSAData: TWSAData;
begin
      FQueue:=TList.Create;
      FSockets:=TList.Create;
      InitializeCriticalSection(FLock);
      FListeningPort:=ListeningPort;
      WSAStartup($101,WSAData);
      inherited Create(False);
end;

destructor TPacketSender.Destroy;
begin
      Terminate;
      WaitFor;
      inherited;
      FQueue.Free;
      FSockets.Free;
      DeleteCriticalSection(FLock);
      WSACleanup;
end;

procedure TPacketSender.Execute;
var
      I: Integer;
      BindAddr: TSockAddrIn;
      Sock: TSocket;
      Packet: PPacket;
begin
      FListeningSocket:=Socket(AF_INET,SOCK_STREAM,IPPROTO_TCP); // create new TCP socket, unbound and unconnected
      try
            I:=1;
            IOCtlSocket(FListeningSocket,FIONBIO,I); // set socket to nonblocking mode. this allows to check for Terminated
            FillChar(BindAddr,SizeOf(BindAddr),0);
            BindAddr.sin_family:=AF_INET;
            BindAddr.sin_port:=HToNS(FListeningPort);
            BindAddr.sin_addr.S_addr:=INADDR_ANY; // fill up TSockAddrIn structure with address to bind to
            Bind(FListeningSocket,BindAddr,SizeOf(BindAddr)); // bind the socket to the specified address and port
            if Listen(FListeningSocket,5)<>0 then // start listening on this socket
                  Abort; //error when listening on this socket
            repeat
                  repeat
                        Sock:=Accept(FListeningSocket,nil,nil); // try to accept a new connection. will return immediately (non-blocking mode)
                        if Sock<>SOCKET_ERROR then
                              FSockets.Add(Ptr(Sock)); //if we got a new socket, add it to the subscriber list
                  until Sock=SOCKET_ERROR;
                  while FQueue.Count>0 do begin //check for items queued to be sent
                        EnterCriticalSection(FLock); //retrieve item inside the critical section
                        try
                              Packet:=FQueue[0];
                              FQueue.Delete(0);
                        finally
                              LeaveCriticalSection(FLock);
                        end;
                        for I:=0 to FSockets.Count-1 do
                              if Winsock.Send(Integer(FSockets[I]),Packet^,Packet.Length,0)<>Packet.Length then
                                    if WSAGetLastError=WSAEWOULDBLOCK then //send buffer full?
                                          Inc(FPacketsDropped) // discard packet
                                    else begin //any other socket error
                                          CloseSocket(Integer(FSockets[I])); //close erraneous socket
                                          FSockets.Delete(I); //and remove it from our list
                                    end
                              else
                                    Inc(FPacketsSent);
                  end;
                  Sleep(20); //all work done for now, wait some
            until Terminated;
      finally
            EnterCriticalSection(FLock);
            CloseSocket(FListeningSocket); //close listening socket
            for I:=0 to FSockets.Count-1 do //delete all subscribed sockets
                  CloseSocket(TSocket(FQueue[I]));
            FSockets.Clear;
            for I:=0 to FQueue.Count-1 do //delete all pending packets
                  FreeMem(FQueue[I]);
            FQueue.Clear;
            LeaveCriticalSection(FLock);
      end;
end;

procedure TPacketSender.Send(var Data; Length: Cardinal);
var
      Packet: PPacket;
begin
      GetMem(Packet,Length+SizeOf(Packet^)); //allocate memory for packet
      Packet.Length:=Length+SizeOf(Packet^); //set length...
      Move(Data,Packet.Data,Length); //...and copy data
      EnterCriticalSection(FLock); //enter critical section
      try
            Inc(FSerial); //increase packet serial number
            Packet.Serial:=FSerial; //set serial number of packets, so that a client can detect dropped packets
            FQueue.Add(Packet); //add packet to send queue
      finally
            LeaveCriticalSection(FLock); //leave critical section
      end;
end;

var
      Listening: TPacketSender;
      S: string;

begin
      Listening:=TPacketSender.Create(1234);
      WriteLn('Empty line aborts.');
      repeat
            ReadLn(S);
            if S<>'' then
                  Listening.Send(Pointer(S)^,Length(S));
      until S='';
      Listening.Free;
end.



program TCPPacketClient;

{$APPTYPE CONSOLE}

uses
      Windows,  // basic OS stuff like sleep
      WinSock,  // socket calls
      SysUtils, // exception handling
      Classes;  // basic classes, threads

type
      TPacket=packed record
            Serial,Length: Longint;
            Data: packed record end;
      end;

var
      WSAData: TWSAData;
      ClientSocket: TSocket;
      S: string;
      Packet: TPacket;
      Addr: TSockAddrIn;

begin
      WSAStartup($101,WSAData);
      ClientSocket:=Socket(AF_INET,SOCK_STREAM,IPPROTO_TCP); // create new TCP socket, unbound and unconnected
      Addr.sin_family:=AF_INET; // fill up TSockAddrIn structure with address to bind to
      Addr.sin_port:=HToNS(1234); //port
      Addr.sin_addr.S_addr:=Inet_Addr('127.0.0.1'); //ip address (no host name!)
      if Connect(ClientSocket,Addr,SizeOf(Addr))=0 then
            while Recv(ClientSocket,Packet,SizeOf(Packet),0)=Sizeof(Packet) do begin
                  SetLength(S,Packet.Length-Sizeof(TPacket));
                  Recv(ClientSocket,Pointer(S)^,Length(S),0);
                  WriteLn(Packet.Serial:6,': ',S);
            end
      else
            WriteLn('Connect Error');
      CloseSocket(ClientSocket);
      WSACleanup;
end.


I believe that this should give you a good understanding and starting point for other developments, no matter if they are using TCP or UDP (which really just works mostly the same, but you may not make that many assumptions as you can with TCP, like havin all packets arrive and such).
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
Oh yeah, fogot to mention how to use the programs:

* The server will (by default) listen on port 1234.
* Any number of clients may connect simultaneously.
* You can type in some text and hit ENTER to send it to the client. Note that an empty line will terminate the server.

* The client connects to a given IP address.
* The client will terminate if the connection is lost. Of course, in a real application receiving data in a separate thread, you can have any termination sequence you like.
* The client will write the received packet serial number and contents on the screen when it arrives.
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss, the example is great. A few questions though:
1. Which function in the server does the actual streaming?
2. If I want the Client to send data to the server, do I just use the same functions of sending as the server?
3. How will the server process the packets? Like the Client does?
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
1. The server has a thread which checks every now and then (that is, about 50 times per second) if there is data queued to be sent out. The code "while FQueue.Count>0 do begin" starts the block which checks and sends data. I made the data queue thread-safe, so that any thread can queue data packets at any time without access collisions (the code is so-called "thread-safe").

2. The client could send data back also with Send(), quite similar to the way data is sent by the server. However, to avoid impact on the stream, you should only send back small amounts of data (if any, that is).

3. The server does not check for incoming data at all. You'd have to add a third loop in the Execute which would check all connections for incoming data every now and then.


reuveni: I did post the code, are you going to post the points you offered?
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss, Could you modify the code so that the client could send data and the server could receive and process it and could you also post a UDP example. I'll add more points if neccesary. Also, do you know anything about TCP Tunneling? Can this be done in Delphi? Thanks!

BTW, I think reuveni already posted a Q containing your points.
0
 
LVL 14

Accepted Solution

by:
AvonWyss earned 300 total points
Comment Utility
DelFreak, here you go. The TCP client and server with two-way communication.

* The client will send an "ACK serial" packet back when it gets a packet from the server. Of course, any packet could be sent to the server at any time, but like this all (new) clients will send back data to the server simultaneously, making it more like a real-world situation.

* The server will read and display packets received including the connection ID. I also added connection/disconnection messages to the server as well as enhanced some small things.



program TCPPacketServer;

{$APPTYPE CONSOLE}

uses
      Windows,  // basic OS stuff like sleep
      WinSock,  // socket calls
      SysUtils, // exception handling
      Classes;  // basic classes, threads

type
      TPacket=packed record
            Serial,Length: Longint;
            Data: packed record end;
      end;
      PPacket=^TPacket;
      TPacketSender=class(TThread)
      private
            FQueue,FSockets: TList;
            FLock: TRTLCriticalSection;
            FListeningPort: Word;
            FListeningSocket: TSocket;
            FSerial,FPacketsSent,FPacketsDropped: Integer;
      protected
            procedure Execute; override;
      public
            constructor Create(ListeningPort: Word);
            procedure Send(var Data; Length: Cardinal);
            destructor Destroy; override;
      end;

{ TPacketSender }

constructor TPacketSender.Create(ListeningPort: Word);
var
      WSAData: TWSAData;
begin
      FQueue:=TList.Create;
      FSockets:=TList.Create;
      InitializeCriticalSection(FLock);
      FListeningPort:=ListeningPort;
      WSAStartup($101,WSAData);
      inherited Create(False);
end;

destructor TPacketSender.Destroy;
begin
      Terminate;
      WaitFor;
      inherited;
      FQueue.Free;
      FSockets.Free;
      DeleteCriticalSection(FLock);
      WSACleanup;
end;

procedure TPacketSender.Execute;
var
      I: Integer;
      BindAddr: TSockAddrIn;
      Sock: TSocket;
      Packet: PPacket;
      Header: TPacket;
      S: string;
begin
      try
            FListeningSocket:=Socket(AF_INET,SOCK_STREAM,IPPROTO_TCP); // create new TCP socket, unbound and unconnected
            try
                  I:=1;
                  IOCtlSocket(FListeningSocket,FIONBIO,I); // set socket to nonblocking mode. this allows to check for Terminated
                  FillChar(BindAddr,SizeOf(BindAddr),0);
                  BindAddr.sin_family:=AF_INET;
                  BindAddr.sin_port:=HToNS(FListeningPort);
                  BindAddr.sin_addr.S_addr:=INADDR_ANY; // fill up TSockAddrIn structure with address to bind to
                  Bind(FListeningSocket,BindAddr,SizeOf(BindAddr)); // bind the socket to the specified address and port
                  if Listen(FListeningSocket,5)<>0 then // start listening on this socket
                        Abort; //error when listening on this socket
                  repeat
                        I:=1;
                        repeat
                              Sock:=Accept(FListeningSocket,nil,nil); // try to accept a new connection. will return immediately (non-blocking mode)
                              if Sock<>SOCKET_ERROR then begin
                                    IOCtlSocket(Sock,FIONBIO,I); // make sure socket is nonblocking
                                    FSockets.Add(Ptr(Sock)); //if we got a new socket, add it to the subscriber list
                                    WriteLn(Format('Client ID %d connected',[Sock])); //write information message
                              end;
                        until Sock=SOCKET_ERROR;
                        while FQueue.Count>0 do begin //check for items queued to be sent
                              EnterCriticalSection(FLock); //retrieve item inside the critical section
                              try
                                    Packet:=FQueue[0];
                                    FQueue.Delete(0);
                              finally
                                    LeaveCriticalSection(FLock);
                              end;
                              for I:=FSockets.Count-1 downto 0 do
                                    if Winsock.Send(TSocket(FSockets[I]),Packet^,Packet.Length,0)<>Packet.Length then
                                          if WSAGetLastError=WSAEWOULDBLOCK then //send buffer full?
                                                Inc(FPacketsDropped) // discard packet
                                          else begin //any other socket error
                                                CloseSocket(TSocket(FSockets[I])); //close erraneous socket
                                                WriteLn(Format('Client ID %d disconnected',[TSocket(FSockets[I])])); //write information message
                                                FSockets.Delete(I); //and remove it from our list
                                          end
                                    else
                                          Inc(FPacketsSent);
                              FreeMem(Packet); //release packet memory
                        end;
                        for I:=0 to FSockets.Count-1 do
                              if Recv(TSocket(FSockets[I]),Header,SizeOf(Header),MSG_PEEK)=SizeOf(Header) then begin //incoming packet waiting?
                                    GetMem(Packet,Header.Length); //yes, then allocate memory for it
                                    if Recv(TSocket(FSockets[I]),Packet^,Header.Length,MSG_PEEK)=Header.Length then begin //try to peek complete packet
                                          Recv(TSocket(FSockets[I]),Packet^,Packet.Length,0); //successful, remove message from buffer
                                          SetString(S,PChar(@Packet.Data),Packet.Length-SizeOf(TPacket)); //copy data to a string
                                          WriteLn(Format('Client ID %d packet: %s',[TSocket(FSockets[I]),S])); //write information message
                                    end;
                                    FreeMem(Packet);
                              end;
                        Sleep(20); //all work done for now, wait some
                  until Terminated;
            finally
                  EnterCriticalSection(FLock);
                  CloseSocket(FListeningSocket); //close listening socket
                  for I:=0 to FSockets.Count-1 do //delete all subscribed sockets
                        CloseSocket(TSocket(FQueue[I]));
                  FSockets.Clear;
                  for I:=0 to FQueue.Count-1 do //delete all pending packets
                        FreeMem(FQueue[I]);
                  FQueue.Clear;
                  LeaveCriticalSection(FLock);
            end;
      except
            Halt(1); //on any error stop the whole program
      end;
end;

procedure TPacketSender.Send(var Data; Length: Cardinal);
var
      Packet: PPacket;
begin
      GetMem(Packet,Length+SizeOf(Packet^)); //allocate memory for packet
      Packet.Length:=Length+SizeOf(Packet^); //set length...
      Move(Data,Packet.Data,Length); //...and copy data
      EnterCriticalSection(FLock); //enter critical section
      try
            Inc(FSerial); //increase packet serial number
            Packet.Serial:=FSerial; //set serial number of packets, so that a client can detect dropped packets
            FQueue.Add(Packet); //add packet to send queue
      finally
            LeaveCriticalSection(FLock); //leave critical section
      end;
end;

var
      Listening: TPacketSender;
      S: string;

begin
      Listening:=TPacketSender.Create(1234);
      WriteLn('Empty line aborts.');
      repeat
            ReadLn(S);
            if S<>'' then
                  Listening.Send(Pointer(S)^,Length(S));
      until S='';
      Listening.Free;
end.



program TCPPacketClient;

{$APPTYPE CONSOLE}

uses
      Windows,  // basic OS stuff like sleep
      WinSock,  // socket calls
      SysUtils, // exception handling
      Classes;  // basic classes, threads

type
      TPacket=packed record
            Serial,Length: Longint;
            Data: packed record end;
      end;

var
      WSAData: TWSAData;
      ClientSocket: TSocket;
      S: string;
      Packet: TPacket;
      Addr: TSockAddrIn;

begin
      WSAStartup($101,WSAData);
      ClientSocket:=Socket(AF_INET,SOCK_STREAM,IPPROTO_TCP); // create new TCP socket, unbound and unconnected
      Addr.sin_family:=AF_INET; // fill up TSockAddrIn structure with address to bind to
      Addr.sin_port:=HToNS(1234); //port
      Addr.sin_addr.S_addr:=Inet_Addr('127.0.0.1'); //ip address (no host name!)
      if Connect(ClientSocket,Addr,SizeOf(Addr))=0 then
            while Recv(ClientSocket,Packet,SizeOf(Packet),0)=Sizeof(Packet) do begin //receive packet header
                  SetLength(S,Packet.Length-Sizeof(TPacket)); //allocate space for packet data
                  Recv(ClientSocket,Pointer(S)^,Length(S),0); //receive packet data
                  WriteLn(Packet.Serial:6,': ',S); //output packet on screen
                  S:='RESERVED'+Format('ACK %d',[Packet.Serial]); //compose packet to send back
                  Packet.Length:=Length(S);
                  Move(Packet,Pointer(S)^,SizeOf(TPacket)); //copy header into string 'RESERVED'
                  Send(ClientSocket,Pointer(S)^,Packet.Length,0); //send packet
            end
      else
            WriteLn('Connect Error');
      CloseSocket(ClientSocket);
      WSACleanup;
end.



I'm going to implement another client/server example using UDP soon, but please post new points for this.
0
 

Author Comment

by:DelFreak
Comment Utility
Okay!
0
 

Author Comment

by:DelFreak
Comment Utility
I forgot to ask, is TPacket a custom class? Also, is there any way not to use the Classes unit, specifically TList? The program I'm making uses Pure WinAPI and when I use the Classes unit I get a TPersistent error. What can I use in place of a TList? Thanks!
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
Look yourself in the code: TPacket is no class! It's only a record which holds two 32-bit values.

Instead of TList, you could use arrays. Specifically, you could work like this:

          FQueue: array of PPacket;
          FSockets: arrays of TSocket;

But you would have to change all code which uses these dynamic arrays. Note that TThread is also in CLASSES.PAS, so removing TList would not suffice to remove CLASSES from your uses list. However, I've never had problems with CLASSES.PAS and it does not blow up the program as the FORMS.PAS (and all TComponent descandants) do. I rather think that the TPersistent error you get is due to some faulty code and not because of the CLASSES unit as such.
0
 
LVL 1

Expert Comment

by:SenDog
Comment Utility
AvonWyss,
How does numbering the packets constitute to streaming? All your program does is send data as any other socket enabled program does. Except of course, the packets in your program are numbered. I seriously doubt if any streaming of data is involved. Please correct me if I'm wrong with my observation.



Cheers,
SenDog
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
SenDog,

Not only a serial number is included, but also a size which allows the client to know how much data it can expect for this one packet. By defining both size and serial number of a data block, it allows the following szenario to happen:

* The client will know when packets were dropped (no matter if UDP or TCP was used for transport)

* The client could thus be enabled to re-request a certain packet if necessary

* Virtually any number of clients can connect to the server and will be served with data from then on to their disconnection

It does not matter what kind of data is sent by the server. It could be audio, video, or just text messages; but it's sent in data blocks with size information and also information to allow loss/drop detection. Of course, the way that this detection is done and handled will depend on the data transferred and cannot be answered by this generic example. But the technique shown here is broadcasting data in blocks to the connected clients and it's prepared to properly detect exceptional situations.

What else would you, SenDog, define as packet streaming, if not a mechanism like this?
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss,
Yes, my mistake. I thought it was initialized as a class. Anyway, my problems with TPersisitent has nothing to do with Forms.pas since I am using pure WinAPI to create my window. Whenever I use any object contained in Classes.pas, I get the TPersistent error. Faulty code is not a possibility.

Also, I see you and SenDog are having a discussion on the code you posted. :)

I have a few comments:

1. Will there be any problem if var Data in TPacket is declared as an Array of String instead of a Packed Record?

2. Also, reading your comments to SenDog's remark, maybe you could modify the code and include a new function that allows the Client to re-request a certain packet when neccessary.

3. How's the UDP example coming along?

Thanks!
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
1. This will not work. Long strings as well as dynamic arrays in Delphi are, in fact, nothing but pointers. If you send a pointer of any kind along to the client, he will not be able to use it; the pointer is only valid in it's original process and address space. If you want to send a list of strings, I'd suggest using a mechanism similar to the TStringList.Text or TStringList.CommaText properties which return one string holding the data of all substrings which can then be reverted to the original set of strings.

2. Since TCP already is reliable and checks for losses, there is no need to implement a resend factory. In UDP, there will be a buffer for old data packets.

3. How are my points coming along?
0
 

Author Comment

by:DelFreak
Comment Utility
1. But using a StringList will need Classes.pas which will cause errors in the test program I am making. Any other suggestions?

2. Yes, TCP is quite reliable and I doubt there will be any problems with regards to data transmission. I just came up with the idea of a resend function since you pointed it out to SenDog. Is it feasible that there could be a possibility for such a function for TCP?

3. Well, I plan to add 100 more which will make it 200 for the TCP example and 200 for the UDP example. Will this suffice?
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
1. You can use whatever you want or need; I took the TStringList as example how the problem could be solved. Of course, any other sourcecode mixing several strings into a single data chunk will do the job just fine.

2. My latest code has support to send messages to the server from the client. By this, the client could send something like "RESEND xxx" and the server could understand this and send the requested packet. The principe is very easy; the only thing missing to impement this quickly in my code is a list of previously sent packets (the packets are currently discarded by the server after having been sent).

3. Look, I already made three well-documented complete programs to provide you with what you asked for. That's already WAY beyond what EE defines as one question (even if it has 300 points on it). Please consider this before cutting low on points.
0
 

Author Comment

by:DelFreak
Comment Utility
How much more points do you think is fair?
0
 

Author Comment

by:DelFreak
Comment Utility
Hi again AvonWyss! Okay, I will add 300 points more for the UDP example. That will make it 300 for the TCP example and 300 for the UDP example. I think this will be fair enough. Please note that the first code you posted was not the code I was looking for since it did not implement streaming and should not be counted. Anyway, I am not cutting low on points. I appreciate your helping me out. Thank you! If you think that 600 points is not enough, please tell me how many points will be fair. Thank you again!
0
 

Author Comment

by:DelFreak
Comment Utility
Also, seeing as this thread is getting long, you may simply send me the UDP code when it is finished. My e-mail address is: delfreak@thedoghousemail.com

Again, thank you!
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
Your proposition sounds fair, if you also give me A grades of course. In this case, accept my comment as answer to this Q now and post a new Q for the UDP sample (which will btw. also solve the thread length issue).
0
 

Author Comment

by:DelFreak
Comment Utility
AvonWyss,
A new question has been posted for the UDP Streaming example.

Anyway, will there be any problem with this:

type
  TPacket = packed record
    Serial, Length: LongInt;
    Data: packed record
            Command: Integer;
            P1, P2, P3, P4, P5: String;
          end;
  end;
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
This will not work either; (long) strings are pointers in Delphi! You could use short strings (declared like this: string[63] for a 63-char long string) but these always take theit length+1 bytes of memory, no matter how much of it is effectively used. Plus their max length is 255 chars, so that I'd avoid them; in your example, 5 strings with 255 chars would use up 1280 bytes which would be transferred - even if all of them are empty.

Btw, I'd suggest not to modify the TPacket record itself but to make another record which you send via the Send method, it will make maintenance easier for you and avoid problems because of different TPacket header sizes.

Please grade this question.
0
 

Author Comment

by:DelFreak
Comment Utility
Okay. You get an A! also, can you please post an example of a new record I couls send with TPacket. The first part should be the command (like login, quit, etc.), and the rest should be parameters (like password, username, etc.). I'd like to have infinite numbers of parameters but if this is not possible, a minimum of 5 and a maximum of 10 will do. Thanks!
0
 

Author Comment

by:DelFreak
Comment Utility
Thank you! :)
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
DelFreak, I have shown how to send and receive strings, right? Using the two functions below you can ancode any array of strings into a single string and vice-versa.

type
     TStringArray=array of string;

function ArrayToString(const Strings: array of string): string;
var
     I,Pos,Len: Integer;
begin
     SetLength(Result,4);
     Len:=Length(Strings);
     Move(Len,Result[1],4);
     Pos:=5;
     for I:=0 to Length(Strings)-1 do begin
          Len:=Length(Strings[I]);
          SetLength(Result,Pos+3+Len);
          Move(Len,Result[Pos],4);
          Move(Strings[I][1],Result[Pos+4],Len);
          Inc(Pos,Len+4);
     end;
end;

function StringToArray(const AString: string): TStringArray;
var
     I,Pos,Len: Integer;
begin
     Move(AString[1],Len,4);
     SetLength(Result,Len);
     Pos:=5;
     for I:=0 to Length(Result)-1 do begin
          Move(AString[Pos],Len,4);
          SetLength(Result[I],Len);
          Move(AString[Pos+4],Result[I][1],Len);
          Inc(Pos,Len+4);
     end;
end;

This will allow you to send as many string parameters as you like in a packet without using a TStringList object.
0
 
LVL 14

Expert Comment

by:AvonWyss
Comment Utility
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
Introduction I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need. This article will address some of the concepts when dealing with a multithreaded delphi database…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now