DelFreak
asked on
UDP or TCP Streams
I am currently studying Sockets in Delphi and I'm having some trouble. Can anyone post source code that implements either TCP Streams or Reliable UDP Streams. The code must use raw Winsock functions. I would really appreciate it. Thanks!
DelPhreak, what exactoly is your problem? Include WINSOCK in your "uses" list, declare a socket handle of the type TSOCKET. Use SOCKET to allocate a socket handle and define what kind of socket you want (TCP is the stream protocol). Then connect to another host using CONNECT, send and receive data using SEND and RECV, and when you're done close the socket and free the handle using CLOSESOCKET.
That's pretty much it.
That's pretty much it.
ASKER
AvonWyss, I want to create a TCP packet stream and also a reliable UDP stream. Can you help me out?
DelFreak, there is no such thing as a reliable UDP stream. UDP is packet oriented and unreliable; if you need a stream, you have to use TCP which is reliable and streamed.
I don't have enough details about your project to give more help than what I have given above. Look up the functions I wrote doen in uppercase in the help file (In the start menu, Borland Delphi -> Help -> MS-SDK help files -> Windows Sockets 2, Index)
I don't have enough details about your project to give more help than what I have given above. Look up the functions I wrote doen in uppercase in the help file (In the start menu, Borland Delphi -> Help -> MS-SDK help files -> Windows Sockets 2, Index)
ASKER
AvonWyss, yes I know that UDP is an unreliable protocol. What I want to do create a TCP or UDP Packet Stream using raw Winsock functions. Can you help me out?
ASKER
I read about this in http://www.xiph.org/archives/vorbis/200007/0093.html
Maybe you can help me out. Thanks!
Maybe you can help me out. Thanks!
DelFreak, I read the URL you gave me. However, I don't see anything specific there; I still don't know what protocol you want to implement nor the exact purpose of your code.
ASKER
AvonWyss, I would like to implement a program that uses TCP or UDP packet streams. I hear it is very reliable especially when transferring large amounts of data. Any code that will give me an idea on how to implement packet streams will be fine. Thanks!
ASKER
Or any code that will allow me to check all outgoing UDP datagrams thus making the connection more reliable. Thanks!
ASKER
Please! I need help!
DelFreak, packet streams are NOT reliable. That's not what they are designed for. However, they are more flexible (in regards of adaptive data rate or multicasting), and they can scale better.
The calls to winsock functions above is all you need to send or receive data using both UDP or TCP. That's all you'll get from me for the points offered.
The calls to winsock functions above is all you need to send or receive data using both UDP or TCP. That's all you'll get from me for the points offered.
ASKER
Okay! I'll add more points. How many do you want for a complete source that implements UDP and TCP Packet Streams?
ASKER
Will 150 suffice? Just tell me if you want more and I'll add more points.
Do you need both a client and a server, or do you want to connect to an existing server? If you need a server, do you want it multithreaded? Do you have any special requirement for the protocol to be used? Give some more info, so that I'll be able to estimate the works to implement it.
ASKER
AvonWyss, Thank you for helping out. I am applying for a new job and it requires that I am skilled with sockets programming and both TCP and UDP protocols. The code you post will help me a lot. After this, I will post a question on tunneling. Would you know about that?
Anyway, I need it in C/S form and it must use raw Winsock functions. It is very important that it is fully commented as I need to learn how it works. A multi-threaded server will be great. No special protocol is required although I would appreciate it if you could make a different example for each of the following protocols:
1. UDP
2. TCP
3. RAW (If possible)
If you need more points, just tell me. I just really need your help. Thank you!
Anyway, I need it in C/S form and it must use raw Winsock functions. It is very important that it is fully commented as I need to learn how it works. A multi-threaded server will be great. No special protocol is required although I would appreciate it if you could make a different example for each of the following protocols:
1. UDP
2. TCP
3. RAW (If possible)
If you need more points, just tell me. I just really need your help. Thank you!
DelFreak, I think I get the idea. Note that RAW data exchange is not working on all stacks (actually, only Windows 2000 has full RAW support; the other Socket 2 implementations allow RAW sockets to be used for ICMP but not much more since not all header fields can be defined manually).
I'm gonna write some simple code for you, to start with a TCP echo server (that is, he sands back everything you send him). I take TCP in a first step because it easily allows testing using the TELNET application. When do you need that stuff to be done?
I'm gonna write some simple code for you, to start with a TCP echo server (that is, he sands back everything you send him). I take TCP in a first step because it easily allows testing using the TELNET application. When do you need that stuff to be done?
ASKER
Anytime soon. Yes, I read that only Win2K has full Raw Socket support. However, it will come in handy. Please don't forget that your example must use the Packet Streaming technique. I would appreciate an example for all 3 protocols but you may start with TCP if you wish. I'll add more points if you need them. Thank you.
AvonWyss
can you also post me the same example I am also
interested in this issue,
if you will post me I will grant you with more 100 points.
Thanks Eli
can you also post me the same example I am also
interested in this issue,
if you will post me I will grant you with more 100 points.
Thanks Eli
Ok, here comes the first part. A simple, multi-threaded echo server listening on port 1234. You can test it using telnet to connect to the machine running the service on port 1234.
I tried to keep it straightforward and clear to understand. However, by this, I left some things away. For instance, the code does assume that noe error ossurs in most socket operations. In a true server, this assumption must not be made; return codes of the WINSOCK functions have to be checked for error conditions.
I use both blocking and nonblocking modes, depending on what suits best for the given task. Note that, by defaulkt, sockets are created in blocking mode; however, sockets inherit their settings from their "parent" when they are created by the means of Accept. That's why I change the mode explicitly to blocking in the DataThread.
The app will run on the command line and has very little overhead.
program TCPServer;
{$APPTYPE CONSOLE}
uses
Windows, // basic OS stuff like sleep
WinSock, // socket calls
SysUtils, // exception handling
Classes; // basic classes, threads
type
TDataThread=class(TThread)
private
FSocket: TSocket;
public
constructor Create(ASocket: TSocket);
procedure Execute; override;
end;
TListeningThread=class(TTh read)
private
FSocket: TSocket;
FListeningPort: Word;
public
constructor Create(ListeningPort: Word);
procedure Execute; override;
end;
{ TListeningThread }
constructor TListeningThread.Create(Li steningPor t: Word);
begin
FListeningPort:=ListeningP ort;
inherited Create(False);
end;
procedure TListeningThread.Execute;
var
BindAddr: TSockAddrIn;
Sock: TSocket;
WSAData: TWSAData;
Arg: Longint;
begin
WSAStartup($101,WSAData); // initialize sockets library for this application
FSocket:=Socket(AF_INET,SO CK_STREAM, IPPROTO_TC P); // create new TCP socket, unbound and unconnected
Arg:=1;
IOCtlSocket(FSocket,FIONBI O,Arg); // set socket to nonblocking mode. this allows to check for Terminated
FillChar(BindAddr,SizeOf(B indAddr),0 );
BindAddr.sin_family:=AF_IN ET;
BindAddr.sin_port:=HToNS(F ListeningP ort);
BindAddr.sin_addr.S_addr:= INADDR_ANY ; // fill up TSockAddrIn structure with address to bind to
Bind(FSocket,BindAddr,Size Of(BindAdd r)); // bind the socket to the specified address and port
Listen(FSocket,5); // start listening on this socket
WriteLn('Listening start on port ',FListeningPort);
repeat
Sock:=Accept(FSocket,nil,n il); // try to accept a new connection. will return immediately (non-blocking mode)
if Sock<>SOCKET_ERROR then // did we get an incoming connection?
TDataThread.Create(Sock) // yes: create thread to handle it
else
Sleep(20); // no: we are idle, wait a little (-> will keep CPU usage low)
until Terminated; // loop only if the thread was not marked as being terminated
CloseSocket(FSocket); // release socket handle
WriteLn('Listening ended');
WSACleanup; // close socket library
end;
{ TDataThread }
constructor TDataThread.Create(ASocket : TSocket);
begin
FSocket:=ASocket;
WriteLn('Accepted socket ',FSocket);
inherited Create(False);
end;
procedure TDataThread.Execute;
var
Len,Arg: Integer;
Buf: packed array[0..1023] of Byte;
begin
Arg:=0;
IOCtlSocket(FSocket,FIONBI O,Arg); // set socket to blocking mode (calls will not return immediately)
repeat
Len:=Recv(FSocket,Buf,Size Of(Buf),0) ; // try to receive data. returns on data received or connection reset
if Len>0 then begin // did we get data?
Send(FSocket,Buf,Len,0); // yes: send it out (echo server)
WriteLn('Data transfer %d');
end else
Terminate; // no: mark thread to terminate
until Terminated; // loop only if the thread was not marked as being terminated
CloseSocket(FSocket); // release socket handle
WriteLn('Disconnected socket ',FSocket);
end;
var
Listening: TThread;
begin
Listening:=TListeningThrea d.Create(1 234);
WriteLn('Press RETURN to abort');
ReadLn;
Listening.Terminate;
Listening.WaitFor;
end.
I tried to keep it straightforward and clear to understand. However, by this, I left some things away. For instance, the code does assume that noe error ossurs in most socket operations. In a true server, this assumption must not be made; return codes of the WINSOCK functions have to be checked for error conditions.
I use both blocking and nonblocking modes, depending on what suits best for the given task. Note that, by defaulkt, sockets are created in blocking mode; however, sockets inherit their settings from their "parent" when they are created by the means of Accept. That's why I change the mode explicitly to blocking in the DataThread.
The app will run on the command line and has very little overhead.
program TCPServer;
{$APPTYPE CONSOLE}
uses
Windows, // basic OS stuff like sleep
WinSock, // socket calls
SysUtils, // exception handling
Classes; // basic classes, threads
type
TDataThread=class(TThread)
private
FSocket: TSocket;
public
constructor Create(ASocket: TSocket);
procedure Execute; override;
end;
TListeningThread=class(TTh
private
FSocket: TSocket;
FListeningPort: Word;
public
constructor Create(ListeningPort: Word);
procedure Execute; override;
end;
{ TListeningThread }
constructor TListeningThread.Create(Li
begin
FListeningPort:=ListeningP
inherited Create(False);
end;
procedure TListeningThread.Execute;
var
BindAddr: TSockAddrIn;
Sock: TSocket;
WSAData: TWSAData;
Arg: Longint;
begin
WSAStartup($101,WSAData); // initialize sockets library for this application
FSocket:=Socket(AF_INET,SO
Arg:=1;
IOCtlSocket(FSocket,FIONBI
FillChar(BindAddr,SizeOf(B
BindAddr.sin_family:=AF_IN
BindAddr.sin_port:=HToNS(F
BindAddr.sin_addr.S_addr:=
Bind(FSocket,BindAddr,Size
Listen(FSocket,5); // start listening on this socket
WriteLn('Listening start on port ',FListeningPort);
repeat
Sock:=Accept(FSocket,nil,n
if Sock<>SOCKET_ERROR then // did we get an incoming connection?
TDataThread.Create(Sock) // yes: create thread to handle it
else
Sleep(20); // no: we are idle, wait a little (-> will keep CPU usage low)
until Terminated; // loop only if the thread was not marked as being terminated
CloseSocket(FSocket); // release socket handle
WriteLn('Listening ended');
WSACleanup; // close socket library
end;
{ TDataThread }
constructor TDataThread.Create(ASocket
begin
FSocket:=ASocket;
WriteLn('Accepted socket ',FSocket);
inherited Create(False);
end;
procedure TDataThread.Execute;
var
Len,Arg: Integer;
Buf: packed array[0..1023] of Byte;
begin
Arg:=0;
IOCtlSocket(FSocket,FIONBI
repeat
Len:=Recv(FSocket,Buf,Size
if Len>0 then begin // did we get data?
Send(FSocket,Buf,Len,0); // yes: send it out (echo server)
WriteLn('Data transfer %d');
end else
Terminate; // no: mark thread to terminate
until Terminated; // loop only if the thread was not marked as being terminated
CloseSocket(FSocket); // release socket handle
WriteLn('Disconnected socket ',FSocket);
end;
var
Listening: TThread;
begin
Listening:=TListeningThrea
WriteLn('Press RETURN to abort');
ReadLn;
Listening.Terminate;
Listening.WaitFor;
end.
ASKER
AvonWyss, Thank you! A few Q's though. Can you specify which part of the code you posted describes Packet Streaming? Why does DataThread use Synchronus mode? Also, maybe you could post a full server with error checking and if possible in Asynchronus mode. Thanks again!
ASKER
I realize that I asked for three examples and 150 points is too small for my request. I have raised the points to 300. I hope that this will suffice. If more is needed, please tell me. Thank you!
ASKER
Also, maybe you could create a client so I don't have to use Telnet. It will help me understand things better. Thanks!
OK, here are the answers to your Q's:
* As said, this is NOT packet streaming, but a small echo server. It's purpose is NOT to stream data, but to show you how socket calls work without having many thousands of lines for code handling packets, error conditions, timeouts, file I/O etc. This code will listen on a TCP port, when a connection comes in it will echo the data received, and it will correctly shutdown. That's it.
* You asked for a multithreaded server. The classigc multithread server uses blocking calls and opens a thread for each data connection. Async mode in mutlithreading environment gives a huge management overhead (thread pooling, event and task sheduling, data synchronizing), so that it cannot be implemented in such ways that you can learn from it without the necessary knowledge of the single parts. So this would not have helped you. Look, I do have the code for that, I use it in server applications I write; however, this code belongs to the firm I work for and therefore I'm not allowed to post it here, and it does have thousands of lines which would not help you understand the issues at all because you wouldn't know what part of the code does what. Read this for more information on blocking/nonblocking operation:
http://www.cyberport.com/~tangent/programming/winsock/articles/io-strategies.html
* Why would it help better to use another client than telnet? For the server provided, telnet is the optimal testing suite; you can set breakpoints in your server code and step through the code, and you can test this echo server completely. Of coursem when it comes to UDP, another test program will be needed. And it may be a good excercise to make a client yourself, it's even easier than the server (no multithreading necessary, only one socket to manage).
* As said, this is NOT packet streaming, but a small echo server. It's purpose is NOT to stream data, but to show you how socket calls work without having many thousands of lines for code handling packets, error conditions, timeouts, file I/O etc. This code will listen on a TCP port, when a connection comes in it will echo the data received, and it will correctly shutdown. That's it.
* You asked for a multithreaded server. The classigc multithread server uses blocking calls and opens a thread for each data connection. Async mode in mutlithreading environment gives a huge management overhead (thread pooling, event and task sheduling, data synchronizing), so that it cannot be implemented in such ways that you can learn from it without the necessary knowledge of the single parts. So this would not have helped you. Look, I do have the code for that, I use it in server applications I write; however, this code belongs to the firm I work for and therefore I'm not allowed to post it here, and it does have thousands of lines which would not help you understand the issues at all because you wouldn't know what part of the code does what. Read this for more information on blocking/nonblocking operation:
http://www.cyberport.com/~tangent/programming/winsock/articles/io-strategies.html
* Why would it help better to use another client than telnet? For the server provided, telnet is the optimal testing suite; you can set breakpoints in your server code and step through the code, and you can test this echo server completely. Of coursem when it comes to UDP, another test program will be needed. And it may be a good excercise to make a client yourself, it's even easier than the server (no multithreading necessary, only one socket to manage).
ASKER
Oh! But I wanted an example of Packet Streaming. that was the main purpose of this question. TCP/UDP sockets, Asynchronus, Synchronus, no problem because i am quite familiar with it. Packet Streaming is what I want to learn about. Will you be able to make an example for me?
Well, initially you asked: "Can anyone post source code that implements either TCP Streams or Reliable UDP Streams. The code must use raw Winsock functions." I posted code that implements a TCP stream echo server, using raw Winsock functions. Because of this, I concluded that you were not familiar with the use of raw Winsock calls, and that's why I made a basic yet working Winsock server app.
What do you want to go into details? I can write you some packet transmission code, but this doens't have much to do with your first question.
What do you want to go into details? I can write you some packet transmission code, but this doens't have much to do with your first question.
ASKER
AvonWyss, I'm sorry for the misunderstanding. When I posted this question I assumed that TCP/UDP Streams reffered to Packet Streaming in general. My mistake. Anyway, I would like to learn about TCP and/or UDP Packet Streaming. If you can, please post an example project that iplements this technique. The implementation must use raw Winsock functions since I don't like using VCL's. I have already raised the points to 300. If more is needed then please tell me.
ASKER
Can anyone please help me.
DelFreak, "Packet Streaming" is quite a loose expression. If you like, I'll implement a small broadcast client and server which sends packets (with any binary data) to clients connected. But give me some days, I have lots of other stuff going on currently.
ASKER
Hi! AvonWyss, Streaming meaning the client app reads the data coming in, and discards that data as newer data arrives. Something like Audio Streaming or any other data. Can you be able to give me an example? TCP and/or UDP will be fine depending on which protocol you're more familiar with. Please reply ASAP. Thanks!
Yes, something like this was in my mind. Since you asked for a reliable transport, I'm going to use TCP. If lossless transfer was not necessary (for Audio and Video there are methods to deal with losses, but that's definetely out of scope here). Coming soon.
ASKER
Thanks!
ASKER
Can you do it for UDP as well? It will really help me out. Also, please make the example in client/server architecture. Thanks!
Ok, here's a packet transmission server and client for the TCP protocol. If you would want to use UDP, it would be pretty much the same idea but the client would have to send acknowledgments to the server (for reliable transmission), making the whole thing larger and less easy to understand. I tried to make the source code such that you can easily focus on the main aspects of WinSock and packets.
If a client is too slow reading data, the server will skip data, but because of TCP and the serial number included in each packet it's easy to detect this by the client. The client I wrote does not provide a feedback other than the TCP ack packets to the server.
Here you go:
program TCPPacketServer;
{$APPTYPE CONSOLE}
uses
Windows, // basic OS stuff like sleep
WinSock, // socket calls
SysUtils, // exception handling
Classes; // basic classes, threads
type
TPacket=packed record
Serial,Length: Longint;
Data: packed record end;
end;
PPacket=^TPacket;
TPacketSender=class(TThrea d)
private
FQueue,FSockets: TList;
FLock: TRTLCriticalSection;
FListeningPort: Word;
FListeningSocket: TSocket;
FSerial,FPacketsSent,FPack etsDropped : Integer;
protected
procedure Execute; override;
public
constructor Create(ListeningPort: Word);
procedure Send(var Data; Length: Cardinal);
destructor Destroy; override;
end;
{ TPacketSender }
constructor TPacketSender.Create(Liste ningPort: Word);
var
WSAData: TWSAData;
begin
FQueue:=TList.Create;
FSockets:=TList.Create;
InitializeCriticalSection( FLock);
FListeningPort:=ListeningP ort;
WSAStartup($101,WSAData);
inherited Create(False);
end;
destructor TPacketSender.Destroy;
begin
Terminate;
WaitFor;
inherited;
FQueue.Free;
FSockets.Free;
DeleteCriticalSection(FLoc k);
WSACleanup;
end;
procedure TPacketSender.Execute;
var
I: Integer;
BindAddr: TSockAddrIn;
Sock: TSocket;
Packet: PPacket;
begin
FListeningSocket:=Socket(A F_INET,SOC K_STREAM,I PPROTO_TCP ); // create new TCP socket, unbound and unconnected
try
I:=1;
IOCtlSocket(FListeningSock et,FIONBIO ,I); // set socket to nonblocking mode. this allows to check for Terminated
FillChar(BindAddr,SizeOf(B indAddr),0 );
BindAddr.sin_family:=AF_IN ET;
BindAddr.sin_port:=HToNS(F ListeningP ort);
BindAddr.sin_addr.S_addr:= INADDR_ANY ; // fill up TSockAddrIn structure with address to bind to
Bind(FListeningSocket,Bind Addr,SizeO f(BindAddr )); // bind the socket to the specified address and port
if Listen(FListeningSocket,5) <>0 then // start listening on this socket
Abort; //error when listening on this socket
repeat
repeat
Sock:=Accept(FListeningSoc ket,nil,ni l); // try to accept a new connection. will return immediately (non-blocking mode)
if Sock<>SOCKET_ERROR then
FSockets.Add(Ptr(Sock)); //if we got a new socket, add it to the subscriber list
until Sock=SOCKET_ERROR;
while FQueue.Count>0 do begin //check for items queued to be sent
EnterCriticalSection(FLock ); //retrieve item inside the critical section
try
Packet:=FQueue[0];
FQueue.Delete(0);
finally
LeaveCriticalSection(FLock );
end;
for I:=0 to FSockets.Count-1 do
if Winsock.Send(Integer(FSock ets[I]),Pa cket^,Pack et.Length, 0)<>Packet .Length then
if WSAGetLastError=WSAEWOULDB LOCK then //send buffer full?
Inc(FPacketsDropped) // discard packet
else begin //any other socket error
CloseSocket(Integer(FSocke ts[I])); //close erraneous socket
FSockets.Delete(I); //and remove it from our list
end
else
Inc(FPacketsSent);
end;
Sleep(20); //all work done for now, wait some
until Terminated;
finally
EnterCriticalSection(FLock );
CloseSocket(FListeningSock et); //close listening socket
for I:=0 to FSockets.Count-1 do //delete all subscribed sockets
CloseSocket(TSocket(FQueue [I]));
FSockets.Clear;
for I:=0 to FQueue.Count-1 do //delete all pending packets
FreeMem(FQueue[I]);
FQueue.Clear;
LeaveCriticalSection(FLock );
end;
end;
procedure TPacketSender.Send(var Data; Length: Cardinal);
var
Packet: PPacket;
begin
GetMem(Packet,Length+SizeO f(Packet^) ); //allocate memory for packet
Packet.Length:=Length+Size Of(Packet^ ); //set length...
Move(Data,Packet.Data,Leng th); //...and copy data
EnterCriticalSection(FLock ); //enter critical section
try
Inc(FSerial); //increase packet serial number
Packet.Serial:=FSerial; //set serial number of packets, so that a client can detect dropped packets
FQueue.Add(Packet); //add packet to send queue
finally
LeaveCriticalSection(FLock ); //leave critical section
end;
end;
var
Listening: TPacketSender;
S: string;
begin
Listening:=TPacketSender.C reate(1234 );
WriteLn('Empty line aborts.');
repeat
ReadLn(S);
if S<>'' then
Listening.Send(Pointer(S)^ ,Length(S) );
until S='';
Listening.Free;
end.
program TCPPacketClient;
{$APPTYPE CONSOLE}
uses
Windows, // basic OS stuff like sleep
WinSock, // socket calls
SysUtils, // exception handling
Classes; // basic classes, threads
type
TPacket=packed record
Serial,Length: Longint;
Data: packed record end;
end;
var
WSAData: TWSAData;
ClientSocket: TSocket;
S: string;
Packet: TPacket;
Addr: TSockAddrIn;
begin
WSAStartup($101,WSAData);
ClientSocket:=Socket(AF_IN ET,SOCK_ST REAM,IPPRO TO_TCP); // create new TCP socket, unbound and unconnected
Addr.sin_family:=AF_INET; // fill up TSockAddrIn structure with address to bind to
Addr.sin_port:=HToNS(1234) ; //port
Addr.sin_addr.S_addr:=Inet _Addr('127 .0.0.1'); //ip address (no host name!)
if Connect(ClientSocket,Addr, SizeOf(Add r))=0 then
while Recv(ClientSocket,Packet,S izeOf(Pack et),0)=Siz eof(Packet ) do begin
SetLength(S,Packet.Length- Sizeof(TPa cket));
Recv(ClientSocket,Pointer( S)^,Length (S),0);
WriteLn(Packet.Serial:6,': ',S);
end
else
WriteLn('Connect Error');
CloseSocket(ClientSocket);
WSACleanup;
end.
I believe that this should give you a good understanding and starting point for other developments, no matter if they are using TCP or UDP (which really just works mostly the same, but you may not make that many assumptions as you can with TCP, like havin all packets arrive and such).
If a client is too slow reading data, the server will skip data, but because of TCP and the serial number included in each packet it's easy to detect this by the client. The client I wrote does not provide a feedback other than the TCP ack packets to the server.
Here you go:
program TCPPacketServer;
{$APPTYPE CONSOLE}
uses
Windows, // basic OS stuff like sleep
WinSock, // socket calls
SysUtils, // exception handling
Classes; // basic classes, threads
type
TPacket=packed record
Serial,Length: Longint;
Data: packed record end;
end;
PPacket=^TPacket;
TPacketSender=class(TThrea
private
FQueue,FSockets: TList;
FLock: TRTLCriticalSection;
FListeningPort: Word;
FListeningSocket: TSocket;
FSerial,FPacketsSent,FPack
protected
procedure Execute; override;
public
constructor Create(ListeningPort: Word);
procedure Send(var Data; Length: Cardinal);
destructor Destroy; override;
end;
{ TPacketSender }
constructor TPacketSender.Create(Liste
var
WSAData: TWSAData;
begin
FQueue:=TList.Create;
FSockets:=TList.Create;
InitializeCriticalSection(
FListeningPort:=ListeningP
WSAStartup($101,WSAData);
inherited Create(False);
end;
destructor TPacketSender.Destroy;
begin
Terminate;
WaitFor;
inherited;
FQueue.Free;
FSockets.Free;
DeleteCriticalSection(FLoc
WSACleanup;
end;
procedure TPacketSender.Execute;
var
I: Integer;
BindAddr: TSockAddrIn;
Sock: TSocket;
Packet: PPacket;
begin
FListeningSocket:=Socket(A
try
I:=1;
IOCtlSocket(FListeningSock
FillChar(BindAddr,SizeOf(B
BindAddr.sin_family:=AF_IN
BindAddr.sin_port:=HToNS(F
BindAddr.sin_addr.S_addr:=
Bind(FListeningSocket,Bind
if Listen(FListeningSocket,5)
Abort; //error when listening on this socket
repeat
repeat
Sock:=Accept(FListeningSoc
if Sock<>SOCKET_ERROR then
FSockets.Add(Ptr(Sock)); //if we got a new socket, add it to the subscriber list
until Sock=SOCKET_ERROR;
while FQueue.Count>0 do begin //check for items queued to be sent
EnterCriticalSection(FLock
try
Packet:=FQueue[0];
FQueue.Delete(0);
finally
LeaveCriticalSection(FLock
end;
for I:=0 to FSockets.Count-1 do
if Winsock.Send(Integer(FSock
if WSAGetLastError=WSAEWOULDB
Inc(FPacketsDropped) // discard packet
else begin //any other socket error
CloseSocket(Integer(FSocke
FSockets.Delete(I); //and remove it from our list
end
else
Inc(FPacketsSent);
end;
Sleep(20); //all work done for now, wait some
until Terminated;
finally
EnterCriticalSection(FLock
CloseSocket(FListeningSock
for I:=0 to FSockets.Count-1 do //delete all subscribed sockets
CloseSocket(TSocket(FQueue
FSockets.Clear;
for I:=0 to FQueue.Count-1 do //delete all pending packets
FreeMem(FQueue[I]);
FQueue.Clear;
LeaveCriticalSection(FLock
end;
end;
procedure TPacketSender.Send(var Data; Length: Cardinal);
var
Packet: PPacket;
begin
GetMem(Packet,Length+SizeO
Packet.Length:=Length+Size
Move(Data,Packet.Data,Leng
EnterCriticalSection(FLock
try
Inc(FSerial); //increase packet serial number
Packet.Serial:=FSerial; //set serial number of packets, so that a client can detect dropped packets
FQueue.Add(Packet); //add packet to send queue
finally
LeaveCriticalSection(FLock
end;
end;
var
Listening: TPacketSender;
S: string;
begin
Listening:=TPacketSender.C
WriteLn('Empty line aborts.');
repeat
ReadLn(S);
if S<>'' then
Listening.Send(Pointer(S)^
until S='';
Listening.Free;
end.
program TCPPacketClient;
{$APPTYPE CONSOLE}
uses
Windows, // basic OS stuff like sleep
WinSock, // socket calls
SysUtils, // exception handling
Classes; // basic classes, threads
type
TPacket=packed record
Serial,Length: Longint;
Data: packed record end;
end;
var
WSAData: TWSAData;
ClientSocket: TSocket;
S: string;
Packet: TPacket;
Addr: TSockAddrIn;
begin
WSAStartup($101,WSAData);
ClientSocket:=Socket(AF_IN
Addr.sin_family:=AF_INET; // fill up TSockAddrIn structure with address to bind to
Addr.sin_port:=HToNS(1234)
Addr.sin_addr.S_addr:=Inet
if Connect(ClientSocket,Addr,
while Recv(ClientSocket,Packet,S
SetLength(S,Packet.Length-
Recv(ClientSocket,Pointer(
WriteLn(Packet.Serial:6,':
end
else
WriteLn('Connect Error');
CloseSocket(ClientSocket);
WSACleanup;
end.
I believe that this should give you a good understanding and starting point for other developments, no matter if they are using TCP or UDP (which really just works mostly the same, but you may not make that many assumptions as you can with TCP, like havin all packets arrive and such).
Oh yeah, fogot to mention how to use the programs:
* The server will (by default) listen on port 1234.
* Any number of clients may connect simultaneously.
* You can type in some text and hit ENTER to send it to the client. Note that an empty line will terminate the server.
* The client connects to a given IP address.
* The client will terminate if the connection is lost. Of course, in a real application receiving data in a separate thread, you can have any termination sequence you like.
* The client will write the received packet serial number and contents on the screen when it arrives.
* The server will (by default) listen on port 1234.
* Any number of clients may connect simultaneously.
* You can type in some text and hit ENTER to send it to the client. Note that an empty line will terminate the server.
* The client connects to a given IP address.
* The client will terminate if the connection is lost. Of course, in a real application receiving data in a separate thread, you can have any termination sequence you like.
* The client will write the received packet serial number and contents on the screen when it arrives.
ASKER
AvonWyss, the example is great. A few questions though:
1. Which function in the server does the actual streaming?
2. If I want the Client to send data to the server, do I just use the same functions of sending as the server?
3. How will the server process the packets? Like the Client does?
1. Which function in the server does the actual streaming?
2. If I want the Client to send data to the server, do I just use the same functions of sending as the server?
3. How will the server process the packets? Like the Client does?
1. The server has a thread which checks every now and then (that is, about 50 times per second) if there is data queued to be sent out. The code "while FQueue.Count>0 do begin" starts the block which checks and sends data. I made the data queue thread-safe, so that any thread can queue data packets at any time without access collisions (the code is so-called "thread-safe").
2. The client could send data back also with Send(), quite similar to the way data is sent by the server. However, to avoid impact on the stream, you should only send back small amounts of data (if any, that is).
3. The server does not check for incoming data at all. You'd have to add a third loop in the Execute which would check all connections for incoming data every now and then.
reuveni: I did post the code, are you going to post the points you offered?
2. The client could send data back also with Send(), quite similar to the way data is sent by the server. However, to avoid impact on the stream, you should only send back small amounts of data (if any, that is).
3. The server does not check for incoming data at all. You'd have to add a third loop in the Execute which would check all connections for incoming data every now and then.
reuveni: I did post the code, are you going to post the points you offered?
ASKER
AvonWyss, Could you modify the code so that the client could send data and the server could receive and process it and could you also post a UDP example. I'll add more points if neccesary. Also, do you know anything about TCP Tunneling? Can this be done in Delphi? Thanks!
BTW, I think reuveni already posted a Q containing your points.
BTW, I think reuveni already posted a Q containing your points.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Okay!
ASKER
I forgot to ask, is TPacket a custom class? Also, is there any way not to use the Classes unit, specifically TList? The program I'm making uses Pure WinAPI and when I use the Classes unit I get a TPersistent error. What can I use in place of a TList? Thanks!
Look yourself in the code: TPacket is no class! It's only a record which holds two 32-bit values.
Instead of TList, you could use arrays. Specifically, you could work like this:
FQueue: array of PPacket;
FSockets: arrays of TSocket;
But you would have to change all code which uses these dynamic arrays. Note that TThread is also in CLASSES.PAS, so removing TList would not suffice to remove CLASSES from your uses list. However, I've never had problems with CLASSES.PAS and it does not blow up the program as the FORMS.PAS (and all TComponent descandants) do. I rather think that the TPersistent error you get is due to some faulty code and not because of the CLASSES unit as such.
Instead of TList, you could use arrays. Specifically, you could work like this:
FQueue: array of PPacket;
FSockets: arrays of TSocket;
But you would have to change all code which uses these dynamic arrays. Note that TThread is also in CLASSES.PAS, so removing TList would not suffice to remove CLASSES from your uses list. However, I've never had problems with CLASSES.PAS and it does not blow up the program as the FORMS.PAS (and all TComponent descandants) do. I rather think that the TPersistent error you get is due to some faulty code and not because of the CLASSES unit as such.
AvonWyss,
How does numbering the packets constitute to streaming? All your program does is send data as any other socket enabled program does. Except of course, the packets in your program are numbered. I seriously doubt if any streaming of data is involved. Please correct me if I'm wrong with my observation.
Cheers,
SenDog
How does numbering the packets constitute to streaming? All your program does is send data as any other socket enabled program does. Except of course, the packets in your program are numbered. I seriously doubt if any streaming of data is involved. Please correct me if I'm wrong with my observation.
Cheers,
SenDog
SenDog,
Not only a serial number is included, but also a size which allows the client to know how much data it can expect for this one packet. By defining both size and serial number of a data block, it allows the following szenario to happen:
* The client will know when packets were dropped (no matter if UDP or TCP was used for transport)
* The client could thus be enabled to re-request a certain packet if necessary
* Virtually any number of clients can connect to the server and will be served with data from then on to their disconnection
It does not matter what kind of data is sent by the server. It could be audio, video, or just text messages; but it's sent in data blocks with size information and also information to allow loss/drop detection. Of course, the way that this detection is done and handled will depend on the data transferred and cannot be answered by this generic example. But the technique shown here is broadcasting data in blocks to the connected clients and it's prepared to properly detect exceptional situations.
What else would you, SenDog, define as packet streaming, if not a mechanism like this?
Not only a serial number is included, but also a size which allows the client to know how much data it can expect for this one packet. By defining both size and serial number of a data block, it allows the following szenario to happen:
* The client will know when packets were dropped (no matter if UDP or TCP was used for transport)
* The client could thus be enabled to re-request a certain packet if necessary
* Virtually any number of clients can connect to the server and will be served with data from then on to their disconnection
It does not matter what kind of data is sent by the server. It could be audio, video, or just text messages; but it's sent in data blocks with size information and also information to allow loss/drop detection. Of course, the way that this detection is done and handled will depend on the data transferred and cannot be answered by this generic example. But the technique shown here is broadcasting data in blocks to the connected clients and it's prepared to properly detect exceptional situations.
What else would you, SenDog, define as packet streaming, if not a mechanism like this?
ASKER
AvonWyss,
Yes, my mistake. I thought it was initialized as a class. Anyway, my problems with TPersisitent has nothing to do with Forms.pas since I am using pure WinAPI to create my window. Whenever I use any object contained in Classes.pas, I get the TPersistent error. Faulty code is not a possibility.
Also, I see you and SenDog are having a discussion on the code you posted. :)
I have a few comments:
1. Will there be any problem if var Data in TPacket is declared as an Array of String instead of a Packed Record?
2. Also, reading your comments to SenDog's remark, maybe you could modify the code and include a new function that allows the Client to re-request a certain packet when neccessary.
3. How's the UDP example coming along?
Thanks!
Yes, my mistake. I thought it was initialized as a class. Anyway, my problems with TPersisitent has nothing to do with Forms.pas since I am using pure WinAPI to create my window. Whenever I use any object contained in Classes.pas, I get the TPersistent error. Faulty code is not a possibility.
Also, I see you and SenDog are having a discussion on the code you posted. :)
I have a few comments:
1. Will there be any problem if var Data in TPacket is declared as an Array of String instead of a Packed Record?
2. Also, reading your comments to SenDog's remark, maybe you could modify the code and include a new function that allows the Client to re-request a certain packet when neccessary.
3. How's the UDP example coming along?
Thanks!
1. This will not work. Long strings as well as dynamic arrays in Delphi are, in fact, nothing but pointers. If you send a pointer of any kind along to the client, he will not be able to use it; the pointer is only valid in it's original process and address space. If you want to send a list of strings, I'd suggest using a mechanism similar to the TStringList.Text or TStringList.CommaText properties which return one string holding the data of all substrings which can then be reverted to the original set of strings.
2. Since TCP already is reliable and checks for losses, there is no need to implement a resend factory. In UDP, there will be a buffer for old data packets.
3. How are my points coming along?
2. Since TCP already is reliable and checks for losses, there is no need to implement a resend factory. In UDP, there will be a buffer for old data packets.
3. How are my points coming along?
ASKER
1. But using a StringList will need Classes.pas which will cause errors in the test program I am making. Any other suggestions?
2. Yes, TCP is quite reliable and I doubt there will be any problems with regards to data transmission. I just came up with the idea of a resend function since you pointed it out to SenDog. Is it feasible that there could be a possibility for such a function for TCP?
3. Well, I plan to add 100 more which will make it 200 for the TCP example and 200 for the UDP example. Will this suffice?
2. Yes, TCP is quite reliable and I doubt there will be any problems with regards to data transmission. I just came up with the idea of a resend function since you pointed it out to SenDog. Is it feasible that there could be a possibility for such a function for TCP?
3. Well, I plan to add 100 more which will make it 200 for the TCP example and 200 for the UDP example. Will this suffice?
1. You can use whatever you want or need; I took the TStringList as example how the problem could be solved. Of course, any other sourcecode mixing several strings into a single data chunk will do the job just fine.
2. My latest code has support to send messages to the server from the client. By this, the client could send something like "RESEND xxx" and the server could understand this and send the requested packet. The principe is very easy; the only thing missing to impement this quickly in my code is a list of previously sent packets (the packets are currently discarded by the server after having been sent).
3. Look, I already made three well-documented complete programs to provide you with what you asked for. That's already WAY beyond what EE defines as one question (even if it has 300 points on it). Please consider this before cutting low on points.
2. My latest code has support to send messages to the server from the client. By this, the client could send something like "RESEND xxx" and the server could understand this and send the requested packet. The principe is very easy; the only thing missing to impement this quickly in my code is a list of previously sent packets (the packets are currently discarded by the server after having been sent).
3. Look, I already made three well-documented complete programs to provide you with what you asked for. That's already WAY beyond what EE defines as one question (even if it has 300 points on it). Please consider this before cutting low on points.
ASKER
How much more points do you think is fair?
ASKER
Hi again AvonWyss! Okay, I will add 300 points more for the UDP example. That will make it 300 for the TCP example and 300 for the UDP example. I think this will be fair enough. Please note that the first code you posted was not the code I was looking for since it did not implement streaming and should not be counted. Anyway, I am not cutting low on points. I appreciate your helping me out. Thank you! If you think that 600 points is not enough, please tell me how many points will be fair. Thank you again!
ASKER
Also, seeing as this thread is getting long, you may simply send me the UDP code when it is finished. My e-mail address is: delfreak@thedoghousemail.c om
Again, thank you!
Again, thank you!
Your proposition sounds fair, if you also give me A grades of course. In this case, accept my comment as answer to this Q now and post a new Q for the UDP sample (which will btw. also solve the thread length issue).
ASKER
AvonWyss,
A new question has been posted for the UDP Streaming example.
Anyway, will there be any problem with this:
type
TPacket = packed record
Serial, Length: LongInt;
Data: packed record
Command: Integer;
P1, P2, P3, P4, P5: String;
end;
end;
A new question has been posted for the UDP Streaming example.
Anyway, will there be any problem with this:
type
TPacket = packed record
Serial, Length: LongInt;
Data: packed record
Command: Integer;
P1, P2, P3, P4, P5: String;
end;
end;
This will not work either; (long) strings are pointers in Delphi! You could use short strings (declared like this: string[63] for a 63-char long string) but these always take theit length+1 bytes of memory, no matter how much of it is effectively used. Plus their max length is 255 chars, so that I'd avoid them; in your example, 5 strings with 255 chars would use up 1280 bytes which would be transferred - even if all of them are empty.
Btw, I'd suggest not to modify the TPacket record itself but to make another record which you send via the Send method, it will make maintenance easier for you and avoid problems because of different TPacket header sizes.
Please grade this question.
Btw, I'd suggest not to modify the TPacket record itself but to make another record which you send via the Send method, it will make maintenance easier for you and avoid problems because of different TPacket header sizes.
Please grade this question.
ASKER
Okay. You get an A! also, can you please post an example of a new record I couls send with TPacket. The first part should be the command (like login, quit, etc.), and the rest should be parameters (like password, username, etc.). I'd like to have infinite numbers of parameters but if this is not possible, a minimum of 5 and a maximum of 10 will do. Thanks!
ASKER
Thank you! :)
DelFreak, I have shown how to send and receive strings, right? Using the two functions below you can ancode any array of strings into a single string and vice-versa.
type
TStringArray=array of string;
function ArrayToString(const Strings: array of string): string;
var
I,Pos,Len: Integer;
begin
SetLength(Result,4);
Len:=Length(Strings);
Move(Len,Result[1],4);
Pos:=5;
for I:=0 to Length(Strings)-1 do begin
Len:=Length(Strings[I]);
SetLength(Result,Pos+3+Len );
Move(Len,Result[Pos],4);
Move(Strings[I][1],Result[ Pos+4],Len );
Inc(Pos,Len+4);
end;
end;
function StringToArray(const AString: string): TStringArray;
var
I,Pos,Len: Integer;
begin
Move(AString[1],Len,4);
SetLength(Result,Len);
Pos:=5;
for I:=0 to Length(Result)-1 do begin
Move(AString[Pos],Len,4);
SetLength(Result[I],Len);
Move(AString[Pos+4],Result [I][1],Len );
Inc(Pos,Len+4);
end;
end;
This will allow you to send as many string parameters as you like in a packet without using a TStringList object.
type
TStringArray=array of string;
function ArrayToString(const Strings: array of string): string;
var
I,Pos,Len: Integer;
begin
SetLength(Result,4);
Len:=Length(Strings);
Move(Len,Result[1],4);
Pos:=5;
for I:=0 to Length(Strings)-1 do begin
Len:=Length(Strings[I]);
SetLength(Result,Pos+3+Len
Move(Len,Result[Pos],4);
Move(Strings[I][1],Result[
Inc(Pos,Len+4);
end;
end;
function StringToArray(const AString: string): TStringArray;
var
I,Pos,Len: Integer;
begin
Move(AString[1],Len,4);
SetLength(Result,Len);
Pos:=5;
for I:=0 to Length(Result)-1 do begin
Move(AString[Pos],Len,4);
SetLength(Result[I],Len);
Move(AString[Pos+4],Result
Inc(Pos,Len+4);
end;
end;
This will allow you to send as many string parameters as you like in a packet without using a TStringList object.
PAQ note: followup fur UDP is in https://www.experts-exchange.com/jsp/qShow.jsp?ta=delphi&qid=20144897
http://www.cyberport.com/~tangent/programming/winsock/