Solved

Socket data loss

Posted on 2004-09-13
18
586 Views
Last Modified: 2010-04-15
Hello,

i got a standard socket connection ( Server - Client ).
But when is send data from the Server to the client or the other way every tenth message got some missing parts at the end or at the beginning, for example:

Server sends:
hello hello hello hello hello (10 times)

the client receives:
...
hello hello hello hello hello
llo hello hello hello hello
hello hello hello hello hello
...

Both, the server and the client are Thread based (Client waiting for messages [while], Server waiting for connections [while] and waiting for messages [while])

i was able to decrease the loss of data by putting Thread.Sleep(30) behind every send, but the problem isnt completely gone then / and it gets too slow.

My Server sending function :
public void Send(Socket sock, string text)
{
      Byte[] sending = new Byte[1024000];
      sending = Encoding.ASCII.GetBytes(text);
      sock.Send(sending, sending.Length, 0);
}

My Client receiving function :
public string Receive()
{
      if(Socket != null)
      {
            byte[] receivdat = new byte[1024000];
            int receivingint = 1024;
            int receivingval = receivingint;
            int receivingall = 0;
            while(receivingint == receivingval)
            {
                  receivingval = Socket.Receive(receivdat, receivingint, 0);
                  receivingall += receivingval;
            }
                  
            string message = System.Text.Encoding.ASCII.GetString(receivdat).Substring(0, receivingall);
            return message;
      }
      return "";
}

Thanks for the help
0
Comment
Question by:roflkind
  • 8
  • 6
  • 4
18 Comments
 
LVL 20

Expert Comment

by:TheAvenger
ID: 12042386
Change the sending to:

sock.Send(sending, text.Length, 0);

Otherwise you send too much data that is not needed
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042398
Are you using a TCP socket? I assume so.
Note that you method cannot work for several reasons:
- Your Socket.Receive() call will always read the data on the start of your buffer, therefore overwriting already received data.
- The receive() call may read any number of bytes, it will not always read the full buffer size
- Due to the nature of the network, you must not rely on the assumption that you'll receive the data in the same blocks as you sent it! TCP is a data stream. If you send, for instance, 1000 bytes, and on the other side do a receive, it may well be that the network has not yet transmitted the full 1000 bytes, therefore only returning a part of it. Also, it my be that the client sent 2*1000 bytes and the server receives 2000 in one call.
- With the while() loop, even if you repect the information given above, you'll not succeed because the loop will wait until the socket has beenclosed before returning. You need to insert and recognize end-of-data-blocks yourself.
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042410
TheAvenger, your comment is not right. Note that the sending array is replaced by the assginment, therefore its length will be appropriate (even more appropriate than the length of the text, which may differ depending on the encoding used and the contents of the string).
0
Complete VMware vSphere® ESX(i) & Hyper-V Backup

Capture your entire system, including the host, with patented disk imaging integrated with VMware VADP / Microsoft VSS and RCT. RTOs is as low as 15 seconds with Acronis Active Restore™. You can enjoy unlimited P2V/V2V migrations from any source (even from a different hypervisor)

 
LVL 20

Expert Comment

by:TheAvenger
ID: 12042432
Right, sorry, my mistake. Btw your comments are correct but this is not the reason for the problem. Actually the reason is:

string message = System.Text.Encoding.ASCII.GetString(receivdat).Substring(0, receivingall);

should be:

string message = System.Text.Encoding.ASCII.GetString(receivdat).Substring(0, receivingall - 1);

because receivingall contains also the last byte, which is '0'. And the comments you mentioned are not the reason, because I suppose this is a test on a single machina or at least two machines in the same LAN, the data is about 50 bytes and will practically always be transmitted at once.
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042445
Thanks for your comment AvonWyss.

Sorry i forgot to mention, that i am receiving my data like this:

string memory = "";

while(Connection.GetInstance().Socket != null)
{                                                
string receivstr = memory + Connection.GetInstance().Receive();
string[] progas = receivstr.Split(Convert.ToChar(";"));
memory = progas[progas.Length-1];

for(int p = 0; p < progas.Length-1; p++)
{
}
}

I think this is what you actually ment, but it still doesnt work.
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042461
TheAvenger, by default sockets (even on the lan and on the local machine) do some caching to avoid sending too many data blocks. You can control this feature with the sockets options when creating a new socket. Anyways, it is NEVER EVER correct to assume that the data you send via TCP will arrive in the same chunks at is was sent, an it will ALWAYS fail under some circumstances which you cannot control.

To do this properly, you must have some protocol on top of TCP to control the data flow. If you transmit text data without line breaks, you can use the .NET TextReader and TextWriter classes (that is, a StreamReader and StreamWriter on a NetworkStream). If not, you can just send an int over the line (4 bytes, use BitConverter class) and then you know how many bytes to read until the full data block has been received.
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042473
roflkind, the problem with the code you posted now is that you don't get all the data at the client, right?
0
 
LVL 20

Expert Comment

by:TheAvenger
ID: 12042480
AvonWyss, you missed one word in my message: PRACTICALLY. I've worked somewhat with sockets and for small amounts of data like 50 bytes at least 99.99% of the cases would be to receive the data at once, no breaking, no caching, no nothing. Having in mind that roflkind asks a question in EE, this means he had the problem not once and not twice. So it cannot be every time from breaking/caching. I repeat once again: PRACTICALLY for these tests only. For all productive or more intensive testing cases you are absolutely right.
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042516
Well, to my experience, if you send two 50 byte blocks without any delay (e.g. in a tight loop), you'll already receve 100 bytes - if the receiving buffer is that large enough, of course. If you use a 50 byte receiving buffer, it will look as if this works. However, this will still fail eventually if much data is processed or if you go on a slow networks. So, even if it does work in 99.99%, it is somethingf that must be avoided! You must not make code based on assumptions that are known to be wrong (and I think that roflkind did not yet know that his assumption was wrong, therefore I tried to make this very clear).

It's just as if you drive your car and don't look right or left on a crossing where there is very little traffic: you'll pass that crossing many many times without an accident, and suddently, you'll have a crash...
0
 
LVL 20

Expert Comment

by:TheAvenger
ID: 12042534
First steps are always made without the complexity of including every detail. However I don't want to argue on techniques of programming, so I am getting out of this question.
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042536
AvonWyss : thats exactly the problem, it doenst matter whether i use the code posted or not it completely screws up the beginning of my messages, the faster the more messages get lost

The Avenger : i tried your solution
string message = System.Text.Encoding.ASCII.GetString(receivdat).Substring(0, receivingall - 1);
but then it completely doesnt work, i am trying to find out what part of the message is missing when i try this code
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042631
So AvonWyss what do you think i should do, to get rid of this problem ?
0
 
LVL 14

Accepted Solution

by:
AvonWyss earned 250 total points
ID: 12042633
roflkind, this code may help:

      public static void SendBlock(Socket socket, byte[] data) {
            socket.Send(BitConverter.GetBytes(data.Length));
            socket.Send(data);
      }
      
      private static void ReceiveBytesBlocking(Socket socket, byte[] data) {
            for (int position=socket.Receive(data); position<data.Length; position+=socket.Receive(data, position, data.Length-position, SocketFlags.None));
      }

      public static byte[] ReceiveBlock(Socket socket) {
            byte[] length=new byte[4];
            ReceiveBytesBlocking(socket, length);
            byte[] result=new byte[BitConverter.ToInt32(length, 0)];
            ReceiveBytesBlocking(socket, result);
            return result;
      }

Now, in your code, just do this:

Server sending function:
public void Send(Socket sock, string text)
{
     SendBlock(sock, Encoding.ASCII.GetBytes(text));
}

Client receiving function:
public string Receive()
{
     return (Socket != null) ? Encoding.ASCII.GetString(ReceiveBlock(sock)) : "";
}

Hope this helps!
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042686
what the hell,
i cant believe this.
its working, its working !!

you dont know how happy i am now :D
i have been working on this problem for, i think 3 month now ? i tried everything posted here in the forums and stuff...
but now its working :D
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042699
thanks TheAvenger and AvonWyss for participating in this thread
especially AvonWyss who posted the right code
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042711
I'm glad it works now! Seems like I'm not so wrong in that case, eh? ;-)

Do you understand the code I posted or are there things you'd like me to comment on?
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042779
no completely not :)
yes i understood it, i googled a bit and now i totally understand everything, its completely mindblowing

thanks again avonwyss
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042786
Ok, great! Have fun! :-)
0

Featured Post

Is Your AD Toolbox Looking More Like a Toybox?

Managing Active Directory can get complicated.  Often, the native tools for managing AD are just not up to the task.  The largest Active Directory installations in the world have relied on one tool to manage their day-to-day administration tasks: Hyena. Start your trial today.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Summary: Persistence is the capability of an application to store the state of objects and recover it when necessary. This article compares the two common types of serialization in aspects of data access, readability, and runtime cost. A ready-to…
The article shows the basic steps of integrating an HTML theme template into an ASP.NET MVC project
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …

803 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question