Solved

Socket data loss

Posted on 2004-09-13
18
582 Views
Last Modified: 2010-04-15
Hello,

i got a standard socket connection ( Server - Client ).
But when is send data from the Server to the client or the other way every tenth message got some missing parts at the end or at the beginning, for example:

Server sends:
hello hello hello hello hello (10 times)

the client receives:
...
hello hello hello hello hello
llo hello hello hello hello
hello hello hello hello hello
...

Both, the server and the client are Thread based (Client waiting for messages [while], Server waiting for connections [while] and waiting for messages [while])

i was able to decrease the loss of data by putting Thread.Sleep(30) behind every send, but the problem isnt completely gone then / and it gets too slow.

My Server sending function :
public void Send(Socket sock, string text)
{
      Byte[] sending = new Byte[1024000];
      sending = Encoding.ASCII.GetBytes(text);
      sock.Send(sending, sending.Length, 0);
}

My Client receiving function :
public string Receive()
{
      if(Socket != null)
      {
            byte[] receivdat = new byte[1024000];
            int receivingint = 1024;
            int receivingval = receivingint;
            int receivingall = 0;
            while(receivingint == receivingval)
            {
                  receivingval = Socket.Receive(receivdat, receivingint, 0);
                  receivingall += receivingval;
            }
                  
            string message = System.Text.Encoding.ASCII.GetString(receivdat).Substring(0, receivingall);
            return message;
      }
      return "";
}

Thanks for the help
0
Comment
Question by:roflkind
  • 8
  • 6
  • 4
18 Comments
 
LVL 20

Expert Comment

by:TheAvenger
ID: 12042386
Change the sending to:

sock.Send(sending, text.Length, 0);

Otherwise you send too much data that is not needed
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042398
Are you using a TCP socket? I assume so.
Note that you method cannot work for several reasons:
- Your Socket.Receive() call will always read the data on the start of your buffer, therefore overwriting already received data.
- The receive() call may read any number of bytes, it will not always read the full buffer size
- Due to the nature of the network, you must not rely on the assumption that you'll receive the data in the same blocks as you sent it! TCP is a data stream. If you send, for instance, 1000 bytes, and on the other side do a receive, it may well be that the network has not yet transmitted the full 1000 bytes, therefore only returning a part of it. Also, it my be that the client sent 2*1000 bytes and the server receives 2000 in one call.
- With the while() loop, even if you repect the information given above, you'll not succeed because the loop will wait until the socket has beenclosed before returning. You need to insert and recognize end-of-data-blocks yourself.
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042410
TheAvenger, your comment is not right. Note that the sending array is replaced by the assginment, therefore its length will be appropriate (even more appropriate than the length of the text, which may differ depending on the encoding used and the contents of the string).
0
 
LVL 20

Expert Comment

by:TheAvenger
ID: 12042432
Right, sorry, my mistake. Btw your comments are correct but this is not the reason for the problem. Actually the reason is:

string message = System.Text.Encoding.ASCII.GetString(receivdat).Substring(0, receivingall);

should be:

string message = System.Text.Encoding.ASCII.GetString(receivdat).Substring(0, receivingall - 1);

because receivingall contains also the last byte, which is '0'. And the comments you mentioned are not the reason, because I suppose this is a test on a single machina or at least two machines in the same LAN, the data is about 50 bytes and will practically always be transmitted at once.
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042445
Thanks for your comment AvonWyss.

Sorry i forgot to mention, that i am receiving my data like this:

string memory = "";

while(Connection.GetInstance().Socket != null)
{                                                
string receivstr = memory + Connection.GetInstance().Receive();
string[] progas = receivstr.Split(Convert.ToChar(";"));
memory = progas[progas.Length-1];

for(int p = 0; p < progas.Length-1; p++)
{
}
}

I think this is what you actually ment, but it still doesnt work.
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042461
TheAvenger, by default sockets (even on the lan and on the local machine) do some caching to avoid sending too many data blocks. You can control this feature with the sockets options when creating a new socket. Anyways, it is NEVER EVER correct to assume that the data you send via TCP will arrive in the same chunks at is was sent, an it will ALWAYS fail under some circumstances which you cannot control.

To do this properly, you must have some protocol on top of TCP to control the data flow. If you transmit text data without line breaks, you can use the .NET TextReader and TextWriter classes (that is, a StreamReader and StreamWriter on a NetworkStream). If not, you can just send an int over the line (4 bytes, use BitConverter class) and then you know how many bytes to read until the full data block has been received.
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042473
roflkind, the problem with the code you posted now is that you don't get all the data at the client, right?
0
 
LVL 20

Expert Comment

by:TheAvenger
ID: 12042480
AvonWyss, you missed one word in my message: PRACTICALLY. I've worked somewhat with sockets and for small amounts of data like 50 bytes at least 99.99% of the cases would be to receive the data at once, no breaking, no caching, no nothing. Having in mind that roflkind asks a question in EE, this means he had the problem not once and not twice. So it cannot be every time from breaking/caching. I repeat once again: PRACTICALLY for these tests only. For all productive or more intensive testing cases you are absolutely right.
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042516
Well, to my experience, if you send two 50 byte blocks without any delay (e.g. in a tight loop), you'll already receve 100 bytes - if the receiving buffer is that large enough, of course. If you use a 50 byte receiving buffer, it will look as if this works. However, this will still fail eventually if much data is processed or if you go on a slow networks. So, even if it does work in 99.99%, it is somethingf that must be avoided! You must not make code based on assumptions that are known to be wrong (and I think that roflkind did not yet know that his assumption was wrong, therefore I tried to make this very clear).

It's just as if you drive your car and don't look right or left on a crossing where there is very little traffic: you'll pass that crossing many many times without an accident, and suddently, you'll have a crash...
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 20

Expert Comment

by:TheAvenger
ID: 12042534
First steps are always made without the complexity of including every detail. However I don't want to argue on techniques of programming, so I am getting out of this question.
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042536
AvonWyss : thats exactly the problem, it doenst matter whether i use the code posted or not it completely screws up the beginning of my messages, the faster the more messages get lost

The Avenger : i tried your solution
string message = System.Text.Encoding.ASCII.GetString(receivdat).Substring(0, receivingall - 1);
but then it completely doesnt work, i am trying to find out what part of the message is missing when i try this code
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042631
So AvonWyss what do you think i should do, to get rid of this problem ?
0
 
LVL 14

Accepted Solution

by:
AvonWyss earned 250 total points
ID: 12042633
roflkind, this code may help:

      public static void SendBlock(Socket socket, byte[] data) {
            socket.Send(BitConverter.GetBytes(data.Length));
            socket.Send(data);
      }
      
      private static void ReceiveBytesBlocking(Socket socket, byte[] data) {
            for (int position=socket.Receive(data); position<data.Length; position+=socket.Receive(data, position, data.Length-position, SocketFlags.None));
      }

      public static byte[] ReceiveBlock(Socket socket) {
            byte[] length=new byte[4];
            ReceiveBytesBlocking(socket, length);
            byte[] result=new byte[BitConverter.ToInt32(length, 0)];
            ReceiveBytesBlocking(socket, result);
            return result;
      }

Now, in your code, just do this:

Server sending function:
public void Send(Socket sock, string text)
{
     SendBlock(sock, Encoding.ASCII.GetBytes(text));
}

Client receiving function:
public string Receive()
{
     return (Socket != null) ? Encoding.ASCII.GetString(ReceiveBlock(sock)) : "";
}

Hope this helps!
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042686
what the hell,
i cant believe this.
its working, its working !!

you dont know how happy i am now :D
i have been working on this problem for, i think 3 month now ? i tried everything posted here in the forums and stuff...
but now its working :D
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042699
thanks TheAvenger and AvonWyss for participating in this thread
especially AvonWyss who posted the right code
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042711
I'm glad it works now! Seems like I'm not so wrong in that case, eh? ;-)

Do you understand the code I posted or are there things you'd like me to comment on?
0
 
LVL 1

Author Comment

by:roflkind
ID: 12042779
no completely not :)
yes i understood it, i googled a bit and now i totally understand everything, its completely mindblowing

thanks again avonwyss
0
 
LVL 14

Expert Comment

by:AvonWyss
ID: 12042786
Ok, great! Have fun! :-)
0

Featured Post

Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

Join & Write a Comment

In order to hide the "ugly" records selectors (triangles) in the rowheaders, here are some suggestions. Microsoft doesn't have a direct method/property to do it. You can only hide the rowheader column. First solution, the easy way The first sol…
Entity Framework is a powerful tool to help you interact with the DataBase but still doesn't help much when we have a Stored Procedure that returns more than one resultset. The solution takes some of out-of-the-box thinking; read on!
In this tutorial you'll learn about bandwidth monitoring with flows and packet sniffing with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're interested in additional methods for monitoring bandwidt…
This video shows how to remove a single email address from the Outlook 2010 Auto Suggestion memory. NOTE: For Outlook 2016 and 2013 perform the exact same steps. Open a new email: Click the New email button in Outlook. Start typing the address: …

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now