Solved

Data is truncated when receiving over TCP: VB.NET Windows Service

Posted on 2010-11-16
18
1,363 Views
Last Modified: 2015-05-03
I'm receiving a packet of information from a 3rd party service.  Sometimes, the data comes across truncated.  This is running on .NET 3.5 as a Windows Service.  The OS is Windows Server 2003 and is running on a virtual machine.  
Any ideas?  Note: I cannot run it on a physical machine at this point.
Imports System.Net.Sockets

Imports System.Net

Imports System.Text

Imports System.Threading

Imports System.IO

Imports Microsoft.VisualBasic

Imports System.IO.File



Public Class Service1

    Public t1 As New Thread(AddressOf Listen)

    Public canStopListening As Boolean = False

    Public pause As Boolean = False



    Protected Overrides Sub OnStart(ByVal args() As String)

        t1.Start()

    End Sub



    Private Sub Listen()

        Const portNumber As Integer = 6030

        Dim tcpListener As New TcpListener(portNumber)

        tcpListener.Start()

        Do

            Dim tcpClient As TcpClient = tcpListener.AcceptTcpClient()

            Dim networkStream As NetworkStream = tcpClient.GetStream()

            Dim bytes(tcpClient.ReceiveBufferSize) As Byte

            networkStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))

            Dim clientdata As String = Encoding.ASCII.GetString(bytes)



            'At this point we have the data, but sometimes it's truncated. Why?

           

            Dim responseString As String = ""

            'Create Document

            responseString = WriteRTF(clientdata)



            Dim sendBytes As [Byte]() = Encoding.ASCII.GetBytes(responseString)

            networkStream.Write(sendBytes, 0, sendBytes.Length)

            tcpClient.Close()

        Loop Until canStopListening

        tcpListener.Stop()

    End Sub

End Class

Open in new window

0
Comment
Question by:robnhood00
  • 11
  • 6
18 Comments
 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 34151900
Can you put the read inside a loop and keep reading until there is no more data?
 Do            
    Do            
       i = networkStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))
    Loop Until i = 0
 Loop Until canStopListening
0
 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 34487271
Still there?
0
 

Author Comment

by:robnhood00
ID: 34507835
yes... sorry..  I can only test in production and won't be be able to test till the end of the month.  
Quick questions on your solution before I test...
In this example...
Do            
       i = networkStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))
    Loop Until i = 0

1) the variable i is a byte correct?
2) when I loop the variable i will be reset each time.  In this example, should I read the bytes into a string first and then concatenate on each loop?

Thanks!
0
 
LVL 11

Accepted Solution

by:
MajorBigDeal earned 500 total points
ID: 34570140
That is OK - take your time. I was just following up but I'm not in any hurry.  

i is an int and it is the count of bytes read.  It will be zero if no data is read in that iteration. The actual data goes into the buffer, which in this example is called "bytes".  That is not a nice name, sorry about that!  It is a byte array (byte[]).  

  Here is another example from http://msdn.microsoft.com/en-us/library/system.net.sockets.networkstream.read.aspx


            // Check to see if this NetworkStream is readable.
            if(myNetworkStream.CanRead){
                byte[] myReadBuffer = new byte[1024];
                StringBuilder myCompleteMessage = new StringBuilder();
                int numberOfBytesRead = 0;

                // Incoming message may be larger than the buffer size.
                do{
                     numberOfBytesRead = myNetworkStream.Read(myReadBuffer, 0, myReadBuffer.Length);

                               myCompleteMessage.AppendFormat("{0}", Encoding.ASCII.GetString(myReadBuffer, 0, numberOfBytesRead));
                                                       
                }
                while(myNetworkStream.DataAvailable);

                // Print out the received message to the console.
                Console.WriteLine("You received the following message : " +
                                             myCompleteMessage);
            }
            else{
                 Console.WriteLine("Sorry.  You cannot read from this NetworkStream.");
0
 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 34769638
Any luck?
0
 

Author Comment

by:robnhood00
ID: 34921454
Hi MajorBigDeal,
I just put the code into production and so far so good.  I'll monitor for a week and let you know if it works.

Thanks for your patience!
0
 

Author Comment

by:robnhood00
ID: 34921720
Looks like it's not working.  The data is being truncated at either the 1460 or 2920.  

If I send the data to the port on the local machine... no problem.  So it's only happening when a remote server tries to send the data across.  

This is a VM machine, so I'm wondering if there are some problems with that and not the actual code.  Any ideas would be helpful.

Thanks!
0
 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 34923195
I think we can use the fact that it works from the local port as leverage for debugging efforts.

When you send the data in from the local port is it one big  packet?  By big I mean larger than 1500 bytes.  I'm wondering if this has something to do with the MTU settings on the switches - maybe the message is being broken up into multiple packets when it traverses the network but not when you use the local port?

Can you run Wireshark on this machine and see what the packets look like when the come in thru the network vs. the local port.  

Also, is there any way to replicate the problem outside of production?  It would be much better if we could debug and correct the problem first and then move to production.
0
 

Author Comment

by:robnhood00
ID: 35117210
Update: I was able to put together a physical box to for testing.  The errors are still happening so we can rule out the VM.
I also found that the sending application is Mirth (just an fyi).  Now that I have a test box I'll be able to use Wireshark and other diagnostics.  I'll send another update as soon as I gather more info.
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 35117996
Very nice! Looking forward to hearing your results.
0
 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 35501978
Did you have any luck?
0
 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 35509703
Still there?
0
 

Author Comment

by:robnhood00
ID: 35708214
Hi MajorBigDeal...
Again..sorry for the delay.  It takes a lot to get things tested at this facility.
So...I was able to install and configure Mirth to receive the data on the VM.  Mirth runs on the Java platform and there have been ZERO problems with the messages.   We're most likely going to restructure the code to work with the Mirth product since it's working, however it would be nice to get the VB.NET code working for future projects.
I feel like there's just something wrong in general with the .NET Framework and how it handles tcp/ip sockets.
Please let me know if this new information sparks any ideas.
Thanks!
0
 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 35893126
I was kind of looking forward to figuring this out but circumstances prevented us from doing the necessary debugging.  The good news is that you got it working and that is important thing.  Congratulations!

I tried my best to help you over the past 4 months and there is a lot of info in this question that could be useful to someone having a similar situation to yours.  If you don't mind, please accept one of my answers and give it an "A".  Thanks and best of luck to you!
0
 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 35919099
Hi again, I'm glad you got it working.  I know you decided to use a different approach but can you go ahead and accept whichever answer or answers that helped you most (even even if was just providing ideas or information).

Thanks!

0
 

Author Comment

by:robnhood00
ID: 35948120
Hi Marjor...
Your feedback and patience was very important.  I would also have liked to solve this issue in DotNet, as I am developing similar apps and it would have been great to lock this up.  
Thanks!
0
 
LVL 11

Expert Comment

by:MajorBigDeal
ID: 35948423
Thanks very much for accepting my answer.  I would also like to solve this.  In order to do that we need to be able to reproduce the problem. Then we can work on resolve it.  If we can't reproduce the problem, or if we can only do it once a month, it is going to be very difficult to solve it!
0
 

Expert Comment

by:jkribaHTS
ID: 40757695
Hi All,
  I have found the solution for this problem,

ROOT CAUSE OF THE ISSUE:
------------------------------------------
TCPLsitner Class is using the the I/O Completion ports for its internal implementation  and it has an internal Timeout.
If the data that is getting received more than the MTU Size i.e. 1460 Bytes...The Remote socket will be getting closed and the data will be getting truncated..

Solution:
-----------------
Override the member NetworkStream.ReadTimeOut with more than 500 Ms..So that the socket will not be Closed in case of data that was getting received more than the MTU size.
Please do not rely on the variables networkStream.DataAvailable   , networkStream.CanRead since they were also breaking if the data size is more than the MTU size..

Execute the Read call in a loop with Chunk by chunk..

NetworkStream networkStream = clientSocket.GetStream();
networkStream.ReadTimeout = 2000;

 int numberOfBytesRead = 0;

                    // Incoming message may be larger than the buffer size hence read it by chunk
                    while(true)
                    {
                        numberOfBytesRead = networkStream.Read(bytesFrom, 0, bytesFrom.Length);

                        if (numberOfBytesRead <= 0)
                            break;

                            myCompleteMessage.AppendFormat("{0}", Encoding.ASCII.GetString(bytesFrom, 0, numberOfBytesRead));
                    }



I have tested the code and its working perfect now, Thanks

Regards
JK
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

VMware Update Manager(VUM) “error code: 15” during ESXi 6.0 Remediate update in VUM operation
When we have a dead host and we lose all connections to the ESXi, and we need to find a way to move all VMs from that dead ESXi host.
How to install and configure Citrix XenApp 6.5 - Part 1. In this video tutorial we have explained step by step installation of Citrix XenApp 6.5 Server on Windows Server 2008 R2 is explained in this video. We have explained the difference between…
In this video tutorial I show you the main steps to install and configure  a VMware ESXi6.0 server. The video has my comments as text on the screen and you can pause anytime when needed. Hope this will be helpful. Verify that your hardware and BIO…

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now