dg043
asked on
Winsock communicatiion limits string length
I am using a Winsock control to convert a user-defined type into a byte array and send it over TCP to a remote host where it is reassembled back into the user_defined type. The server code is:
Private Sub winComms_DataArrival(Index
Dim data_stream
Dim byte_array() As Byte
Dim comms As CommunicationPacket
Dim client As Integer
Me.lstStatus.AddItem "Receiving data ..."
Me.winComms(Index).GetData
byte_array = data_stream
CopyMemory comms, byte_array(0), LenB(comms)
Select Case comms.MessageType
Case vbSimpleMessage
Me.lstStatus.AddItem comms.Message
Case vbFaxObject
Case Else
Me.lstStatus.AddItem "Data type not recognised: " & comms.MessageType
End Select
For client = 1 To clients_connected Step 1
Me.winComms(client).SendDa
Next client
End Sub
and the client code is:
Private Sub cmdSendText_Click()
Dim comms As CommunicationPacket
Dim byte_array() As Byte
Dim sent As Variant
If Me.winComms.State <> 7 Then
Connect
End If
DoEvents
If Me.winComms.State = sckConnected Then
comms.MessageType = vbSimpleMessage
comms.Message = Me.txtMessage
ReDim byte_array(LenB(comms)) As Byte
CopyMemory byte_array(0), comms, LenB(comms)
sent = byte_array
Me.winComms.SendData sent
Me.sbarStatus.SimpleText = "Sending ..."
Else
MsgBox "Not currently connected to host"
End If
End Sub
Both client and server contain the following global declarations:
Private Declare Sub CopyMemory Lib "KERNEL32" Alias "RtlMoveMemory" (hpvDest As Any, hpvSource As Any, ByVal cbCopy As Long)
Private Const vbSimpleMessage As Single = 1
Private Const vbvbFaxObject As Single = 2
Private Type CommunicationPacket
MessageType As Integer
Message As String * 73
End Type
This code was modified from an example on the Microsoft website. My question is: In the declaration of the CommunicationPacket type, a string is declared but as 'String * 73'. Why is it necessary to put the '* 73' on the end and what does this mean? I suspect it has something to do with the number of bytes that the string occupies but why do you only need to add this condition when the type will be sent over a TCP connection and why is 73 the maximum number (any higher and the server crashes with a windows error about being unable to read the memory at a certain location)?
Is there maybe an alternative way of sending a string?
Private Sub winComms_DataArrival(Index
Dim data_stream
Dim byte_array() As Byte
Dim comms As CommunicationPacket
Dim client As Integer
Me.lstStatus.AddItem "Receiving data ..."
Me.winComms(Index).GetData
byte_array = data_stream
CopyMemory comms, byte_array(0), LenB(comms)
Select Case comms.MessageType
Case vbSimpleMessage
Me.lstStatus.AddItem comms.Message
Case vbFaxObject
Case Else
Me.lstStatus.AddItem "Data type not recognised: " & comms.MessageType
End Select
For client = 1 To clients_connected Step 1
Me.winComms(client).SendDa
Next client
End Sub
and the client code is:
Private Sub cmdSendText_Click()
Dim comms As CommunicationPacket
Dim byte_array() As Byte
Dim sent As Variant
If Me.winComms.State <> 7 Then
Connect
End If
DoEvents
If Me.winComms.State = sckConnected Then
comms.MessageType = vbSimpleMessage
comms.Message = Me.txtMessage
ReDim byte_array(LenB(comms)) As Byte
CopyMemory byte_array(0), comms, LenB(comms)
sent = byte_array
Me.winComms.SendData sent
Me.sbarStatus.SimpleText = "Sending ..."
Else
MsgBox "Not currently connected to host"
End If
End Sub
Both client and server contain the following global declarations:
Private Declare Sub CopyMemory Lib "KERNEL32" Alias "RtlMoveMemory" (hpvDest As Any, hpvSource As Any, ByVal cbCopy As Long)
Private Const vbSimpleMessage As Single = 1
Private Const vbvbFaxObject As Single = 2
Private Type CommunicationPacket
MessageType As Integer
Message As String * 73
End Type
This code was modified from an example on the Microsoft website. My question is: In the declaration of the CommunicationPacket type, a string is declared but as 'String * 73'. Why is it necessary to put the '* 73' on the end and what does this mean? I suspect it has something to do with the number of bytes that the string occupies but why do you only need to add this condition when the type will be sent over a TCP connection and why is 73 the maximum number (any higher and the server crashes with a windows error about being unable to read the memory at a certain location)?
Is there maybe an alternative way of sending a string?
![Avatar of Guy Hengel [angelIII / a3]](https://filedb.experts-exchange.com/files/public/2013/11/25/ddbe0ea6-aa6e-4279-b781-57b735dd3ea0.jpeg){kind=small}
Guy Hengel [angelIII / a3]
it is not necessary in general, but you need to have the same length on both the server and the client if you work like this.
ASKER
Thanks. I am aware that both the client and server have to be the same in respect to this declaration. I don't understand WHY you have to include this number (on either the server or the client).
If there is an alternative, easier way of achieving the same result then I would welcome a pointer. I have tried encapsulating the strings in an object (instead of a type) and this doesn't work. I don't really want to declare the strings as global (i.e. outside of the type) becaus I would like to lump everything together
If there is an alternative, easier way of achieving the same result then I would welcome a pointer. I have tried encapsulating the strings in an object (instead of a type) and this doesn't work. I don't really want to declare the strings as global (i.e. outside of the type) becaus I would like to lump everything together
Hi,
Dim Message As String * 73
means that message is a fixed length strin with maximum width set at 73 and is always padded with spaces to fill this length. If you remove (* 73) and declare a dynamic string
Dim Message As String
the problem of the crash will be solved. 73 as a number doesn't have any special significance - i must denote the static size of each message for that example. You can change it to whatever you would like to be th maximum size or remove it completly to have a dynamic string.
-wings
Dim Message As String * 73
means that message is a fixed length strin with maximum width set at 73 and is always padded with spaces to fill this length. If you remove (* 73) and declare a dynamic string
Dim Message As String
the problem of the crash will be solved. 73 as a number doesn't have any special significance - i must denote the static size of each message for that example. You can change it to whatever you would like to be th maximum size or remove it completly to have a dynamic string.
-wings
ASKER
Thanks for confirming what I suspected about the fixed length string. I'm fine with declaring the strings as dynamic but the only trouble is that when I do that (on both the client and the server), I receive the following Application error on the server when I try to send information from the client:
The instruction at "0x779d953b" referenced memory at "0x001fb4a0". The memory could not be "read"
Although, I suspect this doesn't mean a lot; the point is that the system crashes. Incidentally, this is is the same error that you receive on the server when you use fixed length strings and you declare the string too large (over 73 in my example).
I guess that this problem has something to do with the 'CopyMemory' library method and that byte arrays can only be a certain size???? I don't really know; I'm just clutching at straws. Any ideas? Again, even if that's just an alternative way of achieving the same result.
The instruction at "0x779d953b" referenced memory at "0x001fb4a0". The memory could not be "read"
Although, I suspect this doesn't mean a lot; the point is that the system crashes. Incidentally, this is is the same error that you receive on the server when you use fixed length strings and you declare the string too large (over 73 in my example).
I guess that this problem has something to do with the 'CopyMemory' library method and that byte arrays can only be a certain size???? I don't really know; I'm just clutching at straws. Any ideas? Again, even if that's just an alternative way of achieving the same result.
Hi,
The problem is with size calculation
// client
Dim sz As Long
sz = LenB(comms) + Len(comms.Message)
ReDim byte_array(sz) As Byte
CopyMemory byte_array(0), comms, sz
// server
Me.winComms(Index).GetData
' , LenB(comms) not required let it get all the data
byte_array = data_stream
CopyMemory comms, byte_array(0), UBound(byte_array)
this should fix your problem ... and remove the * 73 too :)
Is there any specific reason you are using byte arrays?
ReDim byte_array(LenB(comms)) As Byte
CopyMemory byte_array(0), comms, LenB(comms)
sent = byte_array
Me.winComms.SendData sent
Me.winComms(Index).GetData
byte_array = data_stream
CopyMemory comms, byte_array(0), LenB(comms)
alternativly one can use strings
//client
dim str as Messages
str = "some message ..."
Me.winComms.SendData sent str
//server
dim str as Messages
Me.winComms(Index).GetData
-wings
The problem is with size calculation
// client
Dim sz As Long
sz = LenB(comms) + Len(comms.Message)
ReDim byte_array(sz) As Byte
CopyMemory byte_array(0), comms, sz
// server
Me.winComms(Index).GetData
' , LenB(comms) not required let it get all the data
byte_array = data_stream
CopyMemory comms, byte_array(0), UBound(byte_array)
this should fix your problem ... and remove the * 73 too :)
Is there any specific reason you are using byte arrays?
ReDim byte_array(LenB(comms)) As Byte
CopyMemory byte_array(0), comms, LenB(comms)
sent = byte_array
Me.winComms.SendData sent
Me.winComms(Index).GetData
byte_array = data_stream
CopyMemory comms, byte_array(0), LenB(comms)
alternativly one can use strings
//client
dim str as Messages
str = "some message ..."
Me.winComms.SendData sent str
//server
dim str as Messages
Me.winComms(Index).GetData
-wings
ASKER
That seems logical. However, I have tried this new code and it still produces the Application error that I gave above. Any more ideas??
P.S. As previously stated, the reason why I am using byte arrays is that I want to keep together a logical grouping of items. The only ways I can think of doing this are to have either an object or a user-defined type. However, neither of these seem to work
P.S. As previously stated, the reason why I am using byte arrays is that I want to keep together a logical grouping of items. The only ways I can think of doing this are to have either an object or a user-defined type. However, neither of these seem to work
Hi,
I have tried the size calculations and it works for me - ofcourse i added bits of code to fill in the gaps for making the whole thing work.
can you post a working sample code that i can see?
-wings
I have tried the size calculations and it works for me - ofcourse i added bits of code to fill in the gaps for making the whole thing work.
can you post a working sample code that i can see?
-wings
I just changed your type definition like that:
Private Type CommunicationPacket
MessageType As Integer
Message As String * 1000
End Type
I tested the code for 100 times, it didn't throw any error. My platform is VB6 + WIN2000
Anyway, if it still throws error on your computer, I think you might change the type definition like this:
Private Type CommunicationPacket
MessageType As Integer
Message(1000) As Byte
End Type
Then, use following methods to convert from string to byte array:
Public Sub StringToByte(str As String, bArray() As Byte)
Dim i As Long
For i = 1 To Len(str)
bArray(i - 1) = Asc(Mid(str, i, 1))
Next
bArray(i - 1) = 0
End Sub
Public Function ByteToString(bArray() As Byte) As String
Dim i As Long
Dim str As String
For i = 0 To UBound(bArray)
If bArray(i) > 0 Then
str = str + Chr(bArray(i))
Else
Exit For
End If
Next
ByteToString = str
End Function
Regards,
Jungle
Private Type CommunicationPacket
MessageType As Integer
Message As String * 1000
End Type
I tested the code for 100 times, it didn't throw any error. My platform is VB6 + WIN2000
Anyway, if it still throws error on your computer, I think you might change the type definition like this:
Private Type CommunicationPacket
MessageType As Integer
Message(1000) As Byte
End Type
Then, use following methods to convert from string to byte array:
Public Sub StringToByte(str As String, bArray() As Byte)
Dim i As Long
For i = 1 To Len(str)
bArray(i - 1) = Asc(Mid(str, i, 1))
Next
bArray(i - 1) = 0
End Sub
Public Function ByteToString(bArray() As Byte) As String
Dim i As Long
Dim str As String
For i = 0 To UBound(bArray)
If bArray(i) > 0 Then
str = str + Chr(bArray(i))
Else
Exit For
End If
Next
ByteToString = str
End Function
Regards,
Jungle
You need to declare the string in your type as fixed width. You can't use a dynamic string in it, because then the type only stores a pointer to where the string in memory actualy is. Then when you attempt to copy the data into your structure using the CopyMemory() API, it will run over the boundaries of your type and cause the error. This is essentially a "buffer overflow" error.
yes that is correct - i missed that, sorry.
in the server code add before copying memory
comms.Message = String(UBound(byte_array) - LenB(comms))
CopyMemory comms, byte_array(0), UBound(byte_array)
-wings
in the server code add before copying memory
comms.Message = String(UBound(byte_array) - LenB(comms))
CopyMemory comms, byte_array(0), UBound(byte_array)
-wings
You could try this:
Private Sub winComms_DataArrival(Index
' Move the data directtly to the array so that you don't get problems
Dim byte_array() As Byte
Redim byte_array(bytesTotal-1)
Me.winComms(Index).GetData
etc....
Private Sub winComms_DataArrival(Index
' Move the data directtly to the array so that you don't get problems
Dim byte_array() As Byte
Redim byte_array(bytesTotal-1)
Me.winComms(Index).GetData
etc....
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
However, I would suggest a change to your code for larger transmissions.
In an ideal world the receiving computer should control the transmission. So the server would tell the client to pickup the data. The client then requests the data in chunks, so the client would say first send me butes 1 to 10000, then 10001 to 20000. Normally with large transmissions you would add a CRC32 checksum onto each block of data. In this way the client can check the data as it comes in and if a checksum error is detected the data can be rerequested. (TCP is supposed to look after data integrity but sometimes it fails to find data errors.) Moving smaller chunks of data is always better from a system perspective as it won't have to buffer so much data. Also it is simple to create auto-restart features which are necessay when sending really big files. To do this the client quesues all requests and then works through its queue. When the app restarts it loads its queue and starts operating from the last received point.
But in your case...you may not want to change the way it is working. Many servers will chunk the output so normally so you data arrival should be coded like this simple example using strings (byte array is better but not so easy to demo):
Dim sBuffer As String ' Module level declaration
Const msTerminator as String = "<<EOT>>" ' record terminator string
Private Sub winComms_DataArrival(Index
Dim sNewData As String
Me.winComms(Index).GetData
sBuffer = sBuffer + sNewData
If Right(sBuffer, Len(msTerminator)) = msTerminator Then
' You now have all the data in sBuffer
' So Set a flag to signal that data is now complete
mbComplete = True
End If
End Sub
' Before you start receiving data you need to clear your buffer and job complete signal
In an ideal world the receiving computer should control the transmission. So the server would tell the client to pickup the data. The client then requests the data in chunks, so the client would say first send me butes 1 to 10000, then 10001 to 20000. Normally with large transmissions you would add a CRC32 checksum onto each block of data. In this way the client can check the data as it comes in and if a checksum error is detected the data can be rerequested. (TCP is supposed to look after data integrity but sometimes it fails to find data errors.) Moving smaller chunks of data is always better from a system perspective as it won't have to buffer so much data. Also it is simple to create auto-restart features which are necessay when sending really big files. To do this the client quesues all requests and then works through its queue. When the app restarts it loads its queue and starts operating from the last received point.
But in your case...you may not want to change the way it is working. Many servers will chunk the output so normally so you data arrival should be coded like this simple example using strings (byte array is better but not so easy to demo):
Dim sBuffer As String ' Module level declaration
Const msTerminator as String = "<<EOT>>" ' record terminator string
Private Sub winComms_DataArrival(Index
Dim sNewData As String
Me.winComms(Index).GetData
sBuffer = sBuffer + sNewData
If Right(sBuffer, Len(msTerminator)) = msTerminator Then
' You now have all the data in sBuffer
' So Set a flag to signal that data is now complete
mbComplete = True
End If
End Sub
' Before you start receiving data you need to clear your buffer and job complete signal
The other problem you will have with an automated process is that once in a blue moon the internet will crash in the middle of sending a large file. The data arrival method will crash with an obscure error message. So you need:
On Error Goto ... in you data arrival event to prevent you application from crashing in the event of a problem.
Further you client needs to have a control loop a bit like this to handle time-outs senario
' start the control loop
OK = Remote.SendAndWaitOK("GET 0 TO 10000" + msTerminator)
--------------Remote.cls
Dim WithEvents MyWinsock As Winsock
Dim mdtTimeOut as Date
Private Function SendAndWaitOK(psDataToSend
' Send data to remote and wait for data
' First clear buffers and flags
msBuffer=""
mbComplete = False
' Set up timeout
' Not you data arrival should also extend the timeout after each chunk of data has arrived.
mdtTimeOut = Now + cSeconds(60)
' Send data
WinsokX.SendData psDataToSend
' wait for response
Do
If mbComplete Then
SendAndWaitOK = True
Exit Functions
End If
if Now > mdtTimeOut Then
ErrD = "Timeout"
ErrN = -1
Exit Function
End If
DoEvents
Sleep 50
Loop
End Function
On Error Goto ... in you data arrival event to prevent you application from crashing in the event of a problem.
Further you client needs to have a control loop a bit like this to handle time-outs senario
' start the control loop
OK = Remote.SendAndWaitOK("GET 0 TO 10000" + msTerminator)
--------------Remote.cls
Dim WithEvents MyWinsock As Winsock
Dim mdtTimeOut as Date
Private Function SendAndWaitOK(psDataToSend
' Send data to remote and wait for data
' First clear buffers and flags
msBuffer=""
mbComplete = False
' Set up timeout
' Not you data arrival should also extend the timeout after each chunk of data has arrived.
mdtTimeOut = Now + cSeconds(60)
' Send data
WinsokX.SendData psDataToSend
' wait for response
Do
If mbComplete Then
SendAndWaitOK = True
Exit Functions
End If
if Now > mdtTimeOut Then
ErrD = "Timeout"
ErrN = -1
Exit Function
End If
DoEvents
Sleep 50
Loop
End Function
ASKER
Thanks for all the help everyone. The only solution that I can seem to get to work properly is based on a suggestion by egl044. I have decided to create an object on the server that can convert all its properties into fixed length byte arrays and send them as a group to the client, where they are reconstructed into an identical object