• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 1554
  • Last Modified:

Pass complex objects via socket in c++

Hi,

Can anybody give me an example of passing objects by c++ sockets? Currently i am transforming my objects into delimited strings (manually iterating through attribute values when serialising/deserialising) and sending the resultant data.

my objects are are similar to:

DataObject
{
    int arrayOfValues1[50];
    double arrayOfValues2[50];
}

Open in new window


I currently have a server written in c++ which reads in objects that it uses for calculations and returns an object of outputs. I'm connecting in both c++ and c# apps, but c# is the priority as the main app that requires the data is in c#. the approach i'm using seems to be quite slow - what are the best practice recommendations?
0
basil365
Asked:
basil365
  • 15
  • 5
  • 5
  • +1
1 Solution
 
basil365Author Commented:
Hi,

I am also having issues with transforming the send bytes back into strings on the client side - data seems to go missing or extra data is received. Any ideas what might be causing this?
0
 
mccarlIT Business Systems Analyst / Software DeveloperCommented:
What you are after may be hard to find, as you want a data binding framework that can iteroperate between C++ and C#. I think in this case, you manual serialization/deserialization will be your best bet, especially with such simple objects.

So you main issue remain the performance of your current code to serialize/deserialize? The objects are that simple that it doesn't sound like it should be that hard to do, or that bad in performance. Have you fully profiled your code to make sure that it is indeed the serialization/deserialization that is slow? You said in the question that it "SEEMS" to be slow, so is possible that something else is causing the slowness?

If you can post the code that you are currently using, we could have a look over it and see if there is anything obvious with it.
0
 
philrosenbergCommented:
In this case where your object only contains fixed length variables and no pointers you may well just be able to cast your object to a char* and send it, pseudocode below
   DataObject mydata;
   //some code to assign the variables
   char * sendpointer=(char*) &mydata;
   send(sendpointer,sizeof(DataObject));

Open in new window


Then at the receive end just write it directly into an object
   DataObject my receiveddata;
   char * receivepointer=(char*) &receiveddata;
   recv(receivepointer,sizeof(DataObject));

Open in new window


Note that if you have any pointers in your object or any dynamically allocated memory (using new or malloc) this won't work as your pointers won't be valid on the receiving system.
Phil
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
basil365Author Commented:
thanks for your answers - i'm in the middle of adding proper profiling, this is my object:

#ifndef OUTPUT_MESSAGE_H
#define OUTPUT_MESSAGE_H

#include "Message.h"

class OutputMessage: public Message
{

protected:
	
public:
	OutputMessage();
	~OutputMessage();

	bool AddData(vector<string> buffer);
	vector<string> Serialise();
	void init();

	int a[2];

	//[101, 3]
	int b[303];

	//[201, 3]
	int c[603];

	//[61, 3]
	int d[183];

	//[41, 5]
	int e[205];

	//[4, 3]
	int f[12];

	//[4, 2]
	int g[8];

	//[2, 3]
	int h[6];

	//[2, 2]
	int i[4];

	//[2]
	int j[2];


	//[3]
	int k[3];

	//[3, 3]
	int l[9];


	//[2, 8]
	int m[16];


	//[3, 4, 4]
	int n[48];

	//[3, 4]
	int o[12];

	//[5]
	int p[5];
	//[3]
	int q[3];
	//[2]
	int r[2];
	//[2]
	int s[2];
	//[2]
	int t[2];
	//[2]
	int u[2];
	//[2]
	int v[2];

	int w[2];

	int x[2];
	//[2,2]
	int y[4];

	//[4, 61, 3]
	int z[732];

	//[4, 41, 3]
	int aa[492];

	//[2, 101, 3]
	int bb[606];


	//[2,51,3]
	int cc[306];

	//[2,51,3]
	int dd[306];

	//[2,51,3]
	int ee[306];

	//totals
	//[71,3]
	int ff[213];  

	//[71,3]
	int gg[213];

	
	
private:
};


#endif

Open in new window


and this is the base class:

#ifndef MESSAGE_H
#define MESSAGE_H

#define DOUBLE_DATA_LEN 10
#define INT_DATA_LEN 10

#include <string>
#include <vector>
#include "common.h"

using namespace std;

class Message
{

protected:
	


public:
	Message();
	~Message();


	//the current member we're dealing with
	string cur_type;
	//length of this member
	int cur_length;

	//index into the current member
	int cursor;

	//state vars
	bool started;
	bool finished;
	bool valid;
	bool acceptsData;

	//handles this for each different type
	//must be provided by derived class
	virtual bool AddData(vector<string> buffer) = 0;
	virtual void init() = 0;
	virtual vector<string> Serialise() = 0;


	//set the current member
	void SetType(string type, int len);
	void msgFinished();

	static void trim(string& str);
	static vector<string> split(string s);

	static double ReadDouble(string s);
	static int ReadInt(string s);

	static void Serialise(string member, vector<string> &buffer, int length, double * data);
	static void Serialise(string member, vector<string> &buffer, double data);

	static void Serialise(string member, vector<string> &buffer, int length, int * data);
	static void Serialise(string member, vector<string> &buffer, int data);

	
	static string Message::DoubleToString(double data);
	static string Message::IntToString(int data);

private:
	
	
};


#endif

Open in new window


Would this be suitable for casting to a char array?
0
 
basil365Author Commented:
This method is where the time is being lost (approx 3 seconds):

OutputMessage Simulator::process(vector<string> data, OutputMessage response)
	{
		bool success = true;
		while(data.size() > 0)
		{
			if ( data[0].compare("start") == 0)
			{
				//check is valid 
				cout << "Create new message object";
				response.init();
			
				//remove from buffer
				data.erase(data.begin());
			}
			else if(data[0].compare("end") == 0)
			{
				//check is valid and call pricing
				cout << "Check validity and call Pricing";
				/*if(response != 0)
				{*/
					response.msgFinished();
					//object should be created now
					return response;
				//}

				//remove from buffer
				data.erase(data.begin());
			}
			else if(data[0].compare("type") == 0 )
			{
				//remove "type" from buffer
				data.erase(data.begin());

				//make sure message is instantiated
				/*if(response!=0)
				{*/
					//array length
					int length = Message::ReadInt(data[0]);
					//remove length from buffer
					data.erase(data.begin());

					response.SetType(data[0], length);
					//remove type name from buffer
					data.erase(data.begin());

					//find out the size of this type message
					while(data.size()>0 && data[0].compare("typeend")!=0)
					{			
						response.AddData(data);
						data.erase(data.begin());
					}

					if(data[0].compare("typeend")==0)
					{
						data.erase(data.begin());
					}

				//}
			}
			else if(data[0].compare("")==0)
			{
				data.erase(data.begin());
			}
			else 
			{
				cout << "Invalid data in process()" << endl;
				cout << data[0] << endl;
				success = false;
				break;
			}
		}

		return response;
	}

Open in new window


the vector of strings that is created from the socket data and passed to the above method is of size 5802 which i would not expect to take so long to process so the above method must very inefficient?
0
 
basil365Author Commented:
also on the server side sending the same object is taking a long amount of time (> 2 secs)

cout << "Started send output at: " << currentDateTime() << endl;

					for(int i=0;i<(int)outputData.size();i++)
					{
						//add space
						outputData[i] += " ";
						//copy string data to char array for sending over socket
						char *a = new char[outputData[i].size()+1];
						a[outputData[i].size()]=0;
						memcpy(a, outputData[i].c_str(), outputData[i].size());

						//cout << a << endl;
						//send
						sockServer.SendData(a);
					}

					cout << "Finished send output at: " << currentDateTime() << endl;

Open in new window

0
 
philrosenbergCommented:
Having quickly scrolled through the code the only variable that would cause problems is the cur_type string. Because a string can grow it uses dynamic memory allocation so when casting to char * all that will get copied is the string's pointer to it's memory, which will then not be valid on the receiving computer. The same will be true if you have any vectors in the class (although I couldn't see any).

If you can change this string to fixed length then a simple cast to char* will work, if not then you will have to do some extra work to serialize your string separately. In addition when you try to reassign the string on the receiving end you might find that it tries to access the invalid pointer so if you need to keep a string then you might want to replace it with a string pointer. In this case you allocate your pointer using new in the constructor, serialize the class and string separately. Send the class and string. Reconstruct the class, ignore the value in the string pointer and reallocate it again with new. Then copy the string value to the new string.

It looks like you are using the string as some sort of identifier, is this correct? If so you could replace it with a number such as a long. This should allow casting as a char *
0
 
ambienceCommented:
>> data.erase(data.begin());

Why cant you just iterate over the vector and increment the iterator when you need to skip over to the next element. data.erase() is causing the whole vector to be reallocated and the first element removed. A vector is not a suitable container for such shifting. You have literally thousands of fields in a serialized vector and data.erase is going to be horribly inefficient for such a case.

Just iterate over data and if you must then try using a deque or list of string rather than a vector. That would be must more efficient.

>> I am also having issues with transforming the send bytes back into strings on the client side - data seems to go missing or extra data is received. Any ideas what might be causing this?

Sounds like you are using a streaming connection like TCP but failing to consider that there is no guarantee for data to appear on the receiver end in the same number of chunks. You can do two SendData on the sender and receive the data as  single chunk, or N chunks of 1 byte each or any other variation. It is up to you to "collect" all and demarcate fields.

Have you implemented a protocol/scheme to demarcate the individual fields in a steam of data? Common ways are to either use a delimiter character or prepend data items with length (in a fixed size var). For example, you read a two byte length and use that length to determine how many bytes are to be read.

>> int arrayOfValues1[50];
>> double arrayOfValues2[50];

Since you are converting to strings and vice versa, you are probably safe but otherwise also keep in mind the differences in byte orders when sending and receiving int or double variables.

Sending double over the wire and receiving as double is not defined. It works mostly (if you properly handle the byte ordering) because most architectures implement the same IEEE standard for representing floating point numbers.
0
 
basil365Author Commented:
Hi philrosenberg, i tried to implement your suggestion , but on the server the object is in an invalid state (random field values). Any suggestions?

client:

InputMessage * msg;

		msg= this->ConvertInput(input);

		cout << "Data serialised at: " << currentDateTime() << endl;

		////Setup socket
		bool done = false;
		char recMessage[STRLEN]; //Receive message array
		char sendMessage[STRLEN]; //Send message array 

		ClientSocket csock;
		int port = 8888;
		string ipAddress = "172.18.64.87";

		csock.ConnectToServer(ipAddress.c_str(), port ); //Connect
		
		//test

		char * sendPointer = (char*) &msg;
		csock.SendData(sendPointer, sizeof(InputMessage));

Open in new window


server:
cout<<"==HOSTING=="<<endl; //Tell the user we are hosting
    //sockServer.StartHosting( port ); //StartHosting

	sockServer.Bind(this->port);

	this->done = false;
	//Connected
	bool connected = false;


	while ( !done )
	{
		cout << "Listening for connections..." << endl;

		sockServer.Listen();

		connected = true;

		cout << "Client Connected at: " << currentDateTime() << endl;

		while(connected)
		{
			InputMessage data;

			char * receivedPointer = (char*) &data;

			sockServer.RecvData(receivedPointer,sizeof(InputMessage));
               }
}

Open in new window


socket impl:

bool Socket::SendData( char *buffer ) //SendData
{
    send( mySocket, buffer, strlen( buffer ), 0 ); //Send the data
    return true; //Return true
}
 
bool Socket::RecvData( char *buffer, int size ) //RecvData
{
    int i = recv( mySocket, buffer, size, 0 );
    //buffer[i] = '\0';
    return true;
}

Open in new window

0
 
ambienceCommented:
Read my previous comment especially the portion regarding the TCP stream oriented nature.

You must send/recv data in a loop to make sure you send all and receive every byte sent. both send() and recv() return the number of bytes actually sent or received.

bool Socket::SendData( char *buffer ) //SendData
{
    int [b]actuallySent[/b] = send( mySocket, buffer, strlen( buffer ), 0 );
    return true; //Return true
}
 
// Sample implementation
bool Socket::RecvData( char *buffer, int size ) //RecvData
{
    while(size) {
          int i = recv( mySocket, buffer, size, 0 );
          if(i <= 0) return false; // error  or conn closed
          size -= i; buffer += i;  
    }
    buffer[0] = '\0';  // make sure the buffer is large enough for this
    return true;
}
     

Open in new window

0
 
philrosenbergCommented:
As ambience says you have to check you have received all the data you need.

Sorry my example was just pseudocode, not a full implementation.

If this doesn't work then which items are invalid? As I said strings, vectors or any other array type object other than static arrays defined like
int[5]
won't be copied in this way. Instead the vector will think it has the correct size, but the pointer to the first element will point to incorrect memory and you'll get massive problems.

Another useful check is that sizeof(InputMessage) is the same on both the server and client. If it isn't then it is because of platform dependant differences in variable lengths. This is because C++ only defines the minimum number of bytes for a variable not the exact number. You could ensure you only use fixed width variables. You should have no problem with ints, but apparently wchars have different sizes on windows and linux. See http://msdn.microsoft.com/en-us/library/29dh1w7z(v=vs.80).aspx for more details. Again ambience mentioned some of this.

Phil
0
 
mccarlIT Business Systems Analyst / Software DeveloperCommented:
Personally, I would steer clear of doing the "casting to char*" method, if you are still thinking of having this be interoperable with C# too. There are too many unknowns/differences in how different environments, languages, compilers, etc handle the physical memory layout of the class members that you are probably just asking for trouble.

How are you going with optimising you current solution? I would agree with ambience, that in your receiving code, all the data.erase() calls would be what is making it slow.

Also, in the server sending code that you said is taking a while, why are you copying the string to the char *? If you can change the implementation of Socket::SendData() method to take the data size as a second parameter, then that block of code could be simplified to the following...

cout << "Started send output at: " << currentDateTime() << endl;

					for(int i=0;i<(int)outputData.size();i++)
					{
						//send
						sockServer.SendData(outputData[i].c_str(), outputData[i].size());
						//add space
						sockServer.SendData(" ", 1);
					}

					cout << "Finished send output at: " << currentDateTime() << endl;

Open in new window

0
 
basil365Author Commented:
Thanks for your help Ambience, I'll go through your points today - I started with   philrosenberg's first as it seemed an easier solution.

@ philrosenberg - my objects only have fixed size arrays and int/double/bool attributes. No field actually seems to be getting set correctly. Both server/client recognise the size as the same.

@ mccarl - thanks, i'll add in your changes as i go through Ambience's and let you know.

Thanks all for your help
0
 
basil365Author Commented:
Hi philrosenberg, I've got your solution partially working. My request object is read correctly on the server and processing occurs perfectly, however i'm having issues sending the output object back to the client. I think the issue is that the char * is too big (object size is 22288), my send function is returning -1. Is there a way around this?
0
 
basil365Author Commented:
the last error on the socket after calling send is WSAENOTSOCK10038
0
 
philrosenbergCommented:
Aparently this means the socket handle is invalid

http://support.microsoft.com/kb/819124

Have you accidentally closed the socket? Otherwise perhaps the socket has been corrupted in some way. Can you run the server in a debugger and see if the error still ocurrs, and if so if the contents of your socket handle change at any point while the code is running.

Phil
0
 
basil365Author Commented:
when running in debug mode the receivedPointer has this value '0xcdcdcdcd <Bad Ptr>'. It works correctly in release mode. Any ideas?
0
 
basil365Author Commented:
fixed that issue
0
 
basil365Author Commented:
the connected socket is losing its reference when i receive the input dataL

bool Socket::RecvData( char *buffer, int size ) //RecvData
{
	int j = WSAGetLastError();
	cout<<"ServerSocket:Last Error Post Rec (error) - "<< j << endl;

    while(size)
	{
		int i = recv( mySocket, buffer, size, 0 );
		if(i<=0) 
		{
			j = WSAGetLastError();
			cout<<"ServerSocket:Last Error Post Rec (error) - "<< j << endl;
			return false;
		}
		size -=i;
		buffer+=i;
	}
    //buffer[i] = '\0';
	j = WSAGetLastError();
	cout<<"ServerSocket:Last Error Post Rec (error) - "<< j << endl; //Error	
    return true;
}

Open in new window


As soon as as data is received from 'recv' the mySocket variable value goes from 7908 to 0 Any ideas?
0
 
mccarlIT Business Systems Analyst / Software DeveloperCommented:
Sounds like a buffer overrun problem. What is the code that calls that RecvData method?
0
 
basil365Author Commented:
MatchInputMessage *data;

			int size = sizeof(InputMessage);

			cout << "Expected Message Size is: " << size << endl;

			char * receivedPointer;// = (char*) &data;

			bool isRec =sockServer.RecvData(receivedPointer,size);

			cout << "Received is: " << isRec << endl;

Open in new window


Size in this case is 680.

I initially had
char * receivedPointer = (char*) &data;

Open in new window

, but this causes the bad pointer issue above when running in debug mode
0
 
basil365Author Commented:
The first line of the above snippet should be 'InputMessage *data;'
0
 
mccarlIT Business Systems Analyst / Software DeveloperCommented:
Ok, so on line 7 of the above you declare a pointer, but it points to nowhere in particular because you haven't assigned it anything, so yeah it is a good chance that you will corrupt other variables when the recv function attempts to write data at the locations that it points too. (Surprised that you are getting Seg Faults too with that code)

And even in the line that you initially had, at least you have assigned an address to the pointer, but instead of it being an address to some memory that can store the received data, you are passing the address of the data pointer itself.

Try this...

InputMessage* data = new InputMessage();

			int size = sizeof(InputMessage);

			cout << "Expected Message Size is: " << size << endl;

			char * receivedPointer = (char*) data;

			bool isRec =sockServer.RecvData(receivedPointer,size);

			cout << "Received is: " << isRec << endl;

Open in new window

0
 
basil365Author Commented:
Thanks mccarl - thats done the trick. I must start remembering memory management, but use c/c++ so infrequently i've never devoted any time to it :(
0
 
ambienceCommented:
Once again

InputMessage* data = new InputMessage();
char * receivedPointer = (char*) data;

very dangerous thing to do. From what code you have already posted, it appears as InputMessage is a class that has virtual members that it inherits from Message. If that is indeed the case then when you overwrite receivedPointer you also overwrite the virtual table pointer that happens to reside at the first four bytes of the receivedPointer.

It is only safe to cast to a raw array is when the object is a POD (plain old datatype) - has no virutal methods or virtual inheritence - the object itself and any member of it.
0
 
mccarlIT Business Systems Analyst / Software DeveloperCommented:
Basil, even though you are getting somewhere with this method, I would still advise against it. It will be impossible/very difficult to get this working where one of the ends, either client or server, is coded in C#. As well as being rather dangerous in itself.

Have you tried optimising your original code yet?
0
 
basil365Author Commented:
Thanks - i can remove the inherited object, that was only present for the manually code serialisation methods.

I  haven't tried optimising it yet and regarding the c# to c++ i have a client written in c++ that uses cli to pass objects to/from c#. This is all working currently and speed is no longer an issue.

My only reason for using sockets at all is because the main code (server app) is using an sdk which wont compile with cli enabled.
0
 
ambienceCommented:
Oh well - I should have mentioned that earlier. If you typecast to raw array make sure the struct field padding is set to 1, otherwise you'll see phantoms all around.

// MS VC specific
#pragma pack(push,1)

class/struct goes here

#pragma pack(pop)

The bottom line is that its probably best to use a simple byte buffer.
0
 
philrosenbergCommented:
Hi Basil
Sorry I wasn't around over the weekend to help you out, but it sounds like ambience and mccarl helped you to find the buffer overrun bug.

Good luck with the rest.

Oh and  if you have concerns about implementation in C# then I believe you can compile your client/server in C++ as a library which you can include in your C# project, although I don't use C# so can't help you with that.

Good luck

Phil
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 15
  • 5
  • 5
  • +1
Tackle projects and never again get stuck behind a technical roadblock.
Join Now