Link to home
Start Free TrialLog in
Avatar of shootah
shootah

asked on

Boost Serialization/Deserialization techniques?

I want to use the boost serialization library to serialize data from a network source to a binary archive.  The data is heterogenous and I have wrapper classes that will take care of any special processing needed to serialize and deserialize.  I've created a test case and it seems to write the data correctly but deserialization returns garbage after the second iteration. I have attempted to just open the file stream and recreate the wrapper class object.  The first pass seems to work fine and the data looks good.  After the second iteration I receive garbage back from the boost library.  So I feel like I am not using this library correctly.  Are there any suggestions on how to deserialize a large amount of heterogenous data using boost?  The idea is to serialize a series of these wrapper classes depending on what is received from the network and then recreate this data at another time.  I have provided the code to the wrapper class and the deserialization code.
class CReflectAttributeValuesWrapper{

	friend class boost::serialization::access;

	template<class Archive>
	void save(Archive & ar, const unsigned int version) const
	{
		//save attribute handle value pair set
		RTI::ULong ulLengthBytes = 0;
		AttributeHandle hAttribute;
		char* pszValue = 0;
		stringstream ss;
		RTI::ULong size = m_pAttributes->size();
		static int count = 0;

		ss<<"Object Count: "<<++count<<endl;

		ar<<count;

		ss<<"Object handle: "<<m_hObjectHandle<<endl;

		ar << m_hObjectHandle;

		ss<<"attribute set size is "<<size<<endl;

		ar << size;

		for(unsigned long i = 0; i < size; i++)
		{			
			ulLengthBytes = m_pAttributes->getValueLength(i);
			pszValue = new char[ulLengthBytes];
			m_pAttributes->getValue(i,pszValue,ulLengthBytes);

			hAttribute = m_pAttributes->getHandle ((RTI::ULong)i);

			ss<<"serializing attribute handle whose value is "<<(unsigned long)hAttribute;
			ss<<" whose length is "<<(unsigned long)ulLengthBytes<<endl;
			ss<<"serializing attribute ";

			ss<<endl;

			ar << hAttribute;
			ar << ulLengthBytes;
			ar << boost::serialization::make_array(pszValue,ulLengthBytes);
		}

.....



		ss<<"tag is "<<m_sTheTag<<endl;

		CDiagnosticLogfile::writeMessage(CDiagnosticLogfile::DEBUG,ss.str(),__FILE__,__LINE__);
		
		ar<<m_sTheTag;
	}

	template<class Archive>
	void load(Archive & ar, const unsigned int version)
	{
		RTI::AttributeHandleValuePairSet* pAttributes;
		RTI::ULong ulLengthBytes = 0;
		AttributeHandle hAttribute;
		stringstream ss;		
		int objNum = -1;

		RTI::ULong size;

		ar >> objNum;

		ss<<"Object Number "<<objNum<<endl;

		ar >> m_hObjectHandle;

		ss<<"Object handle: "<<m_hObjectHandle<<endl;

		ar >> size;

		ss<<"attribute set size is "<<size<<endl;

		pAttributes = RTI::AttributeSetFactory::create (size);
		
		cout<<ss.str();

		ss.str("");

		for(unsigned long i = 0; i < size; i++)
		{
			ar >> hAttribute;
			ss<<"deserializing attribute handle whose value is "<<(unsigned long)hAttribute;

			ar >> ulLengthBytes;
			ss<<" whose length is "<<(unsigned long)ulLengthBytes<<endl;

			char* pszValue = new char[ulLengthBytes];

			ar >> boost::serialization::make_array(pszValue,ulLengthBytes);

			ss<<endl;
			pAttributes->add (hAttribute, pszValue,ulLengthBytes);
			cout<<ss.str();

			ss.str("");
		}

		ar>>m_sTheTag;

		ss<<"tag is "<<m_sTheTag<<endl;

        cout<<ss.str();
		
	}
	BOOST_SERIALIZATION_SPLIT_MEMBER()

public:
	CReflectAttributeValuesWrapper(){}
	~CReflectAttributeValuesWrapper(){}

	void setAttributeValues(const RTI::AttributeHandleValuePairSet& attributes)
	{
		m_pAttributes = const_cast <RTI::AttributeHandleValuePairSet*> (&attributes);
	}
	
	void setObjectHandle(RTI::ObjectHandle handle){m_hObjectHandle = handle;}
	void setTag(const char* tag){m_sTheTag = const_cast<char*>(tag);}

	RTI::AttributeHandleValuePairSet* getAttributeValues(){return m_pAttributes;}
	RTI::ObjectHandle getObjectHandle(){return m_hObjectHandle;}
	string getTag(){return m_sTheTag;}

private:

	RTI::AttributeHandleValuePairSet* m_pAttributes;
	RTI::ObjectHandle m_hObjectHandle;
	string m_sTheTag;

};

/********  this is in another executable file *********/
std::ifstream ifs("HLAData.bin",ios::binary);
boost::archive::binary_iarchive ia(ifs);

while(!ifs.eof())
{
	CReflectAttributeValuesWrapper ravw;

	ia>>ravw;	
}

Open in new window

Avatar of Infinity08
Infinity08
Flag of Belgium image

What code was used to serialize the data ?
Avatar of shootah
shootah

ASKER

I've attached the serialize code.
void reflectAttributeValues (
    RTI::ObjectHandle                 theObject,    
    const RTI::AttributeHandleValuePairSet& theAttributes, 
    const char                             *theTag) 
{    
/* 	m_ofStream is declared as a member variable and intialized in constructor like this:
	m_ofStream.open("HLAData.bin",ios::binary);
*/
	boost::archive::binary_oarchive oa(m_ofStream);

	CReflectAttributeValuesWrapper data;

	data.setAttributeValues(theAttributes);
	data.setObjectHandle(theObject);
	data.setTag(theTag);

	oa<<data;
	
	
	/********/
	//irrelevant code below
}

Open in new window

>> I've attached the serialize code.

That only places one object in the serialized file.

But when you're getting the data from the file again, you try to get more than one.
Avatar of shootah

ASKER

That function (ReflectAttributeValues) is a callback function that is called with a high frequency.  So as far as I can tell, each time it is called, an object is placed in the serialized file.  I have a version of this code that writes to a text file, and just from eyeing the file, it looks like it is placing the data in sequence in the file (The file gets to about 100k after a few minutes).
Another thing is on the second iteration, the first value that is written to the first variable during deserialization is "22", which is the number in what looks like header information for each entry.

Attached is a portion of the txt version of the serialized file. 11534337 is a valid handle, 3 is the number of attributes, 186 is the "attribute handle", 4 is the size of the attribute in the attached file.  All of this works fine on the first pass.  When I call the insertion operator again, I get "22" for the first variable (the first number in the header for the next sequence).
22 serialization::archive 7 0 0 11534337 3 186 4 0 0 0 0 158 8 -60 125 -73 60 88 -43 0 0 162 176 63 -31 -126 -35 34 -90 -110 47 -65 -7 -30 -45 -44 -22 -89 -19 -64 30 -117 -17 -85 96 0 0 64 46 21 -32 103 6 58 79 63 -68 17 60 17 3 43 -19 63 -11 67 113 -7 -118 -82 -90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -33 65 -123 -66 -76 -104 108 -65 -21 -29 112 -109 -97 76 -45 -65 -103 52 -66 96 -80 -57 -32 -65 -94 -51 -96 0 -6 -32 -95 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -60 125 -73 60 88 -43 0 0 13 Tag_Name<00>
22 serialization::archive 7 0 0 11534338 3 186 4 0 0 0 0 158 8 -60 125 -73 60 -116 -117 64 0 162 176 63 -31 -126 -104 24 118 -29 21 -65 -7 -30 -30 -69 -89 16 10 -64 30 -105 85 90 0 0 0 64 45 61 36 -1 30 63 -94 63 -69 50 -119 -91 -27 -81 46 63 -12 -85 126 70 -76 -5 -19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -33 66 95 -95 104 -100 -31 -65 -21 -29 58 -98 -119 -19 -6 -65 -103 40 98 4 80 76 26 -65 -94 -57 60 71 -90 -46 -125 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -60 125 -73 60 -116 -117 64 0 13 Tag_name<00>

Open in new window

ASKER CERTIFIED SOLUTION
Avatar of Infinity08
Infinity08
Flag of Belgium image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of shootah

ASKER

Wow.  That sure was a lot of typing on my part for such a simple answer.  Thanks for the help.