Solved

Boost Serialization/Deserialization techniques?

Posted on 2010-09-01
6
1,346 Views
Last Modified: 2012-05-10
I want to use the boost serialization library to serialize data from a network source to a binary archive.  The data is heterogenous and I have wrapper classes that will take care of any special processing needed to serialize and deserialize.  I've created a test case and it seems to write the data correctly but deserialization returns garbage after the second iteration. I have attempted to just open the file stream and recreate the wrapper class object.  The first pass seems to work fine and the data looks good.  After the second iteration I receive garbage back from the boost library.  So I feel like I am not using this library correctly.  Are there any suggestions on how to deserialize a large amount of heterogenous data using boost?  The idea is to serialize a series of these wrapper classes depending on what is received from the network and then recreate this data at another time.  I have provided the code to the wrapper class and the deserialization code.
class CReflectAttributeValuesWrapper{

	friend class boost::serialization::access;

	template<class Archive>
	void save(Archive & ar, const unsigned int version) const
	{
		//save attribute handle value pair set
		RTI::ULong ulLengthBytes = 0;
		AttributeHandle hAttribute;
		char* pszValue = 0;
		stringstream ss;
		RTI::ULong size = m_pAttributes->size();
		static int count = 0;

		ss<<"Object Count: "<<++count<<endl;

		ar<<count;

		ss<<"Object handle: "<<m_hObjectHandle<<endl;

		ar << m_hObjectHandle;

		ss<<"attribute set size is "<<size<<endl;

		ar << size;

		for(unsigned long i = 0; i < size; i++)
		{			
			ulLengthBytes = m_pAttributes->getValueLength(i);
			pszValue = new char[ulLengthBytes];
			m_pAttributes->getValue(i,pszValue,ulLengthBytes);

			hAttribute = m_pAttributes->getHandle ((RTI::ULong)i);

			ss<<"serializing attribute handle whose value is "<<(unsigned long)hAttribute;
			ss<<" whose length is "<<(unsigned long)ulLengthBytes<<endl;
			ss<<"serializing attribute ";

			ss<<endl;

			ar << hAttribute;
			ar << ulLengthBytes;
			ar << boost::serialization::make_array(pszValue,ulLengthBytes);
		}

.....



		ss<<"tag is "<<m_sTheTag<<endl;

		CDiagnosticLogfile::writeMessage(CDiagnosticLogfile::DEBUG,ss.str(),__FILE__,__LINE__);
		
		ar<<m_sTheTag;
	}

	template<class Archive>
	void load(Archive & ar, const unsigned int version)
	{
		RTI::AttributeHandleValuePairSet* pAttributes;
		RTI::ULong ulLengthBytes = 0;
		AttributeHandle hAttribute;
		stringstream ss;		
		int objNum = -1;

		RTI::ULong size;

		ar >> objNum;

		ss<<"Object Number "<<objNum<<endl;

		ar >> m_hObjectHandle;

		ss<<"Object handle: "<<m_hObjectHandle<<endl;

		ar >> size;

		ss<<"attribute set size is "<<size<<endl;

		pAttributes = RTI::AttributeSetFactory::create (size);
		
		cout<<ss.str();

		ss.str("");

		for(unsigned long i = 0; i < size; i++)
		{
			ar >> hAttribute;
			ss<<"deserializing attribute handle whose value is "<<(unsigned long)hAttribute;

			ar >> ulLengthBytes;
			ss<<" whose length is "<<(unsigned long)ulLengthBytes<<endl;

			char* pszValue = new char[ulLengthBytes];

			ar >> boost::serialization::make_array(pszValue,ulLengthBytes);

			ss<<endl;
			pAttributes->add (hAttribute, pszValue,ulLengthBytes);
			cout<<ss.str();

			ss.str("");
		}

		ar>>m_sTheTag;

		ss<<"tag is "<<m_sTheTag<<endl;

        cout<<ss.str();
		
	}
	BOOST_SERIALIZATION_SPLIT_MEMBER()

public:
	CReflectAttributeValuesWrapper(){}
	~CReflectAttributeValuesWrapper(){}

	void setAttributeValues(const RTI::AttributeHandleValuePairSet& attributes)
	{
		m_pAttributes = const_cast <RTI::AttributeHandleValuePairSet*> (&attributes);
	}
	
	void setObjectHandle(RTI::ObjectHandle handle){m_hObjectHandle = handle;}
	void setTag(const char* tag){m_sTheTag = const_cast<char*>(tag);}

	RTI::AttributeHandleValuePairSet* getAttributeValues(){return m_pAttributes;}
	RTI::ObjectHandle getObjectHandle(){return m_hObjectHandle;}
	string getTag(){return m_sTheTag;}

private:

	RTI::AttributeHandleValuePairSet* m_pAttributes;
	RTI::ObjectHandle m_hObjectHandle;
	string m_sTheTag;

};

/********  this is in another executable file *********/
std::ifstream ifs("HLAData.bin",ios::binary);
boost::archive::binary_iarchive ia(ifs);

while(!ifs.eof())
{
	CReflectAttributeValuesWrapper ravw;

	ia>>ravw;	
}

Open in new window

0
Comment
Question by:shootah
  • 3
  • 3
6 Comments
 
LVL 53

Expert Comment

by:Infinity08
ID: 33584352
What code was used to serialize the data ?
0
 

Author Comment

by:shootah
ID: 33586654
I've attached the serialize code.
void reflectAttributeValues (
    RTI::ObjectHandle                 theObject,    
    const RTI::AttributeHandleValuePairSet& theAttributes, 
    const char                             *theTag) 
{    
/* 	m_ofStream is declared as a member variable and intialized in constructor like this:
	m_ofStream.open("HLAData.bin",ios::binary);
*/
	boost::archive::binary_oarchive oa(m_ofStream);

	CReflectAttributeValuesWrapper data;

	data.setAttributeValues(theAttributes);
	data.setObjectHandle(theObject);
	data.setTag(theTag);

	oa<<data;
	
	
	/********/
	//irrelevant code below
}

Open in new window

0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33587312
>> I've attached the serialize code.

That only places one object in the serialized file.

But when you're getting the data from the file again, you try to get more than one.
0
Migrating Your Company's PCs

To keep pace with competitors, businesses must keep employees productive, and that means providing them with the latest technology. This document provides the tips and tricks you need to help you migrate an outdated PC fleet to new desktops, laptops, and tablets.

 

Author Comment

by:shootah
ID: 33587751
That function (ReflectAttributeValues) is a callback function that is called with a high frequency.  So as far as I can tell, each time it is called, an object is placed in the serialized file.  I have a version of this code that writes to a text file, and just from eyeing the file, it looks like it is placing the data in sequence in the file (The file gets to about 100k after a few minutes).
Another thing is on the second iteration, the first value that is written to the first variable during deserialization is "22", which is the number in what looks like header information for each entry.

Attached is a portion of the txt version of the serialized file. 11534337 is a valid handle, 3 is the number of attributes, 186 is the "attribute handle", 4 is the size of the attribute in the attached file.  All of this works fine on the first pass.  When I call the insertion operator again, I get "22" for the first variable (the first number in the header for the next sequence).
22 serialization::archive 7 0 0 11534337 3 186 4 0 0 0 0 158 8 -60 125 -73 60 88 -43 0 0 162 176 63 -31 -126 -35 34 -90 -110 47 -65 -7 -30 -45 -44 -22 -89 -19 -64 30 -117 -17 -85 96 0 0 64 46 21 -32 103 6 58 79 63 -68 17 60 17 3 43 -19 63 -11 67 113 -7 -118 -82 -90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -33 65 -123 -66 -76 -104 108 -65 -21 -29 112 -109 -97 76 -45 -65 -103 52 -66 96 -80 -57 -32 -65 -94 -51 -96 0 -6 -32 -95 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -60 125 -73 60 88 -43 0 0 13 Tag_Name<00>
22 serialization::archive 7 0 0 11534338 3 186 4 0 0 0 0 158 8 -60 125 -73 60 -116 -117 64 0 162 176 63 -31 -126 -104 24 118 -29 21 -65 -7 -30 -30 -69 -89 16 10 -64 30 -105 85 90 0 0 0 64 45 61 36 -1 30 63 -94 63 -69 50 -119 -91 -27 -81 46 63 -12 -85 126 70 -76 -5 -19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -33 66 95 -95 104 -100 -31 -65 -21 -29 58 -98 -119 -19 -6 -65 -103 40 98 4 80 76 26 -65 -94 -57 60 71 -90 -46 -125 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -60 125 -73 60 -116 -117 64 0 13 Tag_name<00>

Open in new window

0
 
LVL 53

Accepted Solution

by:
Infinity08 earned 500 total points
ID: 33588473
>> So as far as I can tell, each time it is called, an object is placed in the serialized file

You're still creating a new boost::archive::binary_oarchive for every object, which you don't do when reading the file.

You should probably align both the serializing and de-serializing loops, to both use the same approach (either one archive instance defined outside of the loop, or an archive instance for each iteration).
0
 

Author Comment

by:shootah
ID: 33589020
Wow.  That sure was a lot of typing on my part for such a simple answer.  Thanks for the help.
0

Featured Post

ScreenConnect 6.0 Free Trial

Discover new time-saving features in one game-changing release, ScreenConnect 6.0, based on partner feedback. New features include a redesigned UI, app configurations and chat acknowledgement to improve customer engagement!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Often, when implementing a feature, you won't know how certain events should be handled at the point where they occur and you'd rather defer to the user of your function or class. For example, a XML parser will extract a tag from the source code, wh…
What is C++ STL?: STL stands for Standard Template Library and is a part of standard C++ libraries. It contains many useful data structures (containers) and algorithms, which can spare you a lot of the time. Today we will look at the STL Vector. …
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…

777 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question