Solved

Boost Serialization/Deserialization techniques?

Posted on 2010-09-01
6
1,349 Views
Last Modified: 2012-05-10
I want to use the boost serialization library to serialize data from a network source to a binary archive.  The data is heterogenous and I have wrapper classes that will take care of any special processing needed to serialize and deserialize.  I've created a test case and it seems to write the data correctly but deserialization returns garbage after the second iteration. I have attempted to just open the file stream and recreate the wrapper class object.  The first pass seems to work fine and the data looks good.  After the second iteration I receive garbage back from the boost library.  So I feel like I am not using this library correctly.  Are there any suggestions on how to deserialize a large amount of heterogenous data using boost?  The idea is to serialize a series of these wrapper classes depending on what is received from the network and then recreate this data at another time.  I have provided the code to the wrapper class and the deserialization code.
class CReflectAttributeValuesWrapper{

	friend class boost::serialization::access;

	template<class Archive>
	void save(Archive & ar, const unsigned int version) const
	{
		//save attribute handle value pair set
		RTI::ULong ulLengthBytes = 0;
		AttributeHandle hAttribute;
		char* pszValue = 0;
		stringstream ss;
		RTI::ULong size = m_pAttributes->size();
		static int count = 0;

		ss<<"Object Count: "<<++count<<endl;

		ar<<count;

		ss<<"Object handle: "<<m_hObjectHandle<<endl;

		ar << m_hObjectHandle;

		ss<<"attribute set size is "<<size<<endl;

		ar << size;

		for(unsigned long i = 0; i < size; i++)
		{			
			ulLengthBytes = m_pAttributes->getValueLength(i);
			pszValue = new char[ulLengthBytes];
			m_pAttributes->getValue(i,pszValue,ulLengthBytes);

			hAttribute = m_pAttributes->getHandle ((RTI::ULong)i);

			ss<<"serializing attribute handle whose value is "<<(unsigned long)hAttribute;
			ss<<" whose length is "<<(unsigned long)ulLengthBytes<<endl;
			ss<<"serializing attribute ";

			ss<<endl;

			ar << hAttribute;
			ar << ulLengthBytes;
			ar << boost::serialization::make_array(pszValue,ulLengthBytes);
		}

.....



		ss<<"tag is "<<m_sTheTag<<endl;

		CDiagnosticLogfile::writeMessage(CDiagnosticLogfile::DEBUG,ss.str(),__FILE__,__LINE__);
		
		ar<<m_sTheTag;
	}

	template<class Archive>
	void load(Archive & ar, const unsigned int version)
	{
		RTI::AttributeHandleValuePairSet* pAttributes;
		RTI::ULong ulLengthBytes = 0;
		AttributeHandle hAttribute;
		stringstream ss;		
		int objNum = -1;

		RTI::ULong size;

		ar >> objNum;

		ss<<"Object Number "<<objNum<<endl;

		ar >> m_hObjectHandle;

		ss<<"Object handle: "<<m_hObjectHandle<<endl;

		ar >> size;

		ss<<"attribute set size is "<<size<<endl;

		pAttributes = RTI::AttributeSetFactory::create (size);
		
		cout<<ss.str();

		ss.str("");

		for(unsigned long i = 0; i < size; i++)
		{
			ar >> hAttribute;
			ss<<"deserializing attribute handle whose value is "<<(unsigned long)hAttribute;

			ar >> ulLengthBytes;
			ss<<" whose length is "<<(unsigned long)ulLengthBytes<<endl;

			char* pszValue = new char[ulLengthBytes];

			ar >> boost::serialization::make_array(pszValue,ulLengthBytes);

			ss<<endl;
			pAttributes->add (hAttribute, pszValue,ulLengthBytes);
			cout<<ss.str();

			ss.str("");
		}

		ar>>m_sTheTag;

		ss<<"tag is "<<m_sTheTag<<endl;

        cout<<ss.str();
		
	}
	BOOST_SERIALIZATION_SPLIT_MEMBER()

public:
	CReflectAttributeValuesWrapper(){}
	~CReflectAttributeValuesWrapper(){}

	void setAttributeValues(const RTI::AttributeHandleValuePairSet& attributes)
	{
		m_pAttributes = const_cast <RTI::AttributeHandleValuePairSet*> (&attributes);
	}
	
	void setObjectHandle(RTI::ObjectHandle handle){m_hObjectHandle = handle;}
	void setTag(const char* tag){m_sTheTag = const_cast<char*>(tag);}

	RTI::AttributeHandleValuePairSet* getAttributeValues(){return m_pAttributes;}
	RTI::ObjectHandle getObjectHandle(){return m_hObjectHandle;}
	string getTag(){return m_sTheTag;}

private:

	RTI::AttributeHandleValuePairSet* m_pAttributes;
	RTI::ObjectHandle m_hObjectHandle;
	string m_sTheTag;

};

/********  this is in another executable file *********/
std::ifstream ifs("HLAData.bin",ios::binary);
boost::archive::binary_iarchive ia(ifs);

while(!ifs.eof())
{
	CReflectAttributeValuesWrapper ravw;

	ia>>ravw;	
}

Open in new window

0
Comment
Question by:shootah
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 
LVL 53

Expert Comment

by:Infinity08
ID: 33584352
What code was used to serialize the data ?
0
 

Author Comment

by:shootah
ID: 33586654
I've attached the serialize code.
void reflectAttributeValues (
    RTI::ObjectHandle                 theObject,    
    const RTI::AttributeHandleValuePairSet& theAttributes, 
    const char                             *theTag) 
{    
/* 	m_ofStream is declared as a member variable and intialized in constructor like this:
	m_ofStream.open("HLAData.bin",ios::binary);
*/
	boost::archive::binary_oarchive oa(m_ofStream);

	CReflectAttributeValuesWrapper data;

	data.setAttributeValues(theAttributes);
	data.setObjectHandle(theObject);
	data.setTag(theTag);

	oa<<data;
	
	
	/********/
	//irrelevant code below
}

Open in new window

0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33587312
>> I've attached the serialize code.

That only places one object in the serialized file.

But when you're getting the data from the file again, you try to get more than one.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:shootah
ID: 33587751
That function (ReflectAttributeValues) is a callback function that is called with a high frequency.  So as far as I can tell, each time it is called, an object is placed in the serialized file.  I have a version of this code that writes to a text file, and just from eyeing the file, it looks like it is placing the data in sequence in the file (The file gets to about 100k after a few minutes).
Another thing is on the second iteration, the first value that is written to the first variable during deserialization is "22", which is the number in what looks like header information for each entry.

Attached is a portion of the txt version of the serialized file. 11534337 is a valid handle, 3 is the number of attributes, 186 is the "attribute handle", 4 is the size of the attribute in the attached file.  All of this works fine on the first pass.  When I call the insertion operator again, I get "22" for the first variable (the first number in the header for the next sequence).
22 serialization::archive 7 0 0 11534337 3 186 4 0 0 0 0 158 8 -60 125 -73 60 88 -43 0 0 162 176 63 -31 -126 -35 34 -90 -110 47 -65 -7 -30 -45 -44 -22 -89 -19 -64 30 -117 -17 -85 96 0 0 64 46 21 -32 103 6 58 79 63 -68 17 60 17 3 43 -19 63 -11 67 113 -7 -118 -82 -90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -33 65 -123 -66 -76 -104 108 -65 -21 -29 112 -109 -97 76 -45 -65 -103 52 -66 96 -80 -57 -32 -65 -94 -51 -96 0 -6 -32 -95 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -60 125 -73 60 88 -43 0 0 13 Tag_Name<00>
22 serialization::archive 7 0 0 11534338 3 186 4 0 0 0 0 158 8 -60 125 -73 60 -116 -117 64 0 162 176 63 -31 -126 -104 24 118 -29 21 -65 -7 -30 -30 -69 -89 16 10 -64 30 -105 85 90 0 0 0 64 45 61 36 -1 30 63 -94 63 -69 50 -119 -91 -27 -81 46 63 -12 -85 126 70 -76 -5 -19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -33 66 95 -95 104 -100 -31 -65 -21 -29 58 -98 -119 -19 -6 -65 -103 40 98 4 80 76 26 -65 -94 -57 60 71 -90 -46 -125 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -60 125 -73 60 -116 -117 64 0 13 Tag_name<00>

Open in new window

0
 
LVL 53

Accepted Solution

by:
Infinity08 earned 500 total points
ID: 33588473
>> So as far as I can tell, each time it is called, an object is placed in the serialized file

You're still creating a new boost::archive::binary_oarchive for every object, which you don't do when reading the file.

You should probably align both the serializing and de-serializing loops, to both use the same approach (either one archive instance defined outside of the loop, or an archive instance for each iteration).
0
 

Author Comment

by:shootah
ID: 33589020
Wow.  That sure was a lot of typing on my part for such a simple answer.  Thanks for the help.
0

Featured Post

Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

IntroductionThis article is the second in a three part article series on the Visual Studio 2008 Debugger.  It provides tips in setting and using breakpoints. If not familiar with this debugger, you can find a basic introduction in the EE article loc…
C++ Properties One feature missing from standard C++ that you will find in many other Object Oriented Programming languages is something called a Property (http://www.experts-exchange.com/Programming/Languages/CPP/A_3912-Object-Properties-in-C.ht…
The viewer will learn how to pass data into a function in C++. This is one step further in using functions. Instead of only printing text onto the console, the function will be able to perform calculations with argumentents given by the user.
The viewer will learn additional member functions of the vector class. Specifically, the capacity and swap member functions will be introduced.

751 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question