Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Boost Serialization/Deserialization techniques?

Posted on 2010-09-01
6
Medium Priority
?
1,360 Views
Last Modified: 2012-05-10
I want to use the boost serialization library to serialize data from a network source to a binary archive.  The data is heterogenous and I have wrapper classes that will take care of any special processing needed to serialize and deserialize.  I've created a test case and it seems to write the data correctly but deserialization returns garbage after the second iteration. I have attempted to just open the file stream and recreate the wrapper class object.  The first pass seems to work fine and the data looks good.  After the second iteration I receive garbage back from the boost library.  So I feel like I am not using this library correctly.  Are there any suggestions on how to deserialize a large amount of heterogenous data using boost?  The idea is to serialize a series of these wrapper classes depending on what is received from the network and then recreate this data at another time.  I have provided the code to the wrapper class and the deserialization code.
class CReflectAttributeValuesWrapper{

	friend class boost::serialization::access;

	template<class Archive>
	void save(Archive & ar, const unsigned int version) const
	{
		//save attribute handle value pair set
		RTI::ULong ulLengthBytes = 0;
		AttributeHandle hAttribute;
		char* pszValue = 0;
		stringstream ss;
		RTI::ULong size = m_pAttributes->size();
		static int count = 0;

		ss<<"Object Count: "<<++count<<endl;

		ar<<count;

		ss<<"Object handle: "<<m_hObjectHandle<<endl;

		ar << m_hObjectHandle;

		ss<<"attribute set size is "<<size<<endl;

		ar << size;

		for(unsigned long i = 0; i < size; i++)
		{			
			ulLengthBytes = m_pAttributes->getValueLength(i);
			pszValue = new char[ulLengthBytes];
			m_pAttributes->getValue(i,pszValue,ulLengthBytes);

			hAttribute = m_pAttributes->getHandle ((RTI::ULong)i);

			ss<<"serializing attribute handle whose value is "<<(unsigned long)hAttribute;
			ss<<" whose length is "<<(unsigned long)ulLengthBytes<<endl;
			ss<<"serializing attribute ";

			ss<<endl;

			ar << hAttribute;
			ar << ulLengthBytes;
			ar << boost::serialization::make_array(pszValue,ulLengthBytes);
		}

.....



		ss<<"tag is "<<m_sTheTag<<endl;

		CDiagnosticLogfile::writeMessage(CDiagnosticLogfile::DEBUG,ss.str(),__FILE__,__LINE__);
		
		ar<<m_sTheTag;
	}

	template<class Archive>
	void load(Archive & ar, const unsigned int version)
	{
		RTI::AttributeHandleValuePairSet* pAttributes;
		RTI::ULong ulLengthBytes = 0;
		AttributeHandle hAttribute;
		stringstream ss;		
		int objNum = -1;

		RTI::ULong size;

		ar >> objNum;

		ss<<"Object Number "<<objNum<<endl;

		ar >> m_hObjectHandle;

		ss<<"Object handle: "<<m_hObjectHandle<<endl;

		ar >> size;

		ss<<"attribute set size is "<<size<<endl;

		pAttributes = RTI::AttributeSetFactory::create (size);
		
		cout<<ss.str();

		ss.str("");

		for(unsigned long i = 0; i < size; i++)
		{
			ar >> hAttribute;
			ss<<"deserializing attribute handle whose value is "<<(unsigned long)hAttribute;

			ar >> ulLengthBytes;
			ss<<" whose length is "<<(unsigned long)ulLengthBytes<<endl;

			char* pszValue = new char[ulLengthBytes];

			ar >> boost::serialization::make_array(pszValue,ulLengthBytes);

			ss<<endl;
			pAttributes->add (hAttribute, pszValue,ulLengthBytes);
			cout<<ss.str();

			ss.str("");
		}

		ar>>m_sTheTag;

		ss<<"tag is "<<m_sTheTag<<endl;

        cout<<ss.str();
		
	}
	BOOST_SERIALIZATION_SPLIT_MEMBER()

public:
	CReflectAttributeValuesWrapper(){}
	~CReflectAttributeValuesWrapper(){}

	void setAttributeValues(const RTI::AttributeHandleValuePairSet& attributes)
	{
		m_pAttributes = const_cast <RTI::AttributeHandleValuePairSet*> (&attributes);
	}
	
	void setObjectHandle(RTI::ObjectHandle handle){m_hObjectHandle = handle;}
	void setTag(const char* tag){m_sTheTag = const_cast<char*>(tag);}

	RTI::AttributeHandleValuePairSet* getAttributeValues(){return m_pAttributes;}
	RTI::ObjectHandle getObjectHandle(){return m_hObjectHandle;}
	string getTag(){return m_sTheTag;}

private:

	RTI::AttributeHandleValuePairSet* m_pAttributes;
	RTI::ObjectHandle m_hObjectHandle;
	string m_sTheTag;

};

/********  this is in another executable file *********/
std::ifstream ifs("HLAData.bin",ios::binary);
boost::archive::binary_iarchive ia(ifs);

while(!ifs.eof())
{
	CReflectAttributeValuesWrapper ravw;

	ia>>ravw;	
}

Open in new window

0
Comment
Question by:shootah
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
6 Comments
 
LVL 53

Expert Comment

by:Infinity08
ID: 33584352
What code was used to serialize the data ?
0
 

Author Comment

by:shootah
ID: 33586654
I've attached the serialize code.
void reflectAttributeValues (
    RTI::ObjectHandle                 theObject,    
    const RTI::AttributeHandleValuePairSet& theAttributes, 
    const char                             *theTag) 
{    
/* 	m_ofStream is declared as a member variable and intialized in constructor like this:
	m_ofStream.open("HLAData.bin",ios::binary);
*/
	boost::archive::binary_oarchive oa(m_ofStream);

	CReflectAttributeValuesWrapper data;

	data.setAttributeValues(theAttributes);
	data.setObjectHandle(theObject);
	data.setTag(theTag);

	oa<<data;
	
	
	/********/
	//irrelevant code below
}

Open in new window

0
 
LVL 53

Expert Comment

by:Infinity08
ID: 33587312
>> I've attached the serialize code.

That only places one object in the serialized file.

But when you're getting the data from the file again, you try to get more than one.
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:shootah
ID: 33587751
That function (ReflectAttributeValues) is a callback function that is called with a high frequency.  So as far as I can tell, each time it is called, an object is placed in the serialized file.  I have a version of this code that writes to a text file, and just from eyeing the file, it looks like it is placing the data in sequence in the file (The file gets to about 100k after a few minutes).
Another thing is on the second iteration, the first value that is written to the first variable during deserialization is "22", which is the number in what looks like header information for each entry.

Attached is a portion of the txt version of the serialized file. 11534337 is a valid handle, 3 is the number of attributes, 186 is the "attribute handle", 4 is the size of the attribute in the attached file.  All of this works fine on the first pass.  When I call the insertion operator again, I get "22" for the first variable (the first number in the header for the next sequence).
22 serialization::archive 7 0 0 11534337 3 186 4 0 0 0 0 158 8 -60 125 -73 60 88 -43 0 0 162 176 63 -31 -126 -35 34 -90 -110 47 -65 -7 -30 -45 -44 -22 -89 -19 -64 30 -117 -17 -85 96 0 0 64 46 21 -32 103 6 58 79 63 -68 17 60 17 3 43 -19 63 -11 67 113 -7 -118 -82 -90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -33 65 -123 -66 -76 -104 108 -65 -21 -29 112 -109 -97 76 -45 -65 -103 52 -66 96 -80 -57 -32 -65 -94 -51 -96 0 -6 -32 -95 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -60 125 -73 60 88 -43 0 0 13 Tag_Name<00>
22 serialization::archive 7 0 0 11534338 3 186 4 0 0 0 0 158 8 -60 125 -73 60 -116 -117 64 0 162 176 63 -31 -126 -104 24 118 -29 21 -65 -7 -30 -30 -69 -89 16 10 -64 30 -105 85 90 0 0 0 64 45 61 36 -1 30 63 -94 63 -69 50 -119 -91 -27 -81 46 63 -12 -85 126 70 -76 -5 -19 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -33 66 95 -95 104 -100 -31 -65 -21 -29 58 -98 -119 -19 -6 -65 -103 40 98 4 80 76 26 -65 -94 -57 60 71 -90 -46 -125 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 63 -16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 -60 125 -73 60 -116 -117 64 0 13 Tag_name<00>

Open in new window

0
 
LVL 53

Accepted Solution

by:
Infinity08 earned 2000 total points
ID: 33588473
>> So as far as I can tell, each time it is called, an object is placed in the serialized file

You're still creating a new boost::archive::binary_oarchive for every object, which you don't do when reading the file.

You should probably align both the serializing and de-serializing loops, to both use the same approach (either one archive instance defined outside of the loop, or an archive instance for each iteration).
0
 

Author Comment

by:shootah
ID: 33589020
Wow.  That sure was a lot of typing on my part for such a simple answer.  Thanks for the help.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

When writing generic code, using template meta-programming techniques, it is sometimes useful to know if a type is convertible to another type. A good example of when this might be is if you are writing diagnostic instrumentation for code to generat…
Go is an acronym of golang, is a programming language developed Google in 2007. Go is a new language that is mostly in the C family, with significant input from Pascal/Modula/Oberon family. Hence Go arisen as low-level language with fast compilation…
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
Suggested Courses

722 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question