StephenJordan
asked on
using boost::serialization with an array of bytes
I am using a library which has an image class, where the image is represented as an array of unsigned char. I need to save a set of images to a single file along with some other data. For this I'm using boost::serialization which seems to work well in general. This is using boost 1.39 and MS Visual Studio 2008.
I wrote a non-intrusive split serialization method for the image class. The image class is taken from the SFML library, the methods I use with it should be self-explanatory.
BOOST_SERIALIZATION_SPLIT_ FREE(sf::I mage)
namespace boost
{
namespace serialization
{
template<class Archive>
void save(Archive & ar, const sf::Image & img, unsigned int version)
{
const unsigned char* pixels = img.GetPixelsPtr();
uint w = img.GetWidth();
uint h = img.GetHeight();
ar << w;
ar << h;
uint sz = w*h*4;
ar << serialization::make_array( pixels, sz);
}
template<class Archive>
void load(Archive & ar, sf::Image & img, unsigned int version)
{
uint w, h;
ar >> w;
ar >> h;
uint sz = w*h*4;
unsigned char* pixels = new unsigned char[sz];
ar >> serialization::make_array( pixels, sz);
img.LoadFromPixels(w, h, pixels);
delete pixels;
}
}
}
When loading some files with this method I can get an exception thrown when loading the byte array, the exception is thrown from basic_binary_iprimitive.hp p line 161:
template<class Archive, class Elem, class Tr>
inline void
basic_binary_iprimitive<Ar chive, Elem, Tr>::load_binary(
void *address,
std::size_t count
){
// note: an optimizer should eliminate the following for char files
std::streamsize s = count / sizeof(Elem);
std::streamsize scount = m_sb.sgetn(
static_cast<Elem *>(address),
s
);
if(scount != s) // EXCEPTION THROWN HERE
boost::serialization::thro w_exceptio n(
archive_exception(archive_ exception: :stream_er ror)
);
// note: an optimizer should eliminate the following for char files
s = count % sizeof(Elem);
if(0 < s){
// if(is.fail())
// boost::serialization::thro w_exceptio n(
// archive_exception(archive_ exception: :stream_er ror)
// );
Elem t;
scount = m_sb.sgetn(& t, 1);
if(scount != 1)
boost::serialization::thro w_exceptio n(
archive_exception(archive_ exception: :stream_er ror)
);
std::memcpy(static_cast<ch ar*>(addre ss) + (count - s), &t, s);
}
}
For some reason the value scount is always less than s. Strangely this does not seem to be true for all images, but does occur often. The same exception is also thrown when serializing the byte array with other methods, such as using a std::vector, or explicitly writing a loop to serialize each byte. (in which case scount==0 and s==1 above.
This is using a boost::archive::binary_iar chive/bina ry_oarchiv e and a normal c++ std::ifstream/ofstream, so no compression should be happening yet.
Does anyone have suggestions as to what could be happening?
I also note the comment in the boost header above:
// note: an optimizer should eliminate the following for char files
Does anyone know what this means and if is relevant? There seems to be a class in boost called boost::serialization::use_ array_opti mization but I can't find any reference to how you actually use it.
I wrote a non-intrusive split serialization method for the image class. The image class is taken from the SFML library, the methods I use with it should be self-explanatory.
BOOST_SERIALIZATION_SPLIT_
namespace boost
{
namespace serialization
{
template<class Archive>
void save(Archive & ar, const sf::Image & img, unsigned int version)
{
const unsigned char* pixels = img.GetPixelsPtr();
uint w = img.GetWidth();
uint h = img.GetHeight();
ar << w;
ar << h;
uint sz = w*h*4;
ar << serialization::make_array(
}
template<class Archive>
void load(Archive & ar, sf::Image & img, unsigned int version)
{
uint w, h;
ar >> w;
ar >> h;
uint sz = w*h*4;
unsigned char* pixels = new unsigned char[sz];
ar >> serialization::make_array(
img.LoadFromPixels(w, h, pixels);
delete pixels;
}
}
}
When loading some files with this method I can get an exception thrown when loading the byte array, the exception is thrown from basic_binary_iprimitive.hp
template<class Archive, class Elem, class Tr>
inline void
basic_binary_iprimitive<Ar
void *address,
std::size_t count
){
// note: an optimizer should eliminate the following for char files
std::streamsize s = count / sizeof(Elem);
std::streamsize scount = m_sb.sgetn(
static_cast<Elem *>(address),
s
);
if(scount != s) // EXCEPTION THROWN HERE
boost::serialization::thro
archive_exception(archive_
);
// note: an optimizer should eliminate the following for char files
s = count % sizeof(Elem);
if(0 < s){
// if(is.fail())
// boost::serialization::thro
// archive_exception(archive_
// );
Elem t;
scount = m_sb.sgetn(& t, 1);
if(scount != 1)
boost::serialization::thro
archive_exception(archive_
);
std::memcpy(static_cast<ch
}
}
For some reason the value scount is always less than s. Strangely this does not seem to be true for all images, but does occur often. The same exception is also thrown when serializing the byte array with other methods, such as using a std::vector, or explicitly writing a loop to serialize each byte. (in which case scount==0 and s==1 above.
This is using a boost::archive::binary_iar
Does anyone have suggestions as to what could be happening?
I also note the comment in the boost header above:
// note: an optimizer should eliminate the following for char files
Does anyone know what this means and if is relevant? There seems to be a class in boost called boost::serialization::use_
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
That's cracked it thanks. I was using a binary archive but hadn't set the stream to binary mode.
- first, check that you are NOT USING ( boost::archive::text_oarch
instead use the native binary archive to save and load your content.
boost::archive::binary_oar
boost::archive::binary_iar
Look at these two urls:
http://www.boost.org/doc/libs/1_42_0/libs/serialization/doc/tutorial.html
http://www.boost.org/doc/libs/1_35_0/libs/serialization/doc/archives.html
Best wishes,
Ahmed Bedier