Link to home
Start Free TrialLog in
Avatar of StephenJordan
StephenJordan

asked on

using boost::serialization with an array of bytes

I am using a library which has an image class, where the image is represented as an array of unsigned char. I need to save a set of images to a single file along with some other data. For this I'm using boost::serialization which seems to work well in general. This is using boost 1.39 and MS Visual Studio 2008.

I wrote a non-intrusive split serialization method for the image class. The image class is taken from the SFML library, the methods I use with it should be self-explanatory.

BOOST_SERIALIZATION_SPLIT_FREE(sf::Image)

namespace boost
{
      namespace serialization
      {
            template<class Archive>
            void save(Archive & ar, const sf::Image & img, unsigned int version)
            {
                  const unsigned char* pixels = img.GetPixelsPtr();
                  uint w = img.GetWidth();
                  uint h = img.GetHeight();

                  ar << w;
                  ar << h;
                  uint sz = w*h*4;
                  ar << serialization::make_array(pixels, sz);
            }
            
            template<class Archive>
            void load(Archive & ar, sf::Image & img, unsigned int version)
            {
                  uint w, h;
                  ar >> w;
                  ar >> h;      
                  
                  uint sz = w*h*4;
                   unsigned char* pixels = new unsigned char[sz];      
                  ar >> serialization::make_array(pixels, sz);

                  img.LoadFromPixels(w, h, pixels);
                  delete pixels;
            }
      
      }
}

When loading some files with this method I can get an exception thrown when loading the byte array, the exception is thrown from basic_binary_iprimitive.hpp line 161:

template<class Archive, class Elem, class Tr>
inline void
basic_binary_iprimitive<Archive, Elem, Tr>::load_binary(
    void *address,
    std::size_t count
){
    // note: an optimizer should eliminate the following for char files
    std::streamsize s = count / sizeof(Elem);
    std::streamsize scount = m_sb.sgetn(
        static_cast<Elem *>(address),
        s
    );
    if(scount != s) // EXCEPTION THROWN HERE
        boost::serialization::throw_exception(
            archive_exception(archive_exception::stream_error)
        );
    // note: an optimizer should eliminate the following for char files
    s = count % sizeof(Elem);
    if(0 < s){
//        if(is.fail())
//            boost::serialization::throw_exception(
//                archive_exception(archive_exception::stream_error)
//        );
        Elem t;
        scount = m_sb.sgetn(& t, 1);
        if(scount != 1)
            boost::serialization::throw_exception(
                archive_exception(archive_exception::stream_error)
            );
        std::memcpy(static_cast<char*>(address) + (count - s), &t, s);
    }
}

For some reason the value scount is always less than s. Strangely this does not seem to be true for all images, but does occur often. The same exception is also thrown when serializing the byte array with other methods, such as using a std::vector, or explicitly writing a loop to serialize each byte. (in which case scount==0 and s==1 above.

This is using a boost::archive::binary_iarchive/binary_oarchive and a normal c++ std::ifstream/ofstream, so no compression should be happening yet.

Does anyone have suggestions as to what could be happening?

I also note the comment in the boost header above:  

// note: an optimizer should eliminate the following for char files

Does anyone know what this means and if is relevant? There seems to be a class in boost called boost::serialization::use_array_optimization but I can't find any reference to how you actually use it.
   




Avatar of afumedo
afumedo


- first, check that you are NOT USING  ( boost::archive::text_oarchive ) to save the array.

instead use the native binary archive to save and load your content.

boost::archive::binary_oarchive // saving
boost::archive::binary_iarchive // loading

Look at these two urls:

http://www.boost.org/doc/libs/1_42_0/libs/serialization/doc/tutorial.html
http://www.boost.org/doc/libs/1_35_0/libs/serialization/doc/archives.html

Best wishes,

Ahmed Bedier
ASKER CERTIFIED SOLUTION
Avatar of ambience
ambience
Flag of Pakistan image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of StephenJordan

ASKER

That's cracked it thanks. I was using a binary archive but hadn't set the stream to binary mode.