Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
• Status: Solved
• Priority: Medium
• Security: Public
• Views: 431

# endian conversion using algorithms and template

In order to convert a value from big endian to little endian, I could convert value to a native value, assemble the number, using the
proper multipliers.   So now - doing this from memory I could do:

unsigned char big[4]; // single value in big-endian order
unsigned long x;
x = ((big[0] * 0x100UL + big[1]) * 0x100 + big[2]) * 0x100 + big[3];

I'm sure there's a combination of algorithms and templates that could achieve this but that's where I'm lacking / hence asking for help converting from big to little and vice versa.  Something akin to:

template<typename T>
T swap_endian( T begin, T end )
{
// some algorithm .. maybe
}

int main()
{
}

0
forums_mp
• 10
• 8
• 3
3 Solutions

Commented:
template<typename T> T swap_endian(T input)
{
//assuming this is primitives only
int s = sizeof(T);
T out = 0;
unsigned char* pos = &out;
for(int i = s - 1; i >= 0; i--)
{
*pos = (unsigned char)(((unsigned char*)&input)[i]);
pos++;
}
return out;
}

Basically, you're just treating input & out as though they are byte arrays, and swapping the bytes.  This will only work for primitive types, though.
0

Commented:
Hi forums_mp,
Are you trying to do this so that you can send data via socket?
If so, you can try using 'ntohs()', 'ntohl()', 'htons()', and 'htonl() functions.

Cheers!
0

Commented:
forums_mp,
Also check out the following link for a wrapper function:

0

Author Commented:
guitaristx:   I might have missed STL in my initial post but I'd like to use an STL algorithm.  If the answer is not plausible.   Then I understand.

Axter:
No, i'm on  a vxWorks platform.  I've got a powepc that uses big endian and a pci bus that's little endian.
so I send a

struct stuff {
//
};

as data it arrives endian 'swapped'
0

Commented:
Did you look at the code in the last link I posted?

0

Author Commented:

I did and my question becomes.  What if I'm not using the 'common' unsigned data type.   I'm sending structs.   Will this work?
For some reason, I'm thinking that one solution would be std::bitset.  Admittidely,  I need to read more on bitset but it appears that I could treat the structs as a bucket of bits and convert them.
0

Commented:
Not easily, since C++ doesn't have an "easy" way to treat structs as aggregate types.  Are you sure that you understand the problem completely?  Let me give you a for-instance:

struct str
{
int x;
char y;
char n;
short z;
}

sizeof(str) should equal 8.  Now, let's see how that would look in memory:
(each byte of each member is separated by one space)
x1 x2 x3 x4 y n z1 z2 //on a big-endian system

This struct on a little-endian system would look like this:
x4 x3 x2 x1 y n z2 z1

where it seems that you're expecting it to be:
z2 z1 n y x4 x3 x2 x1

You can't use a template to automagically figure out what the members are, what their types and lengths are, and convert.  The run-time environment doesn't see a struct as anything more than a block of memory, but just reversing the bytes in the entire struct is going to yield gibberish.  You must convert the structs by-hand, unfortunately.  Templates can't help you here.
0

Author Commented:
Sounds good.  I'll have to pull out the PCI bus analyzer again and review the data once more.  In the  interim,  thanks to you an axter.  The current support for floats doubles, will come in handy at some point.

Let me ask this.  Outside of templates and all that.   Wouldn't bitset be appropriate in terms of treating a struct as a bucket of bits, instead of doing it by 'hand'?
0

Commented:
It depends on why you need bitwise access.  Could you expound a bit more?  The main problem I see is converting your structs back & forth from bitsets.
0

Author Commented:

guitaristx:
I'll have to get back with you on that - most likely monday.  I know there's an endian issue but I'll look at the bit pattern more closely.

///
On an entirely different issue.  Consider:

class X2{
public:
X2() {}
void compute(size_t len = 4096)
{
// check to ensure len is aligned on a 4k boundary
}
};

// later
int main()
{
X2 x;
x.compute(10 );  // bad user passed 10..
// valid values 4096, 8092 etc. up to 16Meg max
}

Within the member function compute.  I'd like to validate that the passed paramater is aligned on a 4k boundary.   How would I achieve this?  I dont think my current approach is 'the best'.
0

Author Commented:

Somehow I missed the fact that I had this open question.   guitaristx  here's my problem.  Which is duplicated in a different post but I think I'll close that other post now.

unsigned int  msg_id;
unsigned int  msg_count;
unsigned int  msg_size;
unsigned int  msg_chksum;
};
struct t01_msg
{
signed int       xdircosine  :16;  // X Direction Cosine  - LSB 2^-15
signed int       ydircosine  :16;  // Y Direction Cosine  - LSB 2^-15
signed int       zdircosine  :16;  // Z Direction Cosine  - LSB 2^-15
// more
};

struct outgoing_msg {
t01_msg  t01_msg_;
// more
};

# endif

class main_win_impl : public main_win
{
Q_OBJECT
main_win_impl(QWidget *parent);
~ main_win_impl();

// later
private:
QSocket         *socket;
outgoing_msg  mPacket;
};

// main_win_impl.cpp
// later
#include <algorithm> //required for std::swap

#define ByteSwap5(x) ByteSwap((unsigned char *) &x,sizeof(x))

static void ByteSwap(unsigned char * b, int n)
{
register int i = 0;
register int j = n-1;
while (i<j)
{
std::swap(b[i], b[j]);
i++, j--;
}
}

main_win_impl::main_win_impl(QWidget *parent)
: main_win (parent)
{
// later
socket = new QSocket();
}

void main_win::send_message()
{
if ( socket->state() == QSocket::Connected )
{
//   for (int idx(0); idx < sizeof(mPacket); ++idx)
//      ByteSwap5(  );
socket->WriteBlock( (char *)&mPacket, sizeof(mPacket));
}
}
0

Commented:
I think I see your problem.  The issue here is that you can't just byte-swap the entire struct, or you'll get gibberish.  You'll have to do it by-hand for each member of the struct, using the functions that Axter recommended:
>> You can try using 'ntohs()', 'ntohl()', 'htons()', and 'htonl() functions.

The fact that you're using bit fields doesn't really matter, since they're 16 bits long anyway.  I would recommend using signed short, rather than signed int, in your t01_msg struct:
struct t01_msg
{
signed short       xdircosine;
signed short       ydircosine;
signed short       zdircosine;
// more
};

Signed shorts are 16 bits in length, and it will simplify the conversions.  As far as swapping the byte-order, it should look like this for the above struct:

void toBigEndian(t01_msg* obj)
{
obj->xdircosine = htons(obj->xdircosine);
obj->ydircosine = htons(obj->ydircosine);
obj->zdircosine = htons(obj->zdircosine);
//more
}

I suppose that the million-dollar question is:
are you using bit fields that are not 8, 16, or 32 bits in length?  If not, you should be all set.  If so, then we've still got a bit more work to do.
0

Author Commented:
>>>> I suppose that the million-dollar question is:
>>>> are you using bit fields that are not 8, 16, or 32 bits in length?

That's the thing.  I'm 'stuck' using definitions like this:

typedef double tag_word_t;

typedef enum
{
sby                 = 1,
a_to_g            = 2,
a_to_a             = 4,
bsigh               = 5,
lns                   = 6
} sys_mode;

struct ts1_data
{
tag_word_t       tag_wd;
sys_mode         ofmode      : 4;
bool                 ofiralg         : 1;
// more
};

{
unsigned short size;
unsigned short id;
unsigned short count;
unsigned short chksum;
};

struct outgoing_data
{
ts1_data   t1_data_;
};

I'm trying to work with a host of solutions to this problem.   Haven't had a chance to really investigate but one approach I'd like to FIRST investigate is akin to:

1. allocate an unsigned char buffer large enough to hold the message.
2. Read the message into the buffer
3. Convert the â€˜endianismâ€™ of the data.
4. Load each member of the model structure FROM the buffer.

For outgoing/transmitted messages.  I'll:
1. allocate an unsigned char buffer large enough to hold the message.
2. Load each member of the model structure INTO the buffer.
3. Convert the â€˜endianismâ€™ of the data.
4. Transmit message from the buffer

0

Commented:
Do you have control of both ends of the communication, or just one?  If you've got control of both ends, then you have the freedom to define the communication protocol however you see fit.  However, if you've only got control of one end of the communication, chances are that there exists a specification for the protocol somewhere.

0

Author Commented:
I have control. The basic premise centers around the fact that I'm a client sending a server data.  because of 'endian issues' with client and server, one needs to handle endianness.  In that regard, I was given a plethora of advice with regard to what works best - more specifically 'portable code' -  in a situation like this.  The approach I thought about first investigating involves what's described above.

The claim is that 'char arrays' is portable/safe in this situation.  The one thing that confuses me about the approach descibed is.  If platform a (transmitter) is 16 bit integers.  platform b (receiver) is 32 bits
If I allocate a char array buffer large enough on platform a.  transmit the data to b.   b (prior to converting to the appropriate struct) sees a buffer thats 16 bits.  But b is 32 bit integers hence the buffer is not large enough.

Am I'm way off here?
0

Commented:
First, I think you've maybe believed that you can just dump your struct into a byte buffer and send it across the wire, which is a Bad Thing.  You're not guaranteed that the two platforms pack structs the same way, and this approach will cause you more headache than it will solve.

The best approach is to let the protocol between the two platforms determine the endian-ness and the format of the data.  What you'll need to do is determine which platform, a or b, is easier to do the byte-swapping (or just pick one).  Then, you'll tailor the protocol so that that platform (say, a, for example) _always_ transmits and receives data according to the endian-ness of the other platform.

[DISCLAIMER - the following is an explanation of how I would approach this problem.  It is not necessarily the "right way," just a way that could work]
Begin by examining the data structures on both sides.  (I believe they're declared the exact same way, right?)  Your protocol should be a "least common denominator" (for lack of a better term) of these data structures.
Create a data structure that will be the 'template' for how data is passed from one platform to the other.
Pad all values out to the nearest byte, 2 bytes, or 4 bytes -
1 bit becomes one byte, 4 bits becomes one byte, 13 bits becomes 2 bytes, 21 bits becomes 4 bytes.
Verify that both hosts read & write to this data structure using the same endian-ness (I'd use big-endian [motorola], since it's considered "network byte order").

Now that the data is defined, you don't really need the data structure we defined above (per se), since you've already got your native data structures.  Reading data from one host simply consists of reading the appropriate number of bytes, dealing with the endian-ness (as necessary), and placing it in the data structure that already exists.

For example:
Let's say that the following data structure (from above) is defined on both hosts:

struct ts1_data
{
tag_word_t       tag_wd;
sys_mode         ofmode      : 4;
bool                 ofiralg         : 1;
};

The fields ofmode and ofiralg will both be padded to 1 byte each when they're transmitted to the other host.
So, whenever reading from one host to another, you'll read 8 bytes first, for tag_wd, and convert to the appropriate endian-ness (if necessary).  Then, you'll read one byte and place the value into ofmode (since no byte-swapping needs to occur for a 1-byte value).  Finally, read the last byte and place the value into ofiralg.
0

Author Commented:

>>>>>>>>First, I think you've maybe believed that you can just dump your struct into a byte buffer and send it across the wire, which is a Bad Thing.  You're not guaranteed that the two platforms pack structs the same way, and this approach will cause you more headache than it will solve.

Come to think of it.  Perhaps I'm confused.  So again consider.

struct ts1_data
{
tag_word_t       tag_wd;
sys_mode         ofmode      : 4;
bool                 ofiralg         : 1;
};

I'm not 'dumping' a struct.  I'm just dumping the contents of the struct.   IOW to dump the ts1_data struct to the buffer.   I envision.

8 byte for tag_wd, dump 4 bits (ofmode), dump 1 bit (ofiralg).

Where I'm confused with this approach is:   ofmode and ofiralg can be combined in 1 eight bit 'buffer'.   Does each get their own 8 bit buffer or are they combined?

I think the answer though lies in a response I received when I first investigated this:

If you always convert a double to 8 bytes in a known encoding, and the bools into a
certain sequence of bytes, then it should work...you're essentially
not just "copying the struct to the bytestream" but enforcing your own
encoding on it.

Now how does one 'enforce' their own encoding on 8bytes etc.

0

Commented:
>> If you always convert a double to 8 bytes in a known encoding, and the bools into a
>> certain sequence of bytes, then it should work...you're essentially
>> not just "copying the struct to the bytestream" but enforcing your own
>> encoding on it.

This is a very good explanation.

>> Now how does one 'enforce' their own encoding on 8bytes etc.

I'll explain by example.  These are all for-instance examples, and are definitely not the ONLY way to do it:
"All C/C++ 'double' values will be transmitted in big-endian format using IEEE-754"
"All boolean values will be one byte in length, with zero indicating false and all other values indicating true"

What's important to remember is that you have complete control over the data transmission, but you must _exert_ that control.  If you can't explain what each bit of data represents, you're not enforcing an encoding.
0

Author Commented:

After much investigation, I perused the web and found an implementation of ostringstream that's interesting.  So now consider:

#include <ostream>
namespace jngcomp {  namespace utils
{
// Output string buffer class
class stringbuf : public std::streambuf
{
public:
stringbuf(char* buffer, int buflen)
: m_buffer(buffer), m_buflen(buflen)
{
setp(m_buffer, m_buffer + m_buflen);
}
char* getpptr()
{
return pptr();
}
void setpptr(char* p)
{
setp(p, m_buffer + m_buflen);
}
protected:
virtual int_type overflow(int_type c)
{
return EOF;
}
char* m_buffer;
int m_buflen;
};

// Output string stream class which is suppled with a buffer to write to.
// Ie, stream equivalent of sprintf.
class ostringstream : public std::ostream
{
public:
ostringstream(char* buffer, int buflen)
: m_buf(buffer, buflen), std::ostream(&m_buf)
{
}
char* getpptr()
{
return m_buf.getpptr();
}
void setpptr(char* p)
{
m_buf.setpptr(p);
}
protected:
stringbuf m_buf;
};
}}

int main()
{
test t_;
t_.idx = 15;
t_.jdx = 25;

char buffer[10];
jngcomp::utils::ostringstream strm(buffer, sizeof(buffer));
strm << t_.idx ;
std::cout << buffer[0] << std::endl;
std::cout << buffer[1] << std::endl;
}

What's interesting to me is the output is
1
5

With this approach how does one get 15 to spread across 4 bytes withouth having to do
strm << 0 << 0 << t_.idx; ?

0

Author Commented:

By the way here's the test struct.

struct test {
int               idx   : 32;
unsigned int jdx   : 32;
bool            kdx   : 1;
};
0

Commented: