Is there a simple way in Python to convert integers to bytes and back?

Hi
I'm doing my new network-game server in Java again, for a python client, btw,
I have this Java helper method below to convert the message bytes into an integer, that I'd like  to verify first.

What would it look like in Python? Is there a simple way to do it that I don't know about? I know that unexpected errors can occur with casting, for Python newbs
Are bit level operations the same operators?
Is getting the first byte simple in Python also?   just  (byte)  x

Java code to work inPython:
byte SecondByte(int x) {
		return (byte) ((x & 0x0000ff00) >> 8);
	}


	static int intFromBytes(byte firstByte, byte secondByte) {
		if ((secondByte&0x80)==128) {
			//negative number
			return (int) ( (firstByte&0x000000ff)| ((secondByte&0x000000ff) << 8) | 0xffff0000 );
		} else {
			//Positive number
			return (int) ( (firstByte&0x000000ff)| ((secondByte&0x000000ff) << 8) );
			}
	}

Open in new window



Thanks
LVL 1
beavoidAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

käµfm³d 👽Commented:
I don't understand your question. An integer is just a sequence of 4 bytes. What are you converting?
0
beavoidAuthor Commented:
That is the Java bit level code I used to convert all the data my network game needed for TCP and UDP transmissions
Is it much different in Python, or do they have a quick way to do it?

Thanks
0
gelonidaCommented:
first questions:

converting 2bytes into a signed integer having values between -32768 and 32767 is not an issue.

However in order to exactly understand how to write the code it would be helpful to know, where these bytes come from and whether you use python2 or python3.

Is following assumption correct?

twobytes = network_or_socket_or_file.read(2)  # forst byte lease significant byte and second byte most significat representign a 16 bit signed itnerger?

Open in new window


Would this be  correcte expected inputs / outputs ?
amples = (
    # two bytes + expected integer value
    ( "\x00\x00", 0 ),
    ( "\xff\xff", -1),
    ( "\x34\x12", 0x1234),
    ( "\xff\x7f", 0x7fff), 
    ( "\x00\x80", -0x8000),
)

Open in new window


In that case you could use the unpack  function  ( https://docs.python.org/2/library/struct.html )
from struct import unpack
value = unpack("<h",  two_bytes)

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Upgrade your Question Security!

Your question, your audience. Choose who sees your identity—and your question—with question security.

beavoidAuthor Commented:
Thanks, this page, listing these bitwise operators for Python, leads me to suspect that my code will be just as I expect if I convert it to Python

shere

Agreed?

Thanks
0
beavoidAuthor Commented:
My Pycharm says 2016.3.3
0
aikimarkCommented:
there is a built-in (intrinsic) bytes() function.  Have you looked at that?
0
gelonidaCommented:
As I tried to write in my previous comment. The best solution depends on exactly where you receive your bytes from and on the surrounding code.
Do you receive these bytes from a file / network socket?

If yes, then you receive probably  a string (python2) or a byte string (python3) and unpack will give you immediately what you want.
Of course you can convert each byte in the byte string into an integer value, and then call the intFromBytes() function. (that you translate almost literally from java to python)

With unpack you could even 'parse' a multi byte string directly into multiple short / long integerts.
Alternatively you can use ctype structs for a sequence of bytes to be parsed.

If you re-implement code  from an existing code base written in one language into a new language then it is sometimes better to not do a word by word translation, but to use features of the new language like unpack / ctype structs.

But again all depends on the exact context of your project.

The answer might also vary slightly depending on whether you use python2 or python3

The reverse function (convert an integer or multiple integers) into a byte string can be done with the pack function from the struct module

In [35]: from struct import pack

In [36]: pack('<h', -32768)
Out[36]: b'\x00\x80'

In [37]: 

Open in new window

0
beavoidAuthor Commented:
My server will still be in Java sockets, so it will be virtually bullet proof, for me. and clients in Python. That is the extent of my project in Python v 2016.3.3
I had it completely working in Java, long E.E discussion, but I need a Python client now.
For a join message to the server,once again, I'll I assign one byte in the byte[] joinmessage for [0]=JOIN_MSG_VALUE and then [client number]
since there won't be more than 256 clients, I can leave that without a conversion. How do I specify an in-game bytes array to be of length 1024, or 256. Python has immutable issues. If join messages are in TCP, length prob won't matter for them, and in-game messages will need to be a specific size. - I just add the bytes in the appropriate order to the array (a list?)
And, for in-game messages, I'll need proper integers, for x,y destinations, etc,
And that needs to convert 4 specific bytes in the message array to an integer converter. and later, int to 4-bytes

I could use my 2 byte conversion solution from my first RTS engine,  but 2 bytes aren't best for RTS. Four are. What would a 4 byte conversion function be in Python from a server msgArray to the client? What would it look like to use already existing Python methods?
Thanks

def bytesToInt(self, fourBytes) :

   # probably a one line, something like x= fourBytes[0]*256+ fourBytes[1]*128+ fourBytes[2]*64+b[3]    
   return x
?

And also IntToBytes(self,x):     # integer to 4 byte byte array

     ?

Thanks
0
beavoidAuthor Commented:
Hi
I found this on a Python page, does it look correct? it seems good.

here on reliable stack overflow
def bytesToNumber(self,b):
    total = 0
    multiplier = 1
    for count in xrange(len(b)-1, -1, -1):
        byte = b[count]
        total += multiplier * byte
        multiplier *= 256
    return total


def numberToByteArray(self,n, howManyBytes=None):
    if howManyBytes == None:
        howManyBytes = numBytes(n)
    b = bytearray(howManyBytes)
    for count in xrange(howManyBytes-1, -1, -1):
        b[count] = int(n % 256)
        n >>= 8
    return b

Open in new window

0
gelonidaCommented:
First can you please tell us the Python version, that you are using.
Python v 2016.3.3 is probably just the version of Pycharm, but what I am interested is the python version.

You can get information about the Python executable by running following short script and posting its output.

import sys
print(sys.executable)
print(sys.version_info)

Open in new window


This information is essential, as it is required to give you the best answer depending on the python version, that you are coding for.
Is it python2 or python3? I don't use pycharm and "2016.3.3" is not conclusive  for me.


Second:

your client and server communicate via sockets, right.
In python what you send to / receive from sockets are byte strings.

I might be wrong, but I have the impression you spend time trying to implement a function, that you don't have to implement or that you should probably implement completely differently.
I think you probably do not need to implement bytesToNumber() or numberToByteArray(). Multiplying / shifting / and bitwise operations should not be needed.
Some simple pack() / unpack() commands should  be sufficient.
Alternatively byte buffers and ctype.structs might be an option.


Could you please explain the exact format of a few messages that you want to send / receive and I can show you how to code the sending / receiving in python.



For visualisation  I just show you an example of three arbitrary messages, their  'specification', the code to create such a message
and example code sending messages to a socket.

1.) a join message: (join the server)
byte[0] = message_id = JOIN_ID  (for example 0, depends on your specification)
byte[1] = client_id ( between 0 and 255)

Open in new window

2.) move message: (tell server that player moved to x, y)
byte[0] = message_id = MOVE_ID (for example 1, depends on your specification)
bytes[1] + bytes[2] = x coordinate to move to (little endian) so x = bytes[2] * 256 + bytes[1]
bytes[3] + bytes[4] = y coordinate to move to (little endian) so y = bytes[4] * 256 + bytes[3]

Open in new window

3.) speak message: (send a unicode text message to user with destination_id)
byte[0] = message_id = SPEAK_ID (for example 2, depends on your specification)
bytes[1] =my_id (id of client who wants to speak)
bytes[2] = destination_id (id of client you want to talk to)
bytes[3] + bytes[4] = text_len length of unicode text message to be sent
bytes[5] . . . bytes[5+text_len-1] the unicode bytes of the text message

Open in new window

from struct import pack

JOIN_ID = 0
MOVE_ID = 1
SPEAK_ID = 2

def mk_join_message(client_id):
    """ create a join message (always length of 2 bytes) """
    return pack("bb", JOIN_ID, client_id)

def mk_move_message(x, y):
    """ create a move message (always length of 5 bytes) """
    return pack("<bhh",MOVE_ID,  x, y)

def mk_speak_message(my_id, dest_id, text):
    """ create a speak message
        speak messages have a variable legth of 5 bytes + length of
        text message to send
    """
    text_as_bytes  = text.encode('utf-8')
    text_len = length(text_as_bytes)
    msg_head = pack("<bbbh", SPEAK_ID, my_id, dest_id, text_len)
    return msg_head + text_as_bytes


def example_session(socket):
    socket.send( mk_join_message(3))
    socket.send(mk_move_message(32768, -100))
    socket.send(mk_speak_message(3, 10, "Hello world"))

Open in new window

1
beavoidAuthor Commented:
Thanks for the version printing code

It printed out this:

/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6
sys.version_info(major=3, minor=6, micro=3, releaselevel='final', serial=0)        you're right, btw, 3.3 was the Pycharm version!

Hope that helps

Why does that matter? I don't seem to remember any Java version updates ever disabling code.
0
beavoidAuthor Commented:
Hi
An example of one of my in-game messages would be, client to server of a move message

[0] MSG_IN_GAME
[1] client number the message came from (although this could be determined by checking the ip address, but I like it in the message) no more than 256 clients
[2] MSG_MOVE
[3]..[4] two bytes combined for x destination, shouldn't be more than 65,535
[5]..[6] two bytes combined for y destination
[7] number_of_units_selected
[8....] list of every 2 byte grouping being a selected unit number, since there won't be more than 65,535 units

Seem okay?
0
gelonidaCommented:
Python had some big changes between python2 and python3 which concern amongst others the management of Unicode and byte strings. SO it might be relevant for your particular client code.

Python2 is still quite often used, though almost all new projects use python3. This incompatibility caused some head aches, but due to that incompatibility python3 got rid of some historical weight.
with modules like six or future it is possible to write code, that runs with py2 and py3.

Great to know you're using 3.6 so writing BW compatible code is not a concern.

Back to the messages:

In your upper example you have two messages, right?
the first message in byte 0 and 1
the second message in byte 2 to 7 + 2 * number_of_units_selected

As number_of_units_selected  is just a byte you have a max of 255 units selected?

so for sending you would need a function like

def mk_in_game_msg(client):
    """
        :param client   an integer value between 0 and 255 specifying the client 
    """

Open in new window


and for the move you would need a function like?

def mk_move_msg(x, y, selected):
    """
        :param x   an integer specifying the x coordinate between 0 and 65535
        :param y   an integer specifying the x coordinate between 0 and 65535
        :param selected  a list of up to 255 unit ids. each unit id is a number between 0 and 65535
    """

Open in new window


Is that it?

If yes and can show you some potential implementation.


what kind of messages does your client receive?
as messages do not have a fixed length it seems, that you have to read the first byte of a message in order to decide
how many further bytes you have to read and in case of a move message you have to read the message until  number_of_units_selected in order to be able to read the entire message, that's it?

so you had to implement a function like

def get_next_message(socket):
    """ :param socket: socket to read a message from
    """"
   
and which yould return you a message byte string, a tuple of parameters, a named tuple, a struct, or a python object.

Do I understand correctly?
0
beavoidAuthor Commented:
Yes, I think U got it all. This sounds all standard

The client receives game state updates from the server after every server cycle.

byte code format From server to client . .

[0] -message type   IN_GAME_UPDATE
[1. . . ] - however I decide to encode my game-state
 

Your thinking of my message format is correct. byte[0] alone is the message type and then the necessary encodings.

Your encoding is likely exactly what I need. Look forward to some pointers.

Thanks
0
beavoidAuthor Commented:
Articles on Google state there should be no concern as to the performance of a Python network-game system's integrity compared to Java's

?

That might also greatly simplify conversions and more.
So, I'll start transferring my experience on a working RTS server system into Python tomorrow.

Would you advise making a static class called Constants to contain all the game's constants, like Message id's, message type, unit max count, game unit type ID's, etc all relevant constants are accessible from anywhere?

Thanks
0
gelonidaCommented:
creating a  class with static members for constants is a reasonoble option.
1
beavoidAuthor Commented:
Thanks
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.