My Python RTS Server start needs some nudges!

Hi
My Python RTS server reboot from my finished Java one needs some clarifications.

The example networking code at this Python page doesn't work, when it clearly should, code below

here

The line with bind() is where it raises exceptions and it's baffling me! - cut and paste!
My code that won't compile is...
import socket, sys


if __name__ == "__main__":

    print('__main__')
    port_num=1234

    try:


        # Create a TCP/IP socket
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

        # Bind the socket to the port
        server_address = ('localhost', port_num)
        print ('starting up on %s port %s', server_address)
        sock.bind(server_address)

    except:
        print('Error')
        exit()

Open in new window


Did I miss a step in pyCharm settings?

Thanks
LVL 1
beavoidAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

gelonidaCommented:
Code looks correct but incomplete to me. What's your error message?

using try / except will hide the Exception, that might help you understanding the issue.
instead of exit() you could write raise that will re-raise the original exception and help you understand.

If I execute the code it works. Well it binds the socket and exits, but that's all that you wrote.

Please note, that there is one issue with sockets, that is not python specific, but that you should treat specifically.
When your program stops and you call it immediately again, then you will probably get an error of type.
OSError: [Errno 98] Address already in use

Open in new window

After a certain time (~30s perhaps. I never measured or read the spec) the script would work again.

This is standard behavoiur for sockets and can be solved by explicitely saying, that you want to reuse the listening socket address again. after creating a socket you have to set a socket option:
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

Open in new window

So far I never encountered a scenario where I did not want to set this option.


Look at following simple example:
import socket

if __name__ == "__main__":
    port_num = 1234

    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    # next line allows to reuse the socket address without the annoying timeout
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    
    server_address = ('localhost', port_num)
    print ('starting up on %s port %s', server_address)
    sock.bind(server_address)

    sock.listen(5) # max 5 connections backlog

    while True:
        print('waiting for connection')
        connection, client_address = sock.accept()
        connection.send("hello".encode())
        data = connection.recv(10) # read max 10 bytes
        print("got %r", data)
        connection.send("bye".encode())
        connection.close()

Open in new window


In order to test you can telnet to localhost 1234
see the hello message, type a small text and press enter, see the bye message and see that the connection is closing again.

You will also see that above code handles only one connection at a time. so if you start two telnets, then the second telnet is blocking until the first connection is closed.


Here  some generic question in order to help you choose the right setup for your project  before you start coding:

How does your server work?
Does each client connect and stay connected over the whole time of the game or do they connect / exchange some data and disconnect.
How many connections must be handled in parallel?

Do you want to handle each connection in a separate thread or in a separate process or do you want to write a single threaded/single process asynchronous server with an asynchronous framework like asyncio or twisted.

Asynchronous server are quite  interesting, but they require, that the entire server code is written in an asynchronous way and it is more complicated if you're not used to it. However in many cases they handle the highest load.

I guess, that a threaded socket server would be easiest to code (but you have to use mutexes (locks) to handle shared data). However if you have a multi core server and you want to use multiple CPUs, then a forking server might be better, but it will raise the question how to exchange data between the multiple processes.
One of Python's biggest weaknesses is, that Python threads do not really use multiple CPUs, as only one python byte code can be executed at the same time due to the Global interpreter lock (GIL). I personally had never issues with that in my private and professional python applications, but it's good to know, that multi threading is mostly NOT the solution to load share an application amongst multiple CPUs.

An asynchronous frameworks is probably a very interesting programming exercise, but wrapping your head around it and being sure that all your code is asynchronous (you can still use thread pools for some stuff) might be a challenge.

I never wrote socket servers that have to keep many connections alive, So if this is the case for your I don't have experience in knowing how many threads can easily be handled and I also don't know how much CPU your server uses.

Depending on your choice you might think of not using the low level socket module
https://docs.python.org/2/library/socket.html
but instead use the higher level abstraction
https://docs.python.org/3/library/socketserver.html with either the ForkingMixIn or the ThreadingMixIn.

Or if you go for asynchronous server, then you could look at asyncio and https://docs.python.org/3/library/asyncio-stream.html

If you really think your server would be CPU limited, then you could look at multhttps://docs.python.org/3/library/multiprocessing.html which has easy mechanisms to exchange data amongst multiple python processes

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
beavoidAuthor Commented:
This is what I have working now, at least for initial connections, that was a gotcha until I figured out the IPAddress system.
I am not concerned about data transmission now, it should be okay now that I can connect with IPAddress . . .
Look okay?

Here is the solution to the client:
 (Server below)
import socket,ipaddress,socket,struct 


PORT = 5007              # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serverAddress='123.456.789.666'
print('\n\n\nHost-->', serverAddress)
s.connect((serverAddress, PORT))


s.close()
print('Closed')

Open in new window


And for the server...
import socket, ipaddress

HOST = ''                 
PORT = 5007             
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
print('accepting')
conn, addr = s.accept()
print('Connected by', addr)
while 1:
    data = conn.recv(1024)
    if not data: break
    conn.sendall(data)
conn.close()

Open in new window

gelonidaCommented:
Hi,

Your code looks OK.

Following comments:

1.) socket option SO_REUSEADDR
As mentioned in my previous post. I strongly suggest to set the socket option SO_REUSEADDR in order to have less surprises. when stopping / starting the server often. You will have occasional erros of the kind "OSError: [Errno 98] Address already in use" If you don't do this.

2.) let the client send / receive something In order to better see that everything really works as expected.

3.) I guess this is intentional, right? Your server just handles one connection and quits afterwards.

4.) Please read my previous comments and questions about architectural choices and potential modules to use depending on your answers choices.

Very probably your server could use one of the above mentioned modules instead of using the low level socket module and implementing everything yourself. (Except of course you really want to do this for educational purposes)


Here the slightly modified server code (setting the .SO_REUSEADDR socket option)
import socket

HOST = ''                 
PORT = 5007             
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# I strongly recommend to add next line in order to 
# have no surprises when restarting the server quickly 
# during development
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((HOST, PORT))
s.listen(1)
print('accepting')
conn, addr = s.accept()
print('Connected by', addr)
while 1:
    data = conn.recv(1024)
    if not data: break
    conn.sendall(data)
conn.close()

Open in new window


and here the slightly adapted client code:
I changed the IP address to localhost (for testing) and I send three messages and receive the related answers
import socket

PORT = 5007              # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serverAddress='localhost'
print('\n\n\nHost-->', serverAddress)
s.connect((serverAddress, PORT))

for cnt in range(3):
    data = b"Hello%d\n" % cnt
    s.sendall(data)
    received = s.recv(1024)
    print("received", repr(received))

s.close()
print('Closed')

Open in new window

Become a Microsoft Certified Solutions Expert

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

beavoidAuthor Commented:
Thanks

I'm still not 100% on the OOP,
what would the server look like as a class?
Do I just put half the constructor code there into an instance method called
def server_Thread_Run():
and call it from the constructor as a Thread?

That makes sense to me, and have the PORT be this.PORT = 5007, or is up at the top okay? (the same)
beavoidAuthor Commented:
How would I best send a RDY message back to the server from the client? - Once game window is up, etc,
I'm getting errors, there is send, and sendall
Does the server need to tell the client which client number it is? or is that not necessary? Maybe for team color?

Thanks
gelonidaCommented:
Apologies @beavoid. Rather busy the last two days. (and still quite busy)
Here just a short answer.

I really don't know enough about the structure of youg game.

but yes in general each client should receive a unique id like a  uuid or an integer so that you can identify it again.  when looses the connection and reconnects.
If each client has a unique player name, then you could use the player name, but what happens if two players choose the same name ??

Concerning the OO aspects.
It's very difficult to say without knowing exactly the interactions happening in yout game.

but I could imagine a class that handles the connections and has a member object named for example state, that stores the entire state of the game (all positions of all objects, all players etc) and delegate the game play to this object.

It allows you to separate the socket communication from the rest of the game.

An inverse logic is also possible, where you have a game object containing a server isntance that takes care of the socket communication.

you could also create a class for parsing the messages and a class for converting actions to messages, that are sent to the clients.
beavoidAuthor Commented:
Thanks
In my first, Java RTS, the server had a socket dedicated Thread for each client and the server loop waited for each client delegate to receive a message per cycle
This seemed to be effective, and best, because does the server loop ever get overwhelmed?
My new Python server, right now, I'm toying with one loop to receive all client messages. Wont some client messages arrive while the server is occupied with other client messages?
Which is established as the optimal solution?

Thanks
beavoidAuthor Commented:
Hi

This is my working server code. It accepts HELO's from the client, but then it collapses and I don't know enough why!?

attached server and client, in the client, edit the server address to be the server's IP address inside ' '

Thanks
PRTSserver.py
PRTSclient.py
gelonidaCommented:
Dojn't have time to try now, but try to use:

        while 1:
            conn, addr = s.accept()
            print('Connected by', addr)
            print('receive')
            data = conn.recv(3)
            print('data[1]',data[1])
            print('data[2]',data[2])
            if not data: break
            # conn.sendall(data)
            conn.close()

Open in new window


instead of
        conn, addr = s.accept()
        print('Connected by', addr)
        while 1:
            print('receive')
            data = conn.recv(3)
            print('data[1]',data[1])
            print('data[2]',data[2])
            if not data: break
            # conn.sendall(data)
            conn.close()

Open in new window

gelonidaCommented:
By the way you use a thread, but the way you use threads you will only handle one connection at a time.
Is this intentional?
beavoidAuthor Commented:
Thanks, that code is good

The threadRun method needs to be called as / made into a proper thread, and each client joining will be handled from within. I might launch a Thread just to handle in-game UDP duplicates for each client, like I did in my last RTS
Could I expect any unanticipated networking problems within this client Thread?
I don't think so.
So, there will be a start-off Thread for joining that launches separate Threads for client handling
The start-off server Thread would be launched as such, correct?
        runThread = Thread(self.threadRun())
and many connections can be handled simultaneously.  

Thanks
gelonidaCommented:
Well I think that should be ok. Your thread will be able to handle many clients. as each client connects, sends only a short message and closes its connection again.

There would only be an issue if you changed the code to something where each client keeps it's connection open during the entire game's session. but I think your approach is more robust and less resource intensive.
beavoidAuthor Commented:
Thanks,

Is league of legends game-networking like an RTS, or an FPS? TCP streams w/ UDP alongside as a failsafe/performance raiser?
Thanks
gelonidaCommented:
I think it's better you ask another question.

I can help with general python client server issues and some advice common to non gaming applications.

If it's too game server related I might not be the right person to answer.

Chances that anybody else will reply in such an old thread might be quite low.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Game Programming

From novice to tech pro — start learning today.