Solved

Python Multithreading Details

Posted on 2010-09-23
7
801 Views
Last Modified: 2012-05-10
Hi guys,

I'd like to know more about how Python does multithreading, if anyone knows. Does it rely on its own threading library with its own scheduler? I am running a very simple program that starts two threads that constantly and forever print a message ending in a newline (by default). Sometimes, I get two messages on the same line, but the actual messages themselves are never interleaved (except for that newline). Does anyone know why that would happen?

Thanks in advance!

Teddy
0
Comment
Question by:Zyloch
7 Comments
 
LVL 28

Expert Comment

by:pepr
Comment Utility
If they use the same output stream, try to flush the buffer after the print or use an unbuffered output.  However, if you do not have the exclusive access to the stream... I am not sure how the threads are interupted and whether it cannot happen during the output.
0
 
LVL 28

Expert Comment

by:pepr
Comment Utility
You may find more details in the documentation for the threading library (http://docs.python.org/library/threading.html#module-threading) or of the lower level thread module (http://docs.python.org/library/thread.html#module-thread).  I do not have first-hand experience with that.
0
 
LVL 3

Accepted Solution

by:
ilalopoulos earned 500 total points
Comment Utility
Python's handling mechanism is quite simple: it switches between threads every n (a small number of bytecodes i think the default is 10) or before intensive I/O operations and uses the GIL (GLobal Interpreter Lock) to synchronize access of the threads (which are real system threads) to the python interpreter.

Now as to why you see the breakage in the newlines only:

Python switches threads between bytecode instructions and not inside a bytecode so a bytecode is a guaranteed atomic operation that will be executed at whole in a thread before another get control.

So using the dis module (Disassembler for Python bytecode) we do the following little test (I have used python 2.7, different major versions can have different results):

>>> import dis
>>> def p():
      print "Hello"
      print "World"

      
>>> dis.dis(p)
  2           0 LOAD_CONST               1 ('Hello')
              3 PRINT_ITEM          
              4 PRINT_NEWLINE      

  3           5 LOAD_CONST               2 ('World')
              8 PRINT_ITEM          
              9 PRINT_NEWLINE      
             10 LOAD_CONST               0 (None)
             13 RETURN_VALUE  

As you can see there are two bytecodes in action the PRINT_ITEM and the PRINT_NEWLINE so a switch CAN happen between them and that is the reason for the behavior you have seen. However the PRINT_ITEM is a bytecode in its own so it executes at whole as an atomic operation.

Take in mind that bytecodes and thus atomic operations are not necessarily constant among python releases since it is stated that:

"CPython implementation detail:: Bytecode is an implementation detail of the CPython interpreter! Noguarantees are made that bytecode will not be added, removed, or changedbetween versions of Python. " (http://docs.python.org/library/dis.htm)

And so you cannot rely on the bytecode findings of one version to be the same like another one.

Finally although you do not ask such a question, if you want to synchronize the printing the simplest way is to use a lock. Before the print a thread should acquire the lock which will release it only after it finishes printing, all the other threads block at that point and wait to acquire the lock. A simple example would be:

import threading

lock = threading.Lock()

class MyThread(threading.Thread):
    def __init__(self, p):
        threading.Thread.__init__(self)
        self.p = p
    def run(self):
        while (1):
            lock.acquire()
            print self.p
            lock.release()

for i in range(0,10):
    myThread = MyThread("abc" + str(i))
    myThread.start()



Cheers
import threading

lock = threading.Lock()

class MyThread(threading.Thread):
    def __init__(self, p):
        threading.Thread.__init__(self)
        self.p = p
    def run(self):
        while (1):
            lock.acquire()
            print self.p
            lock.release()

for i in range(0,10):
    myThread = MyThread("abc" + str(i))
    myThread.start()

Open in new window

py-dis-hello-word.png
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 41

Expert Comment

by:HonorGod
Comment Utility
@ilalopoulos - outstanding update / explanation.
0
 
LVL 5

Expert Comment

by:-Richard-
Comment Utility
Yes, ilalopoulos, very authoritative and I learned a few things myself.  The only thing I would add is that you can find the number of bytecodes executed per Python "timeslice" with the sys.getcheckinterval() method, which will tell you (at least for Python 2.6) it is 100.  Additionally, you can even set the check interval yourself with sys.setcheckinterval(int).  Setting it to a larger number is documented as giving you somewhat better performance although it would reduce the rate at which your threads interact with each other.  
0
 
LVL 31

Expert Comment

by:James Murrell
Comment Utility
This question has been classified as abandoned and is being closed as part of the Cleanup Program.  See my comment at the end of the question for more details.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Dictionaries contain key:value pairs. Which means a collection of tuples with an attribute name and an assigned value to it. The semicolon present in between each key and values and attribute with values are delimited with a comma.  In python we can…
The purpose of this article is to demonstrate how we can upgrade Python from version 2.7.6 to Python 2.7.10 on the Linux Mint operating system. I am using an Oracle Virtual Box where I have installed Linux Mint operating system version 17.2. Once yo…
Learn the basics of lists in Python. Lists, as their name suggests, are a means for ordering and storing values. : Lists are declared using brackets; for example: t = [1, 2, 3]: Lists may contain a mix of data types; for example: t = ['string', 1, T…
Learn the basics of if, else, and elif statements in Python 2.7. Use "if" statements to test a specified condition.: The structure of an if statement is as follows: (CODE) Use "else" statements to allow the execution of an alternative, if the …

763 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

7 Experts available now in Live!

Get 1:1 Help Now