Solved

Speeding up a multi-threaded program

Posted on 2000-03-21
14
209 Views
Last Modified: 2010-04-10
Greetings,

I have a question about trying to speed up a program that is already multi-threaded. I have a program that may have up to 120 threads continuosly waiting on separate sockets for data from another process. When data is received some work is done and a reply message is sent back through the socket. Then the thread goes back to waiting on the socket. When examing the trace file for this operation I noticed that when a particular thread has completed its tasks and goes back into waiting on the socket, there is about 100 milliseconds before the context is switched to another thread.

I know that this amount of time doesn't seem much but I was wondering anything could be done to speed up the context switching between threads. Is there a way of telling Win NT that the thread is about to wait become idle and that it should now switch to another thread?
By the way, I remember a function called Yield that kind of did this?

Thanks
0
Comment
Question by:sinister
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 4
  • 2
  • +4
14 Comments
 
LVL 1

Expert Comment

by:snifong
ID: 2641777
Do a Sleep(0).  It will "force" a context switch.
0
 
LVL 22

Expert Comment

by:nietod
ID: 2641822
>> Do a Sleep(0).
Then the thread can't wait for the socket.

>>  have a program that may have up to
>> 120 threads continuosly
You may wish to reconsider that design.   Threads are very costly.  Especially on non-NT windows.  Even a suspended thread consumes resources.  I've seen rules that recomend that you have no more than 10 at a single time threads in a single process.

>> Is there a way of telling Win NT that the thread is
>> about to wait become idle
Well how are you waiting at the moment?   Are you doing overlapeedd I/O and  oing a WaitForSingleObject() on an event handle from an OVERLAPPED structure?
0
 
LVL 1

Expert Comment

by:snifong
ID: 2641871
Thanks, nietod.
0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 

Author Comment

by:sinister
ID: 2641905
Nietod:

There is no overlapped io. The reason that we have so many threads is that this program is a service for a telephony system, so we want to have a thread dedicated to each telephone line. I have a feeling that a thread pool would be even slower. If I did a sleep(0) then my TCP Recv socket function would not get executed until the thread scheduler back to that thread. Is this correct? To be just off topic a little - What are fibers? I have never of them before but I know they are like threads? By the way, whoever wants the points just post another answer and you can have them. Thanks.
0
 
LVL 22

Expert Comment

by:nietod
ID: 2642000
I sever line in use at a time?  If not just create threads for the lines in use.  

>> sleep(0) then my TCP Recv socket function
>> would not get executed until the thread
>> scheduler back to that thread. Is this correct?
correct.

>> What are fibers?
Sort of like threads within a thread.  A thread can have multiple fibers running within it.  When the thread is running, one of its fibers is actually running.  This current fiber can be switched to another fiber.  However unlike thread scheduling, which is preemptive, the thread and fibers themselves schedule the fibers cooperatively.

The real question is what is the thread doing?  is it waiting for I/O?  If so it should be using overlapped I/O and waiting efficiently.   If not, what is it doing?
0
 
LVL 32

Expert Comment

by:jhance
ID: 2642821
I'll just agree with the comments posted earlier, threads are not a good match to this problem.  There is just too much overhead switching between all these threads that are doing little tiny bits of work.  So you end up spending more time switching thread contexts than you do working.

The asynchronous WINSOCK calls are a much better fit for this and you can easily run 100s or even 1000s of socket connections using async sockets.  Windows handles the messages generated by the sockets very well (Windows was designed to process messages not threads).
0
 
LVL 3

Expert Comment

by:ufolk123
ID: 2643253
Scheduling between lot of thread can be a major overhead as in windows NT schduling policy is system wide.
Have you tried using select().
In one thread you can wait of all open socket connections and whenever any new socket is ready for connection ,add it to list after doing accept().This way all active sockets can be serviced by a single thread.Whenever you need to do some long operation , process it in a child thread passing it the socket handle .
Using Windows messaging is also a good idea as Windows OS seems to be more optimised for messaging based architectures.

0
 
LVL 11

Expert Comment

by:alexo
ID: 2643390
>> so we want to have a thread dedicated to each telephone line.
>> I have a feeling that a thread pool would be even slower

This is wrong.  Managing many threads is expensive, a thread pool is better.

Another option (mentioned previously by nietod) is overlapped IO.  Very efficient on an NT platform.

I tend to disagree with the suggestion to use Async IO.  IMHO, message dispatching will increase the overhead.

One more issue - you can change the length of the time quantum on NT.  This however can have an undesirable effect on the whole system.

Lastly, beware of "premature optimization".  If your program is IO bound (network traffic), optimizing the CPU overhead will not help.  A profiler is your friend (although, in nietod's case, it can be a very unreliable friend...)
0
 

Author Comment

by:sinister
ID: 2645319
Well, thanks everyone for your comments, I think that I will consider these options if our system ever gets redesigned. Who wants the points?
0
 

Author Comment

by:sinister
ID: 2645377
Just too clarify things, I was looking for a quick solution but unfortunately all the suggestions would require a complete redesign. However, these are some good optimization subjects, can anybody refer a book or something about these topics so that can learn more. I find that most books never get into deep detail about multi-threaded programming etc.
0
 
LVL 22

Accepted Solution

by:
nietod earned 50 total points
ID: 2645491
The "Zen of Code Optimization" by Michael Abrash is about the best (maybe only) book on optimization.  I don't think it covers multi-threading in particular but still will be vauable.  

The main rule with multi-threading is to avoid bussy waiting.  Don't have a loop that just waits for some external condition to be met.  In windows use  synchronization objects (events, mutexes etc) to avoid bussy waits.

Its best if you figure out who (if anyone) you think deserves the points.  It avoids disputes and supports the philosophy behind EE--namely that the points shoud serve as an insentive.
0
 
LVL 5

Expert Comment

by:pitonyak
ID: 2646921
Notice that you should not do a sleep(0) before asking for input on the socket, but why don't you check to see if there is data on the socket and if so, then get it, otherwise do a sleep(0)


In perl I use the vec and select statements to see if there is data available. If not then do not bother..

andy
0
 
LVL 22

Expert Comment

by:nietod
ID: 2647172
>> but why don't you check
>> to see if there is data on the socket
>> and if so, then get it, otherwise do a sleep(0)
No, that is a bussy wait.  You will coninually waste CPU time just looking to see if there is data and then going back to sleep.  The code that that tests to see if there is data to read probably doesn't consume a lot of time (althought it is wasted time) but the code that switches to your thread and then almost immediately away from your thread consumes a lot of wasted time.   Beter to use a technique that puts your thread to sleep so that it uses absolutely no CPU time until there is something for it do do.
0
 

Author Comment

by:sinister
ID: 2649774
Thanks
0

Featured Post

Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Errors will happen. It is a fact of life for the programmer. How and when errors are detected have a great impact on quality and cost of a product. It is better to detect errors at compile time, when possible and practical. Errors that make their wa…
C++ Properties One feature missing from standard C++ that you will find in many other Object Oriented Programming languages is something called a Property (http://www.experts-exchange.com/Programming/Languages/CPP/A_3912-Object-Properties-in-C.ht…
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…
The viewer will learn how to use the return statement in functions in C++. The video will also teach the user how to pass data to a function and have the function return data back for further processing.

733 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question