Solved

Speeding up a multi-threaded program

Posted on 2000-03-21
14
206 Views
Last Modified: 2010-04-10
Greetings,

I have a question about trying to speed up a program that is already multi-threaded. I have a program that may have up to 120 threads continuosly waiting on separate sockets for data from another process. When data is received some work is done and a reply message is sent back through the socket. Then the thread goes back to waiting on the socket. When examing the trace file for this operation I noticed that when a particular thread has completed its tasks and goes back into waiting on the socket, there is about 100 milliseconds before the context is switched to another thread.

I know that this amount of time doesn't seem much but I was wondering anything could be done to speed up the context switching between threads. Is there a way of telling Win NT that the thread is about to wait become idle and that it should now switch to another thread?
By the way, I remember a function called Yield that kind of did this?

Thanks
0
Comment
Question by:sinister
  • 4
  • 4
  • 2
  • +4
14 Comments
 
LVL 1

Expert Comment

by:snifong
Comment Utility
Do a Sleep(0).  It will "force" a context switch.
0
 
LVL 22

Expert Comment

by:nietod
Comment Utility
>> Do a Sleep(0).
Then the thread can't wait for the socket.

>>  have a program that may have up to
>> 120 threads continuosly
You may wish to reconsider that design.   Threads are very costly.  Especially on non-NT windows.  Even a suspended thread consumes resources.  I've seen rules that recomend that you have no more than 10 at a single time threads in a single process.

>> Is there a way of telling Win NT that the thread is
>> about to wait become idle
Well how are you waiting at the moment?   Are you doing overlapeedd I/O and  oing a WaitForSingleObject() on an event handle from an OVERLAPPED structure?
0
 
LVL 1

Expert Comment

by:snifong
Comment Utility
Thanks, nietod.
0
 

Author Comment

by:sinister
Comment Utility
Nietod:

There is no overlapped io. The reason that we have so many threads is that this program is a service for a telephony system, so we want to have a thread dedicated to each telephone line. I have a feeling that a thread pool would be even slower. If I did a sleep(0) then my TCP Recv socket function would not get executed until the thread scheduler back to that thread. Is this correct? To be just off topic a little - What are fibers? I have never of them before but I know they are like threads? By the way, whoever wants the points just post another answer and you can have them. Thanks.
0
 
LVL 22

Expert Comment

by:nietod
Comment Utility
I sever line in use at a time?  If not just create threads for the lines in use.  

>> sleep(0) then my TCP Recv socket function
>> would not get executed until the thread
>> scheduler back to that thread. Is this correct?
correct.

>> What are fibers?
Sort of like threads within a thread.  A thread can have multiple fibers running within it.  When the thread is running, one of its fibers is actually running.  This current fiber can be switched to another fiber.  However unlike thread scheduling, which is preemptive, the thread and fibers themselves schedule the fibers cooperatively.

The real question is what is the thread doing?  is it waiting for I/O?  If so it should be using overlapped I/O and waiting efficiently.   If not, what is it doing?
0
 
LVL 32

Expert Comment

by:jhance
Comment Utility
I'll just agree with the comments posted earlier, threads are not a good match to this problem.  There is just too much overhead switching between all these threads that are doing little tiny bits of work.  So you end up spending more time switching thread contexts than you do working.

The asynchronous WINSOCK calls are a much better fit for this and you can easily run 100s or even 1000s of socket connections using async sockets.  Windows handles the messages generated by the sockets very well (Windows was designed to process messages not threads).
0
 
LVL 3

Expert Comment

by:ufolk123
Comment Utility
Scheduling between lot of thread can be a major overhead as in windows NT schduling policy is system wide.
Have you tried using select().
In one thread you can wait of all open socket connections and whenever any new socket is ready for connection ,add it to list after doing accept().This way all active sockets can be serviced by a single thread.Whenever you need to do some long operation , process it in a child thread passing it the socket handle .
Using Windows messaging is also a good idea as Windows OS seems to be more optimised for messaging based architectures.

0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 
LVL 11

Expert Comment

by:alexo
Comment Utility
>> so we want to have a thread dedicated to each telephone line.
>> I have a feeling that a thread pool would be even slower

This is wrong.  Managing many threads is expensive, a thread pool is better.

Another option (mentioned previously by nietod) is overlapped IO.  Very efficient on an NT platform.

I tend to disagree with the suggestion to use Async IO.  IMHO, message dispatching will increase the overhead.

One more issue - you can change the length of the time quantum on NT.  This however can have an undesirable effect on the whole system.

Lastly, beware of "premature optimization".  If your program is IO bound (network traffic), optimizing the CPU overhead will not help.  A profiler is your friend (although, in nietod's case, it can be a very unreliable friend...)
0
 

Author Comment

by:sinister
Comment Utility
Well, thanks everyone for your comments, I think that I will consider these options if our system ever gets redesigned. Who wants the points?
0
 

Author Comment

by:sinister
Comment Utility
Just too clarify things, I was looking for a quick solution but unfortunately all the suggestions would require a complete redesign. However, these are some good optimization subjects, can anybody refer a book or something about these topics so that can learn more. I find that most books never get into deep detail about multi-threaded programming etc.
0
 
LVL 22

Accepted Solution

by:
nietod earned 50 total points
Comment Utility
The "Zen of Code Optimization" by Michael Abrash is about the best (maybe only) book on optimization.  I don't think it covers multi-threading in particular but still will be vauable.  

The main rule with multi-threading is to avoid bussy waiting.  Don't have a loop that just waits for some external condition to be met.  In windows use  synchronization objects (events, mutexes etc) to avoid bussy waits.

Its best if you figure out who (if anyone) you think deserves the points.  It avoids disputes and supports the philosophy behind EE--namely that the points shoud serve as an insentive.
0
 
LVL 5

Expert Comment

by:pitonyak
Comment Utility
Notice that you should not do a sleep(0) before asking for input on the socket, but why don't you check to see if there is data on the socket and if so, then get it, otherwise do a sleep(0)


In perl I use the vec and select statements to see if there is data available. If not then do not bother..

andy
0
 
LVL 22

Expert Comment

by:nietod
Comment Utility
>> but why don't you check
>> to see if there is data on the socket
>> and if so, then get it, otherwise do a sleep(0)
No, that is a bussy wait.  You will coninually waste CPU time just looking to see if there is data and then going back to sleep.  The code that that tests to see if there is data to read probably doesn't consume a lot of time (althought it is wasted time) but the code that switches to your thread and then almost immediately away from your thread consumes a lot of wasted time.   Beter to use a technique that puts your thread to sleep so that it uses absolutely no CPU time until there is something for it do do.
0
 

Author Comment

by:sinister
Comment Utility
Thanks
0

Featured Post

Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

Join & Write a Comment

Article by: SunnyDark
This article's goal is to present you with an easy to use XML wrapper for C++ and also present some interesting techniques that you might use with MS C++. The reason I built this class is to ease the pain of using XML files with C++, since there is…
Often, when implementing a feature, you won't know how certain events should be handled at the point where they occur and you'd rather defer to the user of your function or class. For example, a XML parser will extract a tag from the source code, wh…
The goal of the video will be to teach the user the difference and consequence of passing data by value vs passing data by reference in C++. An example of passing data by value as well as an example of passing data by reference will be be given. Bot…
The viewer will learn how to clear a vector as well as how to detect empty vectors in C++.

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now