Solved

Windows Sleep Microseconds for Low Latency Thread Communication

Posted on 2012-04-05
7
1,634 Views
Last Modified: 2013-08-07
Is there any way I could suspend the thread for 1-2 microseconds? (Note 1 microsecond = 1/1000 miliseond). I know this question has been asked but so far there is no good answer for this. Here is my situation why I need this:

I have a server with several threads running to serve client's requests. Requests are sent by other thread through shared memory which occurs occasionally. However, once request is sent it needs to be processed ASAP. So, I need a fast way to notify one of the serving threads when request is queued in shared memory.

Windows event object was used but it takes 6-7us from SetEvent() to WaitForSingleObject() return on my machine (tried to set process/thread priority but still not much improvement). I tried to use a busy loop to let the serving threads keep pooling the memory which lower the latency to 1-2us, which is good enough, but it burns the CPU while the requests are only sent like once per minutes. If I could insert a micro/nano second sleep into the loop I could at least get my CPU free while keep the latency low.

I would be glad if anyone could suggest me another way to do the thread communication with latency lower than 2us. Thanks
0
Comment
Question by:codeblue229
  • 2
  • 2
7 Comments
 
LVL 16

Expert Comment

by:HooKooDooKu
ID: 37815902
Have you tried Sleep(0) ?

It doesn't sleep for any given set of time.  But what it is supposed to accomplish is to give up the remainder of its time slice to allow other processes access to the CPU.
0
 

Author Comment

by:codeblue229
ID: 37816351
Similiar to windows API "SwitchToThread()", it yields execution to other thread but the CPU would still at peak, because the thread is not actually in sleep mode the CPU will resume execution of the thread and never go idle.
0
 
LVL 22

Accepted Solution

by:
ambience earned 500 total points
ID: 37817506
High resolution has always been a problem under Windows and in general there is no guaranteed way to achieve microsecond level precise sleep for durations < 1ms. Windows 7 has User Mode Scheduling as described here http://msdn.microsoft.com/en-us/library/windows/desktop/dd627187%28v=vs.85%29.aspx. I'm absolutely unsure whether thats relevant or whether it could achieve higher performance compared to the system scheduler but apparently its designed to serve that purpose.

But even that would not guarantee anything because
By default the thread quantum on Windows NT based systems is about 100 milliseconds (I believe for servers). This means that a thread can “hog” the CPU for up to 100 milliseconds before another thread has a chance to be scheduled and actually execute.
http://blogs.msdn.com/b/mediasdkstuff/archive/2009/07/02/why-are-the-multimedia-timer-apis-timesetevent-not-as-accurate-as-i-would-expect.aspx

BTW, why is it absolutely necessary to achieve switching time of <2us? How much time does the worker take to process and send back a reponse (if at all)? Do you think 6us of swtiching delay can be included in the time it takes to process requests?

If you are going to say no the response must be sent within 2us (for example) then its perhaps better not to have worker threads?

---

On a different note, if you are running on multiprocessor machine then perhaps you can assign different processor affinities to your IO and worker threads such that none would compete for the same processor. It would work because even if the threads quantum expires it would get assigned again to the same processor and wont have to wait. That would only peak one of the CPUs and the other one would be IO bound. I haven't tried such an arrangement but it might work.
0
 

Author Comment

by:codeblue229
ID: 37818226
I didn't aware when they introduced the User Mode Scheduling but this surely worth trying, although it is only supported in 64bit applications. Actually I am not quite sure about why it takes 6-7us to do the switching on my fairly idle machine. Whether it is normal context switching overhead or it's the implementation of windows event-wait mechanism? Having a chance to manage the scheduling might help finding it out. Thanks!

The worker threads simply call a blocking API, which immediately send request to another server and then block until server responds. It is the time when the request reach the server matters. There are definite number of API instance (each with different user logon) making up the throuttle rate serving a burst of several requests. That's why it has to be done in worker threads.
0
 
LVL 22

Expert Comment

by:ambience
ID: 37818831
But still trying to optimize for 3-4us - is this effort worth it? Does the rest of the code provide guaranteed bounded response times? I mean given delays can occur anywhere and a thread can hog all other threads, is this delay the only major obstacle?
0

Featured Post

Top 6 Sources for Identifying Threat Actor TTPs

Understanding your enemy is essential. These six sources will help you identify the most popular threat actor tactics, techniques, and procedures (TTPs).

Join & Write a Comment

Does your iMac really need a hardware upgrade? Will upgrading RAM speed-up your computer? If yes, then how can you proceed? Upgrading RAM in your iMac is not as simple as it may seem. This article will help you in getting and installing right RA…
Great sound, comfort and fit, excellent build quality, versatility, compatibility. These are just some of the many reasons for choosing a headset from Sennheiser.
The goal of the video will be to teach the user the concept of local variables and scope. An example of a locally defined variable will be given as well as an explanation of what scope is in C++. The local variable and concept of scope will be relat…
With the advent of Windows 10, Microsoft is pushing a Get Windows 10 icon into the notification area (system tray) of qualifying computers. There are many reasons for wanting to remove this icon. This two-part Experts Exchange video Micro Tutorial s…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now