Solved

COM object methods causing threaded message pump freezing

Posted on 2004-04-18
13
857 Views
Last Modified: 2007-12-19
Hi,

I have a query from one of my team members I thought I'd post as a question here. It's a gnarly one and a bounty of 1000 points is posted for it:

Cheers,

Raymond.


Here it is:


I'm currently performing maintainence on a single threaded application
that makes extensive use of legacy libraries and 3rd party DLLs for file
import/export/conversion, operations that can sometimes take rather a
long time.

We currently have a 'busy' dialog withn a cute animation, progress
bar, status message and cancel button that we show when running a
lengthy conversion, however it's user responsiveness is rather poor as
it relies on application.ProcessMessage calls from callbacks in the
conversion libraries, which are not provided in all libraries and even
when available are not always called as often as one would like.

In order to improve the perceived responsiveness of the Busy dialog I have taken
the rather hazardous approch of developing a multi-threaded implementation in
which the dialog is implemented within a DLL (work-around for thread-unsafeness
of the vCL) which creates a seperate thread to run a message pump for the dialog.

NOTE: due to the huge existing codebase the ideal approch of writing the import
routines to run in worker threads is undesirable, it's much more managable to
improve the user experience by making some localised changes to the "Busy" dialog
than head off into a huge refactor of all the import routines.
 

The core of the dll is as follows...



...


procedure TBusyFormContextThread.Execute;
var
  msg: TMsg;
begin
  // Up thread's priority a little so that BusyForm is responsive even when
  // when application main thread is bogged down
  Priority := tpHigher;

  // Critical that <fBusyForm> created in context of <fBusyFormContextThread>
  // thread
  fBusyForm       := TBusyForm.Create(nil);

  // Run message pump loop for <fBusyForm>
  while GetMessage( msg, 0, 0, 0 ) do
  begin
    TranslateMessage( msg );
    DispatchMessage( msg );
  end;
end;

...

initialization
  IsMultiThread := True;
  Application.Initialize;

  fBusyFormContextThread := TBusyFormContextThread.Create(False);



... communication between the Application and the Busy form is implemented via. custom
messages posted to the Busy form.

This works just fine normally, the application main thread can head off into a lengthy
computation and the busy form remains responsive.

However things break when the application main thread makes a call to a COM object to
run a lengthy conversion. My busy form thread then block/stalls intil the COM object method call
returns, apparently within a call to Dispach() within TControl.WndProc() - Callstack shown below...


TControl.WndProc((3024, 0, 0, 0, 0, 0, 0, 0, 0, 0))
TWinControl.WndProc((3024, 0, 0, 0, 0, 0, 0, 0, 0, 0))
TCustomForm.WndProc((3024, 0, 0, 0, 0, 0, 0, 0, 0, 0))
TWinControl.MainWndProc((3024, 0, 0, 0, 0, 0, 0, 0, 0, 0))
StdWndProc(9241004,3024,0,0)
ThreadProc($1A41A5C)
ThreadWrapper($1A41AA4)

My busy form doesn't explicitly use any COM objects, the only suspect item might be a TAnimate
control.

I'm not at all famaliar with COM programming and the on-line help files (Delphi and MSDN) reading I've been doing has quickly resulted in information overload.  I've been overwhelmed by info. on how to write in-process server COM DLLs and been unable to filter out the small nugget on what I need to do to my vanilla non-COM DLL to prevent it being messed with.

Anyone out there have any idea what I need to do to make things play nicely together???

thanks,

Richard Lang.


0
Comment
Question by:rwilson032697
  • 5
  • 3
  • 2
13 Comments
 
LVL 7

Accepted Solution

by:
sftweng earned 250 total points
ID: 10855752
Port to Linux?

Message queue processing in Windows is a fundamental operating system flaw. No amount of cooperative or pre-emptive thread scheduling or independent task scheduling will ever overcome the message queue bottleneck. Have you ever had to suffer through a Microsoft Outlook startup where everything else is frozen out until the application makes up its mind that its own private world is in order therefore maybe something else might like to run?

One thing you might consider is throwing away all of the message processing in favour of time and event driven multitasking driven from the bios.

Attempting to solve this problem with variations on multi-threading, synchronization with the VCL and various <process>Message calls may be doomed to failure. Perhaps you need to slice through the Gordian knot and just spawn off another application or install dual- or hyper-threaded processors.

The one micro-level bandaid that I can think of centers on the code:
  // Run message pump loop for <fBusyForm>
  while GetMessage( msg, 0, 0, 0 ) do
  begin
    TranslateMessage( msg );
    DispatchMessage( msg );
  end;
which is in a "spin loop". It might make some sense to throttle the loop back a bit, with time delays, some kind of sleep/awake dispatching.

I understand your frustration with complexity. There are much better ways to do these things (e.g., Hoare's massively scalable Communicating Sequential Processes - CSP) but we don't usually have the luxury of doing them given kegacy systems.
0
 
LVL 7

Expert Comment

by:sftweng
ID: 10855778
Another consideration is that Microsoft themselves have apparently given up on the COM and DCOM model with the migration to .NET.

You may have to consider biting the bullet and launching a massive redesign or or isolating the malicious stuff onto a different processor.
0
 
LVL 7

Expert Comment

by:sftweng
ID: 10856116
I'm wondering whether running your BusyForm at tpHigher might be starving the queue.
0
 
LVL 12

Author Comment

by:rwilson032697
ID: 10856486
sftweng:

We appreciate the shortcomings of windows messaging in Windows, hence this Q :-)

The particular issue here is the way calling a COM method appears to stall the message loop that is running in the separate thread from the VCL thread in the DLL. It is worth noting that the application and the DLL have their own copies of the VCL (ie: we are not using run-time packages.)

Earlier investigation suggested that the COM threading appartment model might be a cause of this issue, so we tried changing the apartment model to free-threading (ie: multi-threaded COM). We have not been able to determine if this is a solution as the COM object in question (which is the DWGDirectX ActiveX COM object from the OpwnDWG Alliance at http://www.opendwg.org) appears to fail to create the requested interface when using this COM threading model.

While the message loop may be a 'spin loop', this does not appear to have negative consequences in all the other situations we have tested it (some import filters are very CPU intensive and they operate as expected).

The issue does not seem to be one of the queue being starved, but of being blocked until the COM method call completes (which does appear to be consistent with the COM single thread-apartment model).

Cheers,

Raymond.
0
 
LVL 26

Assisted Solution

by:Russell Libby
Russell Libby earned 250 total points
ID: 10860302

Raymond,
Just a suggestion, but have you tried replacing the GetMessage call with the window handle of your dialog, vs the 0 you are using now?

eg:

while GetMessage( msg, fBusyForm.Handle, 0, 0 ) do
begin
  TranslateMessage( msg );
  DispatchMessage( msg );
end;

If this still has no effect, can you tell me if this COM process is an automation server (vs an inproc library)?  If so, then you may need to implement an IMessageFilter interface into your dialog process to handle the COM concurrency issues.  If you want to try this, let me know, and I can post the code for IMessageFilter and the required call to CoRegisterMessageFilter.

Regards,
Russell


0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 
LVL 12

Author Comment

by:rwilson032697
ID: 10863984
Hi Russell,

I'll get Richard to try your suggestion.

The COM process is inproc (it's a bunch of DLLs). Is there an equivalent to IMessageFilter for these things?

Cheers,

Raymond.
0
 
LVL 26

Expert Comment

by:Russell Libby
ID: 10864063

Raymond,

No IMessageFilter for the inproc stuff (not required). As long as you are sure there aren't any out-of-proc instances that are tagged along with what you are doing, then this should not be the issue. (For out of proc stuff, COM enters a modal loop, thus the need for the filter callback handler)

But, if you are unsure, and want/willing to try it (it won't hurt anything, it will just never get called if this is not the issue), then I will post the IMessageFilter code in the morning.

Regards,
Russell

0
 
LVL 12

Author Comment

by:rwilson032697
ID: 10864167
Russell,

AFAIK, there are no out-of-process COM goodies involved (but then, I wouldn't say I'm 100% sure - how would I tell?)

We do have out-of-process COM servers as a part of the overall system, but they are not used in this particular instance. Though come to think of it, we do have a separate import filter that would use one of these out-of-process objects. Richard did not mention if he had difficulties with that one (it displays the 'busy' dialog in the same way).

Cheers,

Raymond.
0
 
LVL 12

Author Comment

by:rwilson032697
ID: 13187659
We never did get past this issue with the COM blocking and have unfortunately had to shelve this improvement to the SW.

Cheers,

Raymond.

0
 
LVL 12

Author Comment

by:rwilson032697
ID: 15003182
I appreciate the effort put in by sftweng and rlibby so it seems fair ;-)

Cheers,
Raymond.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Have you ever had your Delphi form/application just hanging while waiting for data to load? This is the article to read if you want to learn some things about adding threads for data loading in the background. First, I'll setup a general applica…
Introduction I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need. This article will address some of the concepts when dealing with a multithreaded delphi database…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.
This video shows how to remove a single email address from the Outlook 2010 Auto Suggestion memory. NOTE: For Outlook 2016 and 2013 perform the exact same steps. Open a new email: Click the New email button in Outlook. Start typing the address: …

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now