Solved

Thread Question

Posted on 2002-04-08
21
248 Views
Last Modified: 2010-04-04
Our current application has a reporting tool in it.  It can generate upto 30 different reports.  The problem is, the reports sometime take FOREVER (I'm talking hours here) to generate.  What we want to do is move all the generation code into a thread, pass it the queries and instructions and let it go.  This will enable us to queue jobs and still allow people to continue using our app - in theory :)

I've already done the ground work.  I've written the functions to do the reports, and I tried putting in a thread, and that's were the problem starts.

When the code is executed, my app stops responding until the report has finished.  Why is this?  Is there anyway I can make the app work while the report is generating?

I'm still a thread newby, so any help would be appreciated.

Thanks,

Stu
0
Comment
Question by:Stuart_Johnson
  • 8
  • 4
  • 4
  • +4
21 Comments
 
LVL 9

Expert Comment

by:ginsonic
Comment Utility
listening
0
 
LVL 27

Expert Comment

by:kretzschmar
Comment Utility
maybe you could show a bit code, how you do it now
0
 

Expert Comment

by:classics
Comment Utility
Try setting the thread to a priority lower than the main program thread.  Before calling Resume just set Priority := tpLower;
0
 
LVL 6

Author Comment

by:Stuart_Johnson
Comment Utility
meikl,

The code I use is around 3,000 lines+ long.  In a nut shell, all it does is query an SQL server (repeatedly) and creates Paradox files which are then used back in the main application to show the reports.  The slow process is the actual copying of the data - especially on slower databases (like our old AS/400 for instance).

classics,

I've tried that as well.  I've also set it to idle and that didn't make any difference.  It's as though the thread gets highest priority no matter what.

My understanding of a thread (and this maybe completely misguided) is that if an application executes a thread, the main application can continue doing it's thing without being ground to a halt (if the priority isn't set too high).  Is this the case?

Most of our reports create the temporary tables using just a simple line by append, but when a batch move is called, that's when the system stops responding.  We need to be able to use the main application regardless of how we copy the data - even if the main app is a bit slow (but not frozen).

Thanks so far guys,

Stu.

0
 

Expert Comment

by:classics
Comment Utility
Are you calling a lot of code in Synchronize()?

That will block your main thread and any other thread waiting to use VCL functions.
0
 
LVL 27

Expert Comment

by:kretzschmar
Comment Utility
well stuart,
as delphi didn't provide a real multithreading-model
(all threads do affect the mainthread),
especially if you use the bde (do you?),
therefore
another suggestion could be to start a separat app from your app, which does this work in the background

may this possible for you?

meikl ;-)
0
 
LVL 3

Accepted Solution

by:
SteveWaite earned 150 total points
Comment Utility
Another thread is great until you have to get some value from it within the main gui thread.
if your thread calls Synchronize because it accesses the vcl or data stored in the gui thread then one thread waits for the other. This is why you get a pause.
A separate thread should use notify events to inform the main thread of progress.
So, really you need to avoid Synchronize() and instead wait until your thread has finished work, then use notify event to tell main thread.
Also can you can put it in a dll instead.
Let me know what help you need with those ideas.
0
 
LVL 6

Author Comment

by:Stuart_Johnson
Comment Utility
Hi Classics,

Yes I am calling it from within a Synchronize.  I'm just working off examples I've downloaded, and in each example they wrap the functions in Synchronize.  I'll explain a bit more later on what's actually happening with the thread.

Hi Meikl,

This was considered as well, but it means that we have to rely on this working 100% of the time.  Some of our users are pretty bloody stupid, and going down this path maybe more of a nightmare than anything else.  But it hasn't been ruled out  yet :)  And yes, we do use the BDE.  Is there another way I can avoid the BDE and create Paradox files, but also be able to read from either an AS/400, MS-SQL, Oracle or Interbase (these are our supported databases)?

Hi Steve,

I think you've hit the closest to what I'm actually refering too.  I would like a little more info on this if possible, but I'll write a few more comments below to give a bit more info to everyone.



This thread is created when the application first loads and is terminated when the app is closing.  Basically this thread is a spooler.  We can pass the thread a command, and it will create us a single paradox file (with a unique name).

The thread maintains a list of pending, active and completed jobs (until the thread is terminated).  This list is a TObjectList (which contains classes as:

type
  TJobDetails = class
    JobID: Integer; {Unique job ID}
    JobPriority: Integer; {0 - 5, 0=lowest, 5=highest}
    ReportNumber: Integer;
    Paused: Boolean;
    Progress: Integer; {How far through processing we are}
    JobStatus: TJobStatus; {Pending, Active, Completed, Error}
  end;

I can query the thread, asking for details at anytime by calling:

function QueryJobDetails: Variant; overload;
function QueryJobDetails(JobID: Integer): Variant; overload;

The returned variant is a variant array which I pull to pieces in my app and then display that info in a TListView.  I can then right-click on any of the items and change it's job priority, cancel the job or pause it (you can not pause a job already processing, only jobs with a status of Pending).

And finally, I have an event that is triggered at the commencement and completion of each job.

So, I don't directly update any VCL from within the thread.

Is what I'm doing sounding OK?  I really need to be able to get this working so I can use the app at the same time as generating these reports.  I'm not overly fussed with how it's done, although I really would like to steer clear of the external application until all other avenues are exhausted.

Thanks again,

Stu
0
 
LVL 3

Expert Comment

by:SteveWaite
Comment Utility
Sounds OK. Obviously restrict what is done in Synchronize.
Maybe you could use another thread to do communications between gui and worker threads. So having one controlling thread that spawns the worker thread(s).
0
 
LVL 20

Expert Comment

by:Madshi
Comment Utility
Yeah, the key point is using Synchronize. Synchronize is a hack, which moves the execution back into the main thread. So if you do all the work inside of the Synchronize, you're doing it in the main thread again...

Regards, Madshi.
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 6

Author Comment

by:Stuart_Johnson
Comment Utility
Guys,

I have this working perfectly!  What I would like to do is spit the points up between each of you.  Now, how do we do this.  classics, Steve and Meikl - 50 points each, or do I award the points to one expert?  I'm not fussed, you each helped me out differently although it was Steve who enlightened me on my slow redraws.  It's a shame there was no follow-up on the DLL.

Anyway, let me know which way you want it to go.

Cheers,

Stu
0
 
LVL 3

Expert Comment

by:SteveWaite
Comment Utility
Do you want a dll example, or are you ok now?
0
 
LVL 6

Author Comment

by:Stuart_Johnson
Comment Utility
Hi Steve,

I have my application functioning beautifully at present, so the DLL in this instance is probably not required.  However, it would be interesting for future references, no doubt.  If you have time to do one, I'd appreciate it, but please, don't go to an overly silly amount of effort to do so.  You're help was great within this thread, so you've definately earnt your points.

Stu
0
 
LVL 27

Expert Comment

by:kretzschmar
Comment Utility
yep, grade SteveWaite,
there is no need to grade me

meikl ;-)
0
 
LVL 6

Author Comment

by:Stuart_Johnson
Comment Utility
Are you sure, Meikl?  I'm quite happy to split it.
0
 
LVL 27

Expert Comment

by:kretzschmar
Comment Utility
its your decission, stuart
0
 
LVL 3

Expert Comment

by:SteveWaite
Comment Utility
For later on then..
Do a search for Madshi's dll keyboard hook example (can be found all over ex-ex).
It has a good example of putting work in a dll and sharing the memory correctly. You would use that to transfer your results. The hook stuff is a bonus.
Regards,
Steve
0
 
LVL 8

Expert Comment

by:TOndrej
Comment Utility
When working with BDE, each thread must use its own TSession, TDatabase and TTable/TQuery components.
More details in help:
DevGuide: Developing database applications
"Managing multiple sessions"
0
 
LVL 6

Author Comment

by:Stuart_Johnson
Comment Utility
Steve,

Thanks for you help with this.  I'll test out your suggestions as well.

Madshi, there is a question for you with some points.  Thank you too mate!

Stu
0
 
LVL 6

Author Comment

by:Stuart_Johnson
Comment Utility
TOndrej,

Thank-you for the extra tips.  Much appreciated.

Stu
0
 
LVL 6

Author Comment

by:Stuart_Johnson
Comment Utility
TOndrej,

I was wondering if you could give me a bit of info.  When I create my thread, I'm passing it the alias I need to connect to, plus the username and password.  I create a new TSession and a new TDatabase.  The session works fine, however, when I try to do:

ADatabase.DatabaseName := AliasName

I get External Exception C0000008.  I've looked on the net, but the only resolution I can find is I need to go back a version of the SQL drivers I'm using _IF_ I was using Oracle or Sybase (I'm using SQL Server 2000).

How can I get around this error?  What am I doing wrong?

Thanks,

Stu
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
Introduction I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need. This article will address some of the concepts when dealing with a multithreaded delphi database…
This video discusses moving either the default database or any database to a new volume.
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

13 Experts available now in Live!

Get 1:1 Help Now