• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 267
  • Last Modified:

Thread Question

Our current application has a reporting tool in it.  It can generate upto 30 different reports.  The problem is, the reports sometime take FOREVER (I'm talking hours here) to generate.  What we want to do is move all the generation code into a thread, pass it the queries and instructions and let it go.  This will enable us to queue jobs and still allow people to continue using our app - in theory :)

I've already done the ground work.  I've written the functions to do the reports, and I tried putting in a thread, and that's were the problem starts.

When the code is executed, my app stops responding until the report has finished.  Why is this?  Is there anyway I can make the app work while the report is generating?

I'm still a thread newby, so any help would be appreciated.

Thanks,

Stu
0
Stuart_Johnson
Asked:
Stuart_Johnson
  • 8
  • 4
  • 4
  • +4
1 Solution
 
ginsonicCommented:
listening
0
 
kretzschmarCommented:
maybe you could show a bit code, how you do it now
0
 
classicsCommented:
Try setting the thread to a priority lower than the main program thread.  Before calling Resume just set Priority := tpLower;
0
Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
Stuart_JohnsonAuthor Commented:
meikl,

The code I use is around 3,000 lines+ long.  In a nut shell, all it does is query an SQL server (repeatedly) and creates Paradox files which are then used back in the main application to show the reports.  The slow process is the actual copying of the data - especially on slower databases (like our old AS/400 for instance).

classics,

I've tried that as well.  I've also set it to idle and that didn't make any difference.  It's as though the thread gets highest priority no matter what.

My understanding of a thread (and this maybe completely misguided) is that if an application executes a thread, the main application can continue doing it's thing without being ground to a halt (if the priority isn't set too high).  Is this the case?

Most of our reports create the temporary tables using just a simple line by append, but when a batch move is called, that's when the system stops responding.  We need to be able to use the main application regardless of how we copy the data - even if the main app is a bit slow (but not frozen).

Thanks so far guys,

Stu.

0
 
classicsCommented:
Are you calling a lot of code in Synchronize()?

That will block your main thread and any other thread waiting to use VCL functions.
0
 
kretzschmarCommented:
well stuart,
as delphi didn't provide a real multithreading-model
(all threads do affect the mainthread),
especially if you use the bde (do you?),
therefore
another suggestion could be to start a separat app from your app, which does this work in the background

may this possible for you?

meikl ;-)
0
 
SteveWaiteCommented:
Another thread is great until you have to get some value from it within the main gui thread.
if your thread calls Synchronize because it accesses the vcl or data stored in the gui thread then one thread waits for the other. This is why you get a pause.
A separate thread should use notify events to inform the main thread of progress.
So, really you need to avoid Synchronize() and instead wait until your thread has finished work, then use notify event to tell main thread.
Also can you can put it in a dll instead.
Let me know what help you need with those ideas.
0
 
Stuart_JohnsonAuthor Commented:
Hi Classics,

Yes I am calling it from within a Synchronize.  I'm just working off examples I've downloaded, and in each example they wrap the functions in Synchronize.  I'll explain a bit more later on what's actually happening with the thread.

Hi Meikl,

This was considered as well, but it means that we have to rely on this working 100% of the time.  Some of our users are pretty bloody stupid, and going down this path maybe more of a nightmare than anything else.  But it hasn't been ruled out  yet :)  And yes, we do use the BDE.  Is there another way I can avoid the BDE and create Paradox files, but also be able to read from either an AS/400, MS-SQL, Oracle or Interbase (these are our supported databases)?

Hi Steve,

I think you've hit the closest to what I'm actually refering too.  I would like a little more info on this if possible, but I'll write a few more comments below to give a bit more info to everyone.



This thread is created when the application first loads and is terminated when the app is closing.  Basically this thread is a spooler.  We can pass the thread a command, and it will create us a single paradox file (with a unique name).

The thread maintains a list of pending, active and completed jobs (until the thread is terminated).  This list is a TObjectList (which contains classes as:

type
  TJobDetails = class
    JobID: Integer; {Unique job ID}
    JobPriority: Integer; {0 - 5, 0=lowest, 5=highest}
    ReportNumber: Integer;
    Paused: Boolean;
    Progress: Integer; {How far through processing we are}
    JobStatus: TJobStatus; {Pending, Active, Completed, Error}
  end;

I can query the thread, asking for details at anytime by calling:

function QueryJobDetails: Variant; overload;
function QueryJobDetails(JobID: Integer): Variant; overload;

The returned variant is a variant array which I pull to pieces in my app and then display that info in a TListView.  I can then right-click on any of the items and change it's job priority, cancel the job or pause it (you can not pause a job already processing, only jobs with a status of Pending).

And finally, I have an event that is triggered at the commencement and completion of each job.

So, I don't directly update any VCL from within the thread.

Is what I'm doing sounding OK?  I really need to be able to get this working so I can use the app at the same time as generating these reports.  I'm not overly fussed with how it's done, although I really would like to steer clear of the external application until all other avenues are exhausted.

Thanks again,

Stu
0
 
SteveWaiteCommented:
Sounds OK. Obviously restrict what is done in Synchronize.
Maybe you could use another thread to do communications between gui and worker threads. So having one controlling thread that spawns the worker thread(s).
0
 
MadshiCommented:
Yeah, the key point is using Synchronize. Synchronize is a hack, which moves the execution back into the main thread. So if you do all the work inside of the Synchronize, you're doing it in the main thread again...

Regards, Madshi.
0
 
Stuart_JohnsonAuthor Commented:
Guys,

I have this working perfectly!  What I would like to do is spit the points up between each of you.  Now, how do we do this.  classics, Steve and Meikl - 50 points each, or do I award the points to one expert?  I'm not fussed, you each helped me out differently although it was Steve who enlightened me on my slow redraws.  It's a shame there was no follow-up on the DLL.

Anyway, let me know which way you want it to go.

Cheers,

Stu
0
 
SteveWaiteCommented:
Do you want a dll example, or are you ok now?
0
 
Stuart_JohnsonAuthor Commented:
Hi Steve,

I have my application functioning beautifully at present, so the DLL in this instance is probably not required.  However, it would be interesting for future references, no doubt.  If you have time to do one, I'd appreciate it, but please, don't go to an overly silly amount of effort to do so.  You're help was great within this thread, so you've definately earnt your points.

Stu
0
 
kretzschmarCommented:
yep, grade SteveWaite,
there is no need to grade me

meikl ;-)
0
 
Stuart_JohnsonAuthor Commented:
Are you sure, Meikl?  I'm quite happy to split it.
0
 
kretzschmarCommented:
its your decission, stuart
0
 
SteveWaiteCommented:
For later on then..
Do a search for Madshi's dll keyboard hook example (can be found all over ex-ex).
It has a good example of putting work in a dll and sharing the memory correctly. You would use that to transfer your results. The hook stuff is a bonus.
Regards,
Steve
0
 
TOndrejCommented:
When working with BDE, each thread must use its own TSession, TDatabase and TTable/TQuery components.
More details in help:
DevGuide: Developing database applications
"Managing multiple sessions"
0
 
Stuart_JohnsonAuthor Commented:
Steve,

Thanks for you help with this.  I'll test out your suggestions as well.

Madshi, there is a question for you with some points.  Thank you too mate!

Stu
0
 
Stuart_JohnsonAuthor Commented:
TOndrej,

Thank-you for the extra tips.  Much appreciated.

Stu
0
 
Stuart_JohnsonAuthor Commented:
TOndrej,

I was wondering if you could give me a bit of info.  When I create my thread, I'm passing it the alias I need to connect to, plus the username and password.  I create a new TSession and a new TDatabase.  The session works fine, however, when I try to do:

ADatabase.DatabaseName := AliasName

I get External Exception C0000008.  I've looked on the net, but the only resolution I can find is I need to go back a version of the SQL drivers I'm using _IF_ I was using Oracle or Sybase (I'm using SQL Server 2000).

How can I get around this error?  What am I doing wrong?

Thanks,

Stu
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

  • 8
  • 4
  • 4
  • +4
Tackle projects and never again get stuck behind a technical roadblock.
Join Now