Stuart_Johnson
asked on
Thread Question
Our current application has a reporting tool in it. It can generate upto 30 different reports. The problem is, the reports sometime take FOREVER (I'm talking hours here) to generate. What we want to do is move all the generation code into a thread, pass it the queries and instructions and let it go. This will enable us to queue jobs and still allow people to continue using our app - in theory :)
I've already done the ground work. I've written the functions to do the reports, and I tried putting in a thread, and that's were the problem starts.
When the code is executed, my app stops responding until the report has finished. Why is this? Is there anyway I can make the app work while the report is generating?
I'm still a thread newby, so any help would be appreciated.
Thanks,
Stu
I've already done the ground work. I've written the functions to do the reports, and I tried putting in a thread, and that's were the problem starts.
When the code is executed, my app stops responding until the report has finished. Why is this? Is there anyway I can make the app work while the report is generating?
I'm still a thread newby, so any help would be appreciated.
Thanks,
Stu
ginsonic
listening
maybe you could show a bit code, how you do it now
Try setting the thread to a priority lower than the main program thread. Before calling Resume just set Priority := tpLower;
ASKER
meikl,
The code I use is around 3,000 lines+ long. In a nut shell, all it does is query an SQL server (repeatedly) and creates Paradox files which are then used back in the main application to show the reports. The slow process is the actual copying of the data - especially on slower databases (like our old AS/400 for instance).
classics,
I've tried that as well. I've also set it to idle and that didn't make any difference. It's as though the thread gets highest priority no matter what.
My understanding of a thread (and this maybe completely misguided) is that if an application executes a thread, the main application can continue doing it's thing without being ground to a halt (if the priority isn't set too high). Is this the case?
Most of our reports create the temporary tables using just a simple line by append, but when a batch move is called, that's when the system stops responding. We need to be able to use the main application regardless of how we copy the data - even if the main app is a bit slow (but not frozen).
Thanks so far guys,
Stu.
The code I use is around 3,000 lines+ long. In a nut shell, all it does is query an SQL server (repeatedly) and creates Paradox files which are then used back in the main application to show the reports. The slow process is the actual copying of the data - especially on slower databases (like our old AS/400 for instance).
classics,
I've tried that as well. I've also set it to idle and that didn't make any difference. It's as though the thread gets highest priority no matter what.
My understanding of a thread (and this maybe completely misguided) is that if an application executes a thread, the main application can continue doing it's thing without being ground to a halt (if the priority isn't set too high). Is this the case?
Most of our reports create the temporary tables using just a simple line by append, but when a batch move is called, that's when the system stops responding. We need to be able to use the main application regardless of how we copy the data - even if the main app is a bit slow (but not frozen).
Thanks so far guys,
Stu.
Are you calling a lot of code in Synchronize()?
That will block your main thread and any other thread waiting to use VCL functions.
That will block your main thread and any other thread waiting to use VCL functions.
well stuart,
as delphi didn't provide a real multithreading-model
(all threads do affect the mainthread),
especially if you use the bde (do you?),
therefore
another suggestion could be to start a separat app from your app, which does this work in the background
may this possible for you?
meikl ;-)
as delphi didn't provide a real multithreading-model
(all threads do affect the mainthread),
especially if you use the bde (do you?),
therefore
another suggestion could be to start a separat app from your app, which does this work in the background
may this possible for you?
meikl ;-)
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Hi Classics,
Yes I am calling it from within a Synchronize. I'm just working off examples I've downloaded, and in each example they wrap the functions in Synchronize. I'll explain a bit more later on what's actually happening with the thread.
Hi Meikl,
This was considered as well, but it means that we have to rely on this working 100% of the time. Some of our users are pretty bloody stupid, and going down this path maybe more of a nightmare than anything else. But it hasn't been ruled out yet :) And yes, we do use the BDE. Is there another way I can avoid the BDE and create Paradox files, but also be able to read from either an AS/400, MS-SQL, Oracle or Interbase (these are our supported databases)?
Hi Steve,
I think you've hit the closest to what I'm actually refering too. I would like a little more info on this if possible, but I'll write a few more comments below to give a bit more info to everyone.
This thread is created when the application first loads and is terminated when the app is closing. Basically this thread is a spooler. We can pass the thread a command, and it will create us a single paradox file (with a unique name).
The thread maintains a list of pending, active and completed jobs (until the thread is terminated). This list is a TObjectList (which contains classes as:
type
TJobDetails = class
JobID: Integer; {Unique job ID}
JobPriority: Integer; {0 - 5, 0=lowest, 5=highest}
ReportNumber: Integer;
Paused: Boolean;
Progress: Integer; {How far through processing we are}
JobStatus: TJobStatus; {Pending, Active, Completed, Error}
end;
I can query the thread, asking for details at anytime by calling:
function QueryJobDetails: Variant; overload;
function QueryJobDetails(JobID: Integer): Variant; overload;
The returned variant is a variant array which I pull to pieces in my app and then display that info in a TListView. I can then right-click on any of the items and change it's job priority, cancel the job or pause it (you can not pause a job already processing, only jobs with a status of Pending).
And finally, I have an event that is triggered at the commencement and completion of each job.
So, I don't directly update any VCL from within the thread.
Is what I'm doing sounding OK? I really need to be able to get this working so I can use the app at the same time as generating these reports. I'm not overly fussed with how it's done, although I really would like to steer clear of the external application until all other avenues are exhausted.
Thanks again,
Stu
Yes I am calling it from within a Synchronize. I'm just working off examples I've downloaded, and in each example they wrap the functions in Synchronize. I'll explain a bit more later on what's actually happening with the thread.
Hi Meikl,
This was considered as well, but it means that we have to rely on this working 100% of the time. Some of our users are pretty bloody stupid, and going down this path maybe more of a nightmare than anything else. But it hasn't been ruled out yet :) And yes, we do use the BDE. Is there another way I can avoid the BDE and create Paradox files, but also be able to read from either an AS/400, MS-SQL, Oracle or Interbase (these are our supported databases)?
Hi Steve,
I think you've hit the closest to what I'm actually refering too. I would like a little more info on this if possible, but I'll write a few more comments below to give a bit more info to everyone.
This thread is created when the application first loads and is terminated when the app is closing. Basically this thread is a spooler. We can pass the thread a command, and it will create us a single paradox file (with a unique name).
The thread maintains a list of pending, active and completed jobs (until the thread is terminated). This list is a TObjectList (which contains classes as:
type
TJobDetails = class
JobID: Integer; {Unique job ID}
JobPriority: Integer; {0 - 5, 0=lowest, 5=highest}
ReportNumber: Integer;
Paused: Boolean;
Progress: Integer; {How far through processing we are}
JobStatus: TJobStatus; {Pending, Active, Completed, Error}
end;
I can query the thread, asking for details at anytime by calling:
function QueryJobDetails: Variant; overload;
function QueryJobDetails(JobID: Integer): Variant; overload;
The returned variant is a variant array which I pull to pieces in my app and then display that info in a TListView. I can then right-click on any of the items and change it's job priority, cancel the job or pause it (you can not pause a job already processing, only jobs with a status of Pending).
And finally, I have an event that is triggered at the commencement and completion of each job.
So, I don't directly update any VCL from within the thread.
Is what I'm doing sounding OK? I really need to be able to get this working so I can use the app at the same time as generating these reports. I'm not overly fussed with how it's done, although I really would like to steer clear of the external application until all other avenues are exhausted.
Thanks again,
Stu
Sounds OK. Obviously restrict what is done in Synchronize.
Maybe you could use another thread to do communications between gui and worker threads. So having one controlling thread that spawns the worker thread(s).
Maybe you could use another thread to do communications between gui and worker threads. So having one controlling thread that spawns the worker thread(s).
Yeah, the key point is using Synchronize. Synchronize is a hack, which moves the execution back into the main thread. So if you do all the work inside of the Synchronize, you're doing it in the main thread again...
Regards, Madshi.
Regards, Madshi.
ASKER
Guys,
I have this working perfectly! What I would like to do is spit the points up between each of you. Now, how do we do this. classics, Steve and Meikl - 50 points each, or do I award the points to one expert? I'm not fussed, you each helped me out differently although it was Steve who enlightened me on my slow redraws. It's a shame there was no follow-up on the DLL.
Anyway, let me know which way you want it to go.
Cheers,
Stu
I have this working perfectly! What I would like to do is spit the points up between each of you. Now, how do we do this. classics, Steve and Meikl - 50 points each, or do I award the points to one expert? I'm not fussed, you each helped me out differently although it was Steve who enlightened me on my slow redraws. It's a shame there was no follow-up on the DLL.
Anyway, let me know which way you want it to go.
Cheers,
Stu
Do you want a dll example, or are you ok now?
ASKER
Hi Steve,
I have my application functioning beautifully at present, so the DLL in this instance is probably not required. However, it would be interesting for future references, no doubt. If you have time to do one, I'd appreciate it, but please, don't go to an overly silly amount of effort to do so. You're help was great within this thread, so you've definately earnt your points.
Stu
I have my application functioning beautifully at present, so the DLL in this instance is probably not required. However, it would be interesting for future references, no doubt. If you have time to do one, I'd appreciate it, but please, don't go to an overly silly amount of effort to do so. You're help was great within this thread, so you've definately earnt your points.
Stu
yep, grade SteveWaite,
there is no need to grade me
meikl ;-)
there is no need to grade me
meikl ;-)
ASKER
Are you sure, Meikl? I'm quite happy to split it.
its your decission, stuart
For later on then..
Do a search for Madshi's dll keyboard hook example (can be found all over ex-ex).
It has a good example of putting work in a dll and sharing the memory correctly. You would use that to transfer your results. The hook stuff is a bonus.
Regards,
Steve
Do a search for Madshi's dll keyboard hook example (can be found all over ex-ex).
It has a good example of putting work in a dll and sharing the memory correctly. You would use that to transfer your results. The hook stuff is a bonus.
Regards,
Steve
When working with BDE, each thread must use its own TSession, TDatabase and TTable/TQuery components.
More details in help:
DevGuide: Developing database applications
"Managing multiple sessions"
More details in help:
DevGuide: Developing database applications
"Managing multiple sessions"
ASKER
Steve,
Thanks for you help with this. I'll test out your suggestions as well.
Madshi, there is a question for you with some points. Thank you too mate!
Stu
Thanks for you help with this. I'll test out your suggestions as well.
Madshi, there is a question for you with some points. Thank you too mate!
Stu
ASKER
TOndrej,
Thank-you for the extra tips. Much appreciated.
Stu
Thank-you for the extra tips. Much appreciated.
Stu
ASKER
TOndrej,
I was wondering if you could give me a bit of info. When I create my thread, I'm passing it the alias I need to connect to, plus the username and password. I create a new TSession and a new TDatabase. The session works fine, however, when I try to do:
ADatabase.DatabaseName := AliasName
I get External Exception C0000008. I've looked on the net, but the only resolution I can find is I need to go back a version of the SQL drivers I'm using _IF_ I was using Oracle or Sybase (I'm using SQL Server 2000).
How can I get around this error? What am I doing wrong?
Thanks,
Stu
I was wondering if you could give me a bit of info. When I create my thread, I'm passing it the alias I need to connect to, plus the username and password. I create a new TSession and a new TDatabase. The session works fine, however, when I try to do:
ADatabase.DatabaseName := AliasName
I get External Exception C0000008. I've looked on the net, but the only resolution I can find is I need to go back a version of the SQL drivers I'm using _IF_ I was using Oracle or Sybase (I'm using SQL Server 2000).
How can I get around this error? What am I doing wrong?
Thanks,
Stu