aztec
asked on
can do 64 threads, but not 65.....
Thanks to all who helped me with threads the last few weeks, I got my app up and running and it's working pretty good... but I've hit a new and strange snag:
Seems everything works fine when I have the number of threads less than or equal to 64, but as soon as I set it to 65 or over, I get an I/O Error 104, and the debugger points to the "while not eof(tinfile) do" line of this section of code in my "worker" thread:
while not eof(tinfile) do
begin
{* Start of Critical Section *}
EnterCriticalSection(CritS ect);
readln(tinfile, thinrec);
tbytes_processed:=tbytes_p rocessed+l ength(thin rec)+2;
thperc:=Form1.calc_status( ttotsize_i nfile, tbytes_processed);
if thperc >= tprevhperc+1 then
begin
tprevhperc:=thperc;
Form1.gauge2.progress:=thp erc;
Form1.refresh;
end;
LeaveCriticalSection(CritS ect);
etc. etc.....
I'm using a "WaitForMultipleObjects" statement to wait for all my threads to finish, before proceeding. I've got a feeling the problem might lie in there....when I execute my app outside of the IDE, it *immediately* finishes in the blink of an eye (no error 104 comes up). Which leads me to believe 65 threads or greater causes a problem for the "WaitForMultipleObjects" cause it seems to fly right past it (...65 is a very suspicious number to fail at). Here's my exact code for that part of my app:
procedure TWaitThr.Execute;
var
hndlArr : Array[0..9999] of THandle;
thrArr : Array[0..9999] of TWorkerThread;
I, dnsx : Integer;
begin
FreeOnTerminate := True;
for I := 0 to Form1.SpinEdit3.Value-1 do
begin
thrArr[I]:= TWorkerThread.Create(True) ;
thrArr[I].Resume;
hndlArr[I] := thrArr[I].Handle;
end;
WaitForMultipleObjects(For m1.SpinEdi t3.Value, @hndlArr, True, INFINITE);
Synchronize(UpdateLabel);
end;
...I call TWaitThr.Create from my main portion of my app. TWaitThr in turn calls and creates my "worker" threads.... then it waits.
Thanks!
Shawn
P.S: I'm using D6 Enterprise version.
Seems everything works fine when I have the number of threads less than or equal to 64, but as soon as I set it to 65 or over, I get an I/O Error 104, and the debugger points to the "while not eof(tinfile) do" line of this section of code in my "worker" thread:
while not eof(tinfile) do
begin
{* Start of Critical Section *}
EnterCriticalSection(CritS
readln(tinfile, thinrec);
tbytes_processed:=tbytes_p
thperc:=Form1.calc_status(
if thperc >= tprevhperc+1 then
begin
tprevhperc:=thperc;
Form1.gauge2.progress:=thp
Form1.refresh;
end;
LeaveCriticalSection(CritS
etc. etc.....
I'm using a "WaitForMultipleObjects" statement to wait for all my threads to finish, before proceeding. I've got a feeling the problem might lie in there....when I execute my app outside of the IDE, it *immediately* finishes in the blink of an eye (no error 104 comes up). Which leads me to believe 65 threads or greater causes a problem for the "WaitForMultipleObjects" cause it seems to fly right past it (...65 is a very suspicious number to fail at). Here's my exact code for that part of my app:
procedure TWaitThr.Execute;
var
hndlArr : Array[0..9999] of THandle;
thrArr : Array[0..9999] of TWorkerThread;
I, dnsx : Integer;
begin
FreeOnTerminate := True;
for I := 0 to Form1.SpinEdit3.Value-1 do
begin
thrArr[I]:= TWorkerThread.Create(True)
thrArr[I].Resume;
hndlArr[I] := thrArr[I].Handle;
end;
WaitForMultipleObjects(For
Synchronize(UpdateLabel);
end;
...I call TWaitThr.Create from my main portion of my app. TWaitThr in turn calls and creates my "worker" threads.... then it waits.
Thanks!
Shawn
P.S: I'm using D6 Enterprise version.
ASKER
ahhhh! OK Madshi thanks!
But is there any way around this? Perhaps another way to "wait" ? I'd like to be able to use more than 64 threads!
Ciao
Shawn
But is there any way around this? Perhaps another way to "wait" ? I'd like to be able to use more than 64 threads!
Ciao
Shawn
ASKER
ahhhh Ok Madshi thanks!But is there any way around this? Perhaps a different approach to use for "wait"? I'd like to be able to use more than 64 threads.
Ciao
Shawn
Ciao
Shawn
ASKER
(sorry for the double post.... it had looked like my first didn't go thru, and I re-posted)
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I am waiting for ALL the threads to finish before I proceed (and yes, they are all my own threads). I had *thought* about using more than one WaitForMultipleObjects statement...so this will work then?
Why so many threads? Just to try to get the *fastest* execution out of my app! Overkill? Maybe! :-)
Shawn
Why so many threads? Just to try to get the *fastest* execution out of my app! Overkill? Maybe! :-)
Shawn
>> Why so many threads? Just to try to get the *fastest* execution out of my app! Overkill? Maybe! :-)
Well, maybe with so many threads it will end up being slower than with only 1 thread!
Look, a thread can't do magic. Most PCs only have one CPU in it. And your program runs fastest if it uses all the CPU power. One thread can do that already, >64 threads can not make the CPU run at 110% of its power, that's not possible. So what do >64 threads bring? It will have one disadvantage: Namely the thread switching costs performance. You can say e.g. 5% of the CPU goes lost just by switching all your threads. It has one advantage, though: If the threads sometimes are waiting for something, e.g. for data to arrive at the COM port or such stuff, then threads are very good. If you had only one thread in such a situation, it would wait for the data, and the CPU would go to sleep. But when doing file stuff, multiple threads might do more damage than help. You should do some performance tests.
But back to your problem:
Let's say you have 100 threads running. If you now want to wait for ALL threads to be finished, this is exactly the same as first waiting for 64 threads to be finished, and then waiting for the remaining 36 threads to be finished. So you easily call WaitForMultipleObjects as often as you need to wait for all your threads. No problem.
Regards, Madshi.
Well, maybe with so many threads it will end up being slower than with only 1 thread!
Look, a thread can't do magic. Most PCs only have one CPU in it. And your program runs fastest if it uses all the CPU power. One thread can do that already, >64 threads can not make the CPU run at 110% of its power, that's not possible. So what do >64 threads bring? It will have one disadvantage: Namely the thread switching costs performance. You can say e.g. 5% of the CPU goes lost just by switching all your threads. It has one advantage, though: If the threads sometimes are waiting for something, e.g. for data to arrive at the COM port or such stuff, then threads are very good. If you had only one thread in such a situation, it would wait for the data, and the CPU would go to sleep. But when doing file stuff, multiple threads might do more damage than help. You should do some performance tests.
But back to your problem:
Let's say you have 100 threads running. If you now want to wait for ALL threads to be finished, this is exactly the same as first waiting for 64 threads to be finished, and then waiting for the remaining 36 threads to be finished. So you easily call WaitForMultipleObjects as often as you need to wait for all your threads. No problem.
Regards, Madshi.
ASKER
OK, I will try the multiple "WaitForMultipleObjects" statements. Madshi, would this be the way to do it if I allow a maximum of 128 threads (user enters # of threads in Form1.SpinEdit3):
if Form1.SpinEdit3.Value <= 64 then
WaitForMultipleObjects(For m1.SpinEdi t3.Value, @hndlArr, True, INFINITE)
else if (Form1.SpinEdit3.Value > 64) and (Form1.SpinEdit3.Value <= 128) then
begin
WaitForMultipleObjects(64, @hndlArr, True, INFINITE);
WaitForMultipleObjects(For m1.SpinEdi t3.Value-6 4, @hndlArr[64], True, INFINITE);
end;
..does this look correct to you? I am wondering about the "@hndlArr[64]". Is this the correct way to do it, so it will start with the the 65th element of hndlArr?
Regarding hndlArr:
(1) Do I need brackets around it, like this :
@(hndlArr[64]) ?
(2) Should I initialize all elements of the hndlArr first before using it? If so, what do I initialize it with ... a zero?
Ciao!
Shawn
if Form1.SpinEdit3.Value <= 64 then
WaitForMultipleObjects(For
else if (Form1.SpinEdit3.Value > 64) and (Form1.SpinEdit3.Value <= 128) then
begin
WaitForMultipleObjects(64,
WaitForMultipleObjects(For
end;
..does this look correct to you? I am wondering about the "@hndlArr[64]". Is this the correct way to do it, so it will start with the the 65th element of hndlArr?
Regarding hndlArr:
(1) Do I need brackets around it, like this :
@(hndlArr[64]) ?
(2) Should I initialize all elements of the hndlArr first before using it? If so, what do I initialize it with ... a zero?
Ciao!
Shawn
Looks alright on a quick look.
(1) No brackets are needed.
(2) Initializing is safer, you should initialize with zero.
Regards, Madshi.
(1) No brackets are needed.
(2) Initializing is safer, you should initialize with zero.
Regards, Madshi.
ASKER
thanks Madshi - works fine!
Ciao
Shawn
Ciao
Shawn
"nCount
Specifies the number of object handles in the array pointed to by lpHandles. The maximum number of object handles is MAXIMUM_WAIT_OBJECTS."
And guess what MAXIMUM_WAIT_OBJECTS is like? Yes, you guessed right, it's 64... :-)
Regards, Madshi.