Solved

multiple threads

Posted on 2002-07-10
40
527 Views
Last Modified: 2010-04-04
Hi,
   How do I run multiple threads of the same group of 3 or 4 procedures in my app?
   Or do I have to put the code that I want to have mult. threads for, in its own separate unit first?

I know *nothing* about threads/threading processes!

Thanks
   Shawn

P.S: I'm using Delphi 6.
0
Comment
Question by:aztec
  • 19
  • 12
  • 6
  • +1
40 Comments
 
LVL 11

Expert Comment

by:robert_marquardt
Comment Utility
You should first tell us what the procedures do then we can tell you if threading can help at all.
Do you have more than one CPU in your computer?
0
 

Author Comment

by:aztec
Comment Utility
Sure, I'm reading a file containing email addresses, then attempting to verify each one of them.
  Ideally, I'd like to split the file into chunks and assign each chunk to an individual thread.

Thanks
   Shawn
0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
I would read the file in the main thread, and pass each email address to an instance of your "checking" thread.
As each thread is created, keep a counter on the number of threads running, and decrement the counter as each thread completes.
Your main file reading routine simply reads the email address and idles when the thread count gets to some pre-determined maximum. That way, as each thread dies, a new thread will be created, keeping the threadcount at maximum until the file eventually runs out.
The max number of thread allowed will depend on the number of processors, speed etc, and will be best determined by trial and error. Too many, and you bog everything down, too few and you won't get the most benefit.
With regards to multithreading your procedures, make sure that local variable are declared in each procedure as needed, or if need to be "global" to a group of procedures, make the procedures part of the thread class, and declare the variables in the thread class. That way, each instance of the 'checking' thread has it's own set of variables, and they won't interfere with each other.
0
 
LVL 11

Expert Comment

by:robert_marquardt
Comment Utility
I would do it another way.
The main program reads the file. either one address per request or all addresses at once. That depends on the size of the file.

Create a worker threads.

Each thread requests an address from the main program (synchronized) and handles it. The thread dies if it does not get an address on request.

Now create new threads until the throughput does not improve anymore. The throughput is measured in the main program by timing the requests.
0
 

Author Comment

by:aztec
Comment Utility
Thanks, I will try both of your methods suggested.

But the thing is - I've never programmed threads before and have no idea where or how to start!

Can you perhaps give me a basic skeleton or outline in which I can fill in my specific code?

Thanks!
   Shawn
0
 
LVL 7

Expert Comment

by:Cynna
Comment Utility
aztec,

Before you go on, I commented on similar question in another thread, please read my last comment here:

http://www.experts-exchange.com/delphi/Q_20320235.html

0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
Robert, as usual there is more than one correct way to write code, however I question the logic behind creating worker threads that die if they don't get an address. Why create them them if there is nothing fo them to do? Plus there is the hassle of the thread sync'ing with the main thread.

Aztec, try something like this:

it's not complete, I gotta go out now, but it's maybe a starting point for you...



unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs;

type
  TCheckThread = class(TThread)
  private
    fAddress : string;
    FFormattedOK: boolean;
    FValidAddress: boolean;
    procedure SetAddress(const Value: string);
    function IsFormattedOK(value : string) : boolean;
    function CheckAddress : boolean;
  public
    procedure Execute;override;
    property Address : string read FAddress write SetAddress;
    property FormattedOK : boolean read FFormattedOK write FFormattedOK;
    property ValidAddress : boolean read FValidAddress write FValidAddress;
  end;

  TForm1 = class(TForm)
  private
    procedure CheckThreadTerminate(Sender : TObject);
    procedure CheckAddressFile;
    { Private declarations }
  public
    { Public declarations }
  end;

Const
MAX_THREADS = 30;

var
  Form1: TForm1;
  ThreadCount : integer;

implementation

{$R *.DFM}


{ TCheckThread }

function TCheckThread.CheckAddress: boolean;
begin
  // your code to actually check the e-mail address goes here....
end;

procedure TCheckThread.Execute;
begin
  fValidAddress := false;
  if fFormattedOK then
      fValidAddress := CheckAddress;
end;

function TCheckThread.IsFormattedOK(value: string): boolean;
begin
  // your code to verify a correctly formatted address goes here....
end;

procedure TCheckThread.SetAddress(const Value: string);
begin
  if IsFormattedOK(Value) then
    begin
      FAddress := Value;
      FFormattedOK := true;
    end
  else
    FFormattedOK := false;
end;


{ TForm1 }

procedure TForm1.CheckAddressFile;
var
  MyAddress : string;
  AThread : TCheckThread;
begin
  //open the file
  // while not file.eof do...
   while ThreadCount >= MAX_THREADS do
     application.processmessages; // loop if at max thread...

    // read one address into "Myaddress" variable
    AThread := TCheckThread.create(true); // create suspended
    inc(ThreadCount);
    with PrimaryThread do
      begin
        OnTerminate := CheckThreadTerminate;
        FreeOnTerminate := true;
        Address := MyAddress; // set the address to be checked
        Resume; // and let the thread run...
      end;
   // loop back to read next address
end;

procedure TForm1.CheckThreadTerminate(Sender: TObject);
begin
  with (Sender as TCheckThread) do
    begin
      //check "formattedOK' and "ValidAddress" properties to see if it worked...
    end;
  dec(ThreadCount);
end;

end.


0
 
LVL 11

Expert Comment

by:robert_marquardt
Comment Utility
mullet_attack, one thread per address may produce too much overhead with creating and destroying threads.
If a thread does not get an address from the main thread means that no more addresses are available.
Synchronizing with the main program should be less time consuming than creating a thread.

My approach also gives feedback on performance. Start threads until performance does not increase anymore. Performance measured by counting address requests per minute (or second).
0
 

Author Comment

by:aztec
Comment Utility
Robert - how do you "synchronize"?

Thanks
   Shawn
0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
what if there are 5000 addresses to check? Do you propose that threads get created at will? what if it takes 10 seconds to check each address? Will you have 5000 threads all trying to run? Hope you have lots of ram and processors :-)

Or, do you mean create a limited number of worker threads, say 30? If so , then really what is the difference apart from semantics?
0
 
LVL 11

Expert Comment

by:robert_marquardt
Comment Utility
The difference is that my threads will not handle only one message. That eliminates the overhead of constantly starting new threads.
0
 

Author Comment

by:aztec
Comment Utility
Robert/Mullet/Cynna - I wish to have each thread output its result (ie. a valid email address found) to just *one* main Output File. I'd assume that I'd require some sort of synchronization between threads then, yes? If so, can you point out briefly how this would be accomplished?

Thanks
   Shawn
0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
the code below is a working version of my previous code
it 'pretends' there are 10 thousand adresses in the file, creating and destroying a thread for each one. I know there are other factors (such as the loop timing, and updating the labels) but the thing runs in 46 seconds on my PIII 1Ghz, Win2k 384Mb ram PC. That's about 4 and a half milliseconds per thread overhead. Not much.
Time stays pretty constant for 5 to around 50 MAX_THREADS, then starts to drop off, as expected. The optimum number of threads in a real app will be determined by what the thread has to do, eg internet bandwidth available, response time from the "thing" that checks the email addresses etc.

unit Unit1;

interface

uses
 Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  StdCtrls;

type
 TCheckThread = class(TThread)
 private
   fAddress : string;
   FFormattedOK: boolean;
   FValidAddress: boolean;
   procedure SetAddress(const Value: string);
   function IsFormattedOK(value : string) : boolean;
   function CheckAddress : boolean;
 public
   procedure Execute;override;
   property Address : string read FAddress write SetAddress;
   property FormattedOK : boolean read FFormattedOK write FFormattedOK;
   property ValidAddress : boolean read FValidAddress write FValidAddress;
 end;

 TForm1 = class(TForm)
    Button1: TButton;
    Label1: TLabel;
    Label2: TLabel;
    procedure Button1Click(Sender: TObject);
 private
   procedure CheckThreadTerminate(Sender : TObject);
   procedure CheckAddressFile;
   { Private declarations }
 public
   { Public declarations }
 end;

Const
MAX_THREADS = 30;

var
 Form1: TForm1;
 ThreadCount : integer;

implementation

{$R *.DFM}


{ TCheckThread }

function TCheckThread.CheckAddress: boolean;
begin
 // your code to actually check the e-mail address goes here....
end;

procedure TCheckThread.Execute;
begin
 fValidAddress := false;
 if fFormattedOK then
     fValidAddress := CheckAddress;
end;

function TCheckThread.IsFormattedOK(value: string): boolean;
begin
 // your code to verify a correctly formatted address goes here....
end;

procedure TCheckThread.SetAddress(const Value: string);
begin
 if IsFormattedOK(Value) then
   begin
     FAddress := Value;
     FFormattedOK := true;
   end
 else
   FFormattedOK := false;
end;


{ TForm1 }

procedure TForm1.CheckAddressFile;
var
 MyAddress : string;
 AThread : TCheckThread;
 t : integer;
begin
  for t := 0 to 9999 do
    begin
      while ThreadCount >= MAX_THREADS do
        begin
          application.processmessages; // loop if at max thread...
          label1.caption := Format('Number of addreses = %d',[t]);
          label2.caption := Format('Number of threads = %d',[ThreadCount]);
        end;
       // read one address into "Myaddress" variable
       AThread := TCheckThread.create(true); // create suspended
       inc(ThreadCount);
       with AThread do
         begin
           OnTerminate := CheckThreadTerminate;
           FreeOnTerminate := true;
           Address := MyAddress; // set the address to be checked
           Resume; // and let the thread run...
         end;
     end;
   ShowMessage('Done');
end;

procedure TForm1.CheckThreadTerminate(Sender: TObject);
begin
 with (Sender as TCheckThread) do
   begin
     //check "formattedOK' and "ValidAddress" properties to see if it worked...
   end;
 dec(ThreadCount);
end;

procedure TForm1.Button1Click(Sender: TObject);
begin
  CheckAddressfile;
end;

end.
0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
threadterminate is executed in the context of the main thread, therefore it is already syncronised.

eg

procedure TForm1.CheckThreadTerminate(Sender: TObject);
begin
with (Sender as TCheckThread) do
  begin
    if ValidAddress then
      writetofile(address + 'ok or whatever');
  end;
dec(ThreadCount);
end;
0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
BTW Cynna, I agree "Threads can make your head hurt "

Aztec, have a good long look at the threads example that comes with Delphi (sort threads example) and you will see what is meant by "head hurt" :-)
0
 

Author Comment

by:aztec
Comment Utility
thanks Mullet - but what if there is like a million email addresses in my Input File.... it will make a million threads? This will be still efficient?

Thanks
   Shawn
0
 

Author Comment

by:aztec
Comment Utility
...and also, it is not necessary to put the thread procedures in a separate Unit by themselves?

Shawn

0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
Aztec, number of thread is limited to the value of the constant MAX_THREADS

If the process of checking the email address is relatively long by comparison to the thread creation time, say 5 seconds compared to 4.5 milliseconds, then it will really make no difference if you use my method or Roberts.
If, on the otherhand the checking process per email is very fast, say 10 milliseconds, then I agree with Robert that the thread overhead would be significant.

As far as putting thread code in it's own unit, it's not mandatory, but I usually would do it just to make the code more readable and organised. I just put it in one unit for the example to make posting the code here easier.

Mullet.
0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
Folks, I just re-wrote my code to do it Robert's way, and I have to concede that it is faster.

From a programming perspecive, I like to create a thread, give it what data it needs, and then send it out into the world to do it's job, instead of it calling back to the main thread to get the data it needs.

I suppose both approaches have their merits, but in this case from what you say about your app, if it was me i'd go with Robert's code for speed.

On the otherhand, my code is probably easier for a newbie thread writer. Good luck !

Here's the code using Roberts method. I added a delay to the thread's execute method so you can see what it is doing (like I said, it's _faster_!)


unit Unit1;

interface

uses
 Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  StdCtrls;

type
 TCheckThread = class(TThread)
 private
   fAddress : string;
   fNumber : integer;
   FFormattedOK: boolean;
   FValidAddress: boolean;
   procedure SetAddress(const Value: string);
   function IsFormattedOK(value : string) : boolean;
   function CheckAddress : boolean;
   procedure GetNextAddress;
 public
   procedure Execute;override;
   property Address : string read FAddress write SetAddress;
   property FormattedOK : boolean read FFormattedOK write FFormattedOK;
   property ValidAddress : boolean read FValidAddress write FValidAddress;
 end;

 TForm1 = class(TForm)
    Button1: TButton;
    Label1: TLabel;
    Label2: TLabel;
    procedure Button1Click(Sender: TObject);
 private
   procedure CheckAddressFile;
   { Private declarations }
 public
   function GetNextNumber:integer;
   { Public declarations }
 end;

Const
MAX_THREADS = 30;
MAX_ADDRESSES = 99;
var
 Form1: TForm1;
 AddressCounter : integer;

implementation

{$R *.DFM}


{ TCheckThread }

function TCheckThread.CheckAddress: boolean;
begin
 // your code to actually check the e-mail address goes here....
end;

procedure TCheckThread.Execute;
var
  NewNum : integer;
begin
 fNumber := -1;
 Synchronize(GetNextAddress);
 while fNumber > -1 do
   begin
     fValidAddress := false;
     if fFormattedOK then
         fValidAddress := CheckAddress;
     Synchronize(GetNextAddress);
     sleep(5000); //simulate some processing time for thread
   end;
end;

procedure TCheckThread.GetNextAddress;
begin
  fNumber := Form1.GetNextNumber;
end;


function TCheckThread.IsFormattedOK(value: string): boolean;
begin
 // your code to verify a correctly formatted address goes here....
end;

procedure TCheckThread.SetAddress(const Value: string);
begin
 if IsFormattedOK(Value) then
   begin
     FAddress := Value;
     FFormattedOK := true;
   end
 else
   FFormattedOK := false;
end;


{ TForm1 }

procedure TForm1.CheckAddressFile;
var
 MyAddress : string;
 AThread : TCheckThread;
 t : integer;
begin
  for t := 0 to MAX_THREADS-1 do //create 30 worker threads
    begin
       AThread := TCheckThread.create(true); // create suspended
       with AThread do
         begin
           FreeOnTerminate := true;
           Resume; // and let the thread run...
         end;
     end;
end;


procedure TForm1.Button1Click(Sender: TObject);
begin
  AddressCounter := 0;
  CheckAddressfile;
end;

function TForm1.GetNextNumber: integer;
begin
  if AddressCounter < MAX_ADDRESSES - 1 then
    begin
      inc(AddressCounter);
      result := AddressCounter;
    end
  else
    begin
      result := -1; // flag to say we are finished
//      ShowMessage('Done');
    end;
    label1.caption := format('Address counter = %d',[AddressCounter]);
end;


end.
0
 
LVL 11

Expert Comment

by:robert_marquardt
Comment Utility
Points to mullet_attack because he did the work.
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 
LVL 7

Expert Comment

by:Cynna
Comment Utility
I second that; mullet_attack should get the points.
0
 

Author Comment

by:aztec
Comment Utility
Thanks Mullet - but in your new revision, I don't see where I would read in the addresses from my Input File. And also I can't see where I'd write to my Output File.

I also don't see what the 'MyAddress' variable does in procedure TForm1.CheckAddressFile.


Thanks
   Shawn
0
 
LVL 2

Accepted Solution

by:
mullet_attack earned 100 total points
Comment Utility
This is a tidied up "revision". previous code was just quick to demonstrate the other method, so it was messy.

I haven't run his code, so there might be a small error or two, but it does compile.

I added the "linenumber" variable so that when your results are returned you can write then out to the file in the same order they were read in. May not be important, but you can do it if you want.

unit Unit1;

interface

uses
 Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  StdCtrls;

type
 TCheckThread = class(TThread)
 private
   fAddress : string;
   fLineNumber : integer;
   FFormattedOK: boolean;
   FValidAddress: boolean;
   function IsFormattedOK : boolean;
   function CheckAddress : boolean;
   procedure GetNextAddress;
   procedure AddressResult;
 public
   procedure Execute;override;
 end;

 TForm1 = class(TForm)
    Button1: TButton;
    Label1: TLabel;
    Label2: TLabel;
    procedure Button1Click(Sender: TObject);
 private
   procedure CheckAddressFile;
   { Private declarations }
 public
   procedure GetNextAddress(var Address : string ; var LineNumber : integer);
   procedure AddressResult(Address : string; LineNumber : integer ;FormattedOK, ValidAddress : boolean);
 end;

Const
MAX_THREADS = 30;
MAX_ADDRESSES = 99;
var
 Form1: TForm1;
 AddressCounter : integer;

implementation

{$R *.DFM}


{ TCheckThread }


procedure TCheckThread.Execute;
var
  NewNum : integer;
begin
 fLineNumber := -1;
 Synchronize(GetNextAddress);
 while fLineNumber > -1 do
   begin
     if IsFormattedOK then
       CheckAddress;
     Synchronize(AddressResult);
     Synchronize(GetNextAddress);
     sleep(5000); //simulate some processing time for thread
   end;
end;

procedure TCheckThread.GetNextAddress;
begin
  Form1.GetNextAddress(fAddress, FLineNumber);
end;


function TCheckThread.IsFormattedOK: boolean;
var
  OK : boolean; //just so code compiles...
begin
 // your code to verify a correctly formatted address goes here....
   if OK then
     fFormattedOK := true
   else
     fFormattedOK := false;
   result := fFormattedOK;
end;

function TCheckThread.CheckAddress: boolean;
var
  OK : boolean; //just so code compiles...
begin
 // your code to actually check the e-mail address goes here....
   if OK then
     fValidAddress := true
   else
     fValidAddress := false;
   result := fValidAddress;
end;


procedure TCheckThread.AddressResult;
begin
  Form1.AddressResult(fAddress, fLineNumber, fFormattedOK, fValidAddress);
end;


{ TForm1 }

procedure TForm1.CheckAddressFile;
var
 MyAddress : string;
 AThread : TCheckThread;
 t : integer;
begin
  for t := 0 to MAX_THREADS-1 do //create 30 worker threads
    begin
       AThread := TCheckThread.create(true); // create suspended
       with AThread do
         begin
           FreeOnTerminate := true;
           Resume; // and let the thread run...
         end;
     end;
end;


procedure TForm1.Button1Click(Sender: TObject);
begin
  AddressCounter := 0;
  CheckAddressfile;
end;

procedure TForm1.GetNextAddress(var Address : string ; var LineNumber : integer);
var
  AddressFromFile : string;
begin
  //read next address from file
  // if not EOF then
  //  begin
       inc(AddressCounter);
       Address := AddressFromFile;
       LineNumber := AddressCounter;
  //  end
  //    else
      Address := '';
      LineNumber := -1;

end;


procedure TForm1.AddressResult(Address: string; LineNumber: integer;
  FormattedOK, ValidAddress: boolean);
begin
// your code to process the results goes here
end;

end.
0
 

Author Comment

by:aztec
Comment Utility
ok, that clears it up... sorry to be so dense - this threading stuff is brand new to me!


One last thing, Robert had mentioned this before:

"Start threads until performance does not increase anymore. Performance measured by counting address requests per minute (or second). "

...I'm not sure how to do this. It appears your code sets the threads at 30 and is not changeable during run-time. There is some way to change this on-the-fly and add more threads if higher performance can be attained?

   The way Robert described sounds like a very nice "dynamic" solution that would 'self-adjust' itself until the optimum # of threads were arrived at.

Thanks
   Shawn
0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
The only dense people are the ones that _don't_ ask questions....

I wrote the code primarily to compare methods, hence the constant number of threads.

I'm not sure the added complexity of dynamic thread quantity would would be worth the effort versus performance gain, but then again I may be wrong here too !

Robert, what do you think? If you have ellegant solution I would be interested in seeing it also.

BTW, my solution was based on an app that had data packets coming in at random over IP socket, sometimes 1 per 10 seonds, othertimes every 10ms, in bursts. So at low traffic rates there was only ever 1 thread, but as traffice increased so would the number of threads in use, so it was dynamic. Aztec app has a constant, high rate of "traffic" until the file reading is done, hence Robert's method is more suited.
0
 

Author Comment

by:aztec
Comment Utility
Mullett - while we are waiting for perhaps Robert to reply, do you know how I could modify your proposed solution so the number of threads could be changed "on-the-fly" (ie. during run-time) ?

To have this ability would be extremely useful....

Thanks
   Shawn
0
 
LVL 11

Expert Comment

by:robert_marquardt
Comment Utility
Performance gains are only needed if the solution does not work good enough :-)
That is if you have MANY addresses or if you have to run the program more than once.

The performance could be measured in GetNextAddress. Get the time when the request happens and calculate how many requests per second or minute (depends on how fast it really is).
Since the thread objects are not stored anywhere simply extract a CreateThread function from CheckAddressFile so you can start another thread.
0
 

Author Comment

by:aztec
Comment Utility
Ran into one little snag while I am coding all this - to do the main validation of the email address, I call a pre-made ActiveX component which I have dropped onto my main form:

isOK:= Connection1.Validate(fAddress);

so now that this line of code is in the TCheckThread.CheckAddress function, it generates a compile error "Undeclared Identifier: Connection1".

How to rectify this?

Thanks!
   Shawn
0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
this is where threading becomes fun !

try this :

add a property to the checkthread ie
property MyThing : TMyActiveXthing read fMyThing write fMyThing;

create the thread suspended
set the MyThing property equal to Connection1
resume the thread

in the threads validation, call fMyThing.Validate(Address)

this may cause all sorts of errors, in which case you should try having each check thread dynamically create it's own TMyActiveXThing for it's own use. Again it will depend on how thread-safe the activeX component is
0
 

Author Comment

by:aztec
Comment Utility
ummmm, how/where do you add a property?
0
 

Author Comment

by:aztec
Comment Utility
Heres what I did to the TCheckThread class:

TCheckThread = class(TThread)
private
  fAddress : string;
  fLineNumber : integer;
  FValidAddress: boolean;
  conflevel: integer;
  conflevel_str, error_str : string;
  function CheckAddress : boolean;
  procedure GetNextAddress;
  procedure AddressResult;
public
  fMyThing : TConnection;
  procedure Execute ;override;
  property MyThing : TConnection read fMyThing write fMyThing;
end;


I try to do this as you suggest:

TCheckThread.MyThing:= Connection1;

but I get a compile error:
"Method Identifier Expected"
0
 

Author Comment

by:aztec
Comment Utility
Hi Mullett - you still with me? ;-)

Whta if I just remove the drop-in TConnection component from my main form, and just type in that declaration within the TCheckThread Class under private (or public) declarations, like this:

Connection1 : TConnection;

...wouldn't that work? I only call Connection1 from with the thread procedures, not from any TForm1 procedures.

Thanks
   Shawn
0
 

Author Comment

by:aztec
Comment Utility
I made the changes I suggested and it compiled right! But when I run it, I can this error:

---------------------------
Debugger Exception Notification
---------------------------
Project vmate.exe raised exception class EInOutError with message 'I/O error 104'. Process stopped. Use Step or Run to continue.
---------------------------
OK   Help  
---------------------------


on the "if not eof(valid_file) then" line (this is the file that I am reading my addresses from that I want to verify):

procedure TForm1.GetNextAddress(var Address : string ; var LineNumber : integer; var timeoutval : integer);
label
  readagain;
begin
  timeoutval:=SpinEdit1.Value;

  // read next address from file

readagain:
  if not eof(valid_file) then
  begin
    readln(valid_file, hinrec);
    inrec:=lowercase(hinrec);

    bytes_processed:=bytes_processed+length(inrec)+2;
    hperc:=calc_status(totsize_valfile, bytes_processed);
    if hperc >= prevhperc+1 then
    begin
      prevhperc:=hperc;
      gauge1.progress:=hperc;
      Form1.refresh;
      Application.ProcessMessages;
    end;

     etc. etc......


Hope you can help! You've been a great help to me so far!

Thanks
   Shawn
0
 

Author Comment

by:aztec
Comment Utility
OK, I think I found the cause of the 104 I/O error - it's because my input file (valid_file) was being closed up before my threads had a chance to read from it. Here's my code for when the user clicks the GO button to get things started:


procedure TForm1.BitBtn2Click(Sender: TObject);
label finish1;
begin


{* do all prep work right up to verifying the address...*}


  BitBtn1.enabled:=false;
  BitBtn2.enabled:=false;

  if fileexists(validfile_str) = false then
  begin
    MessageDlg('Input file not found : '+validfile_str, mtError,
      [mbOk], 0);
    goto finish1;
  end;

  elapsedtime := now;
  gauge1.progress:=0;

  bytes_processed:=0;
  hperc:=0;
  prevhperc:=0;
  goodcount:=0;
  badcount:=0;

  assignfile(valid_file, homedir+'infile.tmp');
  reset(valid_file);

  assignfile(rptfile, homedir+'summary.rpt');
  rewrite(rptfile);

  assignfile(good_file, homedir+'goodones.txt');
  rewrite(good_file);

  assignfile(bad_file, homedir+'badones.txt');
  rewrite(bad_file);


  panel2.caption:=' Processing...';
  panel2.update;



  {* get threads going here * }

  AddressCounter := 0;
  CheckAddressfile;


  write(rptfile, '               Summary Report ');
  writeln(rptfile, '     ' + DateTimeToStr(Now));
  writeln(rptfile, '     ');
  writeln(rptfile, '     ');
  writeln(rptfile, 'Input File            : ', validfile_str);
  writeln(rptfile, '     ');
  writeln(rptfile, '     ');
  writeln(rptfile, '     ');
  writeln(rptfile, 'Total adds in file : ', badcount+goodcount:12);
  writeln(rptfile, '     ');
  writeln(rptfile, 'Total good ones : ', goodcount:12);
  writeln(rptfile, 'Total bad ones  : ', badcount:12);
  writeln(rptfile, '     ');
  writeln(rptfile, '     ');
  writeln(rptfile, '     ');
  ElapsedTime := Now - ElapsedTime;
  writeln(rptfile, 'ELAPSED TIME: ', (ElapsedTime*86400):0:2, ' secs');  


  system.Close(valid_file);
  system.close(good_file);
  system.close(bad_file);
  system.close(rptfile);
  sysutils.deletefile(homedir+'infile.tmp');

  panel2.Caption:=' Finished!';
  gauge1.progress:=0;

finish1:
  BitBtn1.enabled:=true;
  BitBtn2.enabled:=true;
  Form1.Update;

end; {* BitBtn2Click *}

---------------------

When I step thru this in debug, it will launch the 30 threads in procedure CheckAddressfile, then after it launches them, it continues right on with executing this code:

  write(rptfile, '               Summary Report ');
  writeln(rptfile, '     ' + DateTimeToStr(Now));
  writeln(rptfile, '     ');

...don't I need some kind of statement after the 'CheckAddressFile' call, and before the 'write(rptfile,' stuff, in order to wait there for a signal from the threads that they're all finished? Otherwise, it's just gonna plow right straight through.

Thanks
   Shawn

0
 

Author Comment

by:aztec
Comment Utility
Hi, is anyone there? I've nearly got it - I just need to know how to tell my main program how to sit and wait for all the threads to complete, and THEN proceed...right now my main program is just racing right thru to completion!

(Also, is there a way to go into de-bug mode in the a thread...?)

Thanks
   Shawn
0
 

Author Comment

by:aztec
Comment Utility
Hello?

....anybody?
0
 
LVL 7

Expert Comment

by:Cynna
Comment Utility
aztec,

> Hi, is anyone there?

Well...yes, there is - but you are asking too much involvement for too little points.
It's a basic market psychology that EE is based on - the more points you offer, the more
time experts are willing to invest in your question. For 50 points you can't really expect
people to spend too much time on your question, especially if it has prolonged, like this
one. Exceptions to the rule are, of course, questions that expert has personal interest in.
Good example of this is mullet_attack, who, IMHO, more than deserves your points.

Please don't get me wrong, I'm just guessing on why you didn't get any responses.
If you have any more questions, you should close this thread and ask another question,
referencing to this post.

OK, enough preaching, on to your question:

> I just need to know how to tell my main program how to sit and wait for all the threads to complete, and THEN proceed

Use global thread counter. Increase it on each thread Execute, and decrease it immediately
before thread destruction. Always enclose this in try..finally block to make sure your main
thread doesn't hang. In the main thread create your threads, and loop until counter is 0 again.
Don't forget to use critical sections to protect each global counter increase/decrease.

Simple example of what I mean...

1.Use common unit globalvars.pas in all your threads, and put global var ThreadCount and TCProtect critical section protection object in it:

unit globalvars;

uses Syncobjs, ...
...

var  ThreadCount: Integer;
       TCProtect: TCriticalSection;
   
     ...
     initialization
           TCProtect:= TcriticalSection.Create;
             
     finalization
           TCProtect.Free;
     ...






2. In Execute method of your threads:

 procedure TCheckThread.Execute;
 begin
   try
     TCProtect.Enter;
          Inc(ThreadCount);
     TCProtect.Leave;

     //...rest of your code...

   finally
     TCProtect.Enter;
          Dec(ThreadCount);
     TCProtect.Leave;

   end;
 end;

3. After creation of threads main thread should sit and wait until all threads are finished:

......

 {* get threads going here * }

 AddressCounter := 0;
 ThreadCount:=0;
 CheckAddressfile;

 // Wait for completion:
 while ThreadCount>0 do begin
    Application.ProccessMessagess;
    Sleep(1);
 end;

 write(rptfile, '               Summary Report ');

......





> (Also, is there a way to go into de-bug mode in the a thread...?)

The best way, in my experience, is good old log file.
While we're at the debug subject, what happened to :) http://www.experts-exchange.com/delphi/Q_20320549.html
0
 

Author Comment

by:aztec
Comment Utility
Thanks for the input Cynna - of course I don't mind giving more points - 100, 200 , whatever is fair! I just want to get a solution that works.

I've tried your above suggestion - it compiles fine...but something is fundamentally wrong somewhere: When I run my prgram outside of the IDE, it just flies right through everything immediately. No errors arise, but nothing gets done.
   When I run the program in the IDE (but not in debug mode), I get this runtime error:

---------------------------
Debugger Exception Notification
---------------------------
Project vmate.exe raised exception class EInOutError with message 'I/O error 104'. Process stopped. Use Step or Run to continue.
---------------------------
OK   Help  
---------------------------


...on the line I indicate with the arrow:


procedure TForm1.GetNextAddress(var Address : string ; var LineNumber : integer; var timeoutval : integer);
label
  readagain;
begin
  timeoutval:=SpinEdit1.Value;

  // read next address from file

readagain:
----->  if not eof(valid_file) then
  begin
    readln(valid_file, hinrec);
    inrec:=lowercase(hinrec);
     etc. etc.....



...and sometimes I get this error:

---------------------------
Debugger Exception Notification
---------------------------
Project vmate.exe raised exception class EAccessViolation with message 'Access violation at address 00467486 in module 'vmate.exe'. Read of address 00000050'. Process stopped. Use Step or Run to continue.
---------------------------
OK   Help  
---------------------------


...in this line with the arrow:

function TConnection.GetDefaultInterface: IConnectionSync;
begin
---->  if FIntf = nil then
    Connect;
  Assert(FIntf <> nil, 'DefaultInterface is NULL. Component is not connected to Server. You must call ''Connect'' or ''ConnectTo'' before this operation');
  Result := FIntf;
end;


(this code is from the Validation component I am using in my app).


Doing some debugging, it appears that my main code is still not "waiting" for the threads to end, but is plowing right thru....it writes the final report (for which there is zero results) and completes the ButtonClick event (after which, my app *should* be complete and return the user to a blank screen awaiting the next execution). But instead it goes right back into procedure TForm1.GetNextAddress and fails on the line I pointed out above with the I/O 104 error.


...I am basically using Mullet's suggestions exactly. This is very complicated to do a seemingly simple thing! :-(


Thanks
   Shawn

P.S: Cynna, I've not forgotten about http://www.experts-exchange.com/delphi/Q_20320549.html. But this one just takes a higher priority for me at this time. I will return to that other one once I solve this!
0
 
LVL 2

Expert Comment

by:mullet_attack
Comment Utility
Aztec,

sorry for the delay, my ISP has a had a DNS fault for a couple of days, so I couldn't get on to EE.

Pls post the code where you create the threads.

I also suggest that you stop using labels and gotos in your code. There is probably a time to use them, but I have yet to find it. basically never, never use goto...
Now, no offense intended, but if you don't know how to code without the goto's, you need to learn that _before_ continuing on with thread stuff.

on the debugging issue, yes you can debug threads in the IDE, but it gets messy.
put this routine in a globally available unit :

procedure DebugStr(s : string);
var
  P : PChar;
begin
 getmem(P, length(s) + 1);
 try
   StrPCopy(P,S);
   OutputDebugString(P);
 finally
   freemem(P);
 end;    
end;


then, in your code use this :

DebugStr('this is my debug message');

When running in the IDE, use VIEW->DEBUG WINDOWS->event log to see the messages. To view them outside the IDE, goto (oops :-)) www.sysinternals.com and download the debug viewer - works great.

I usually encase my debug code in conditional compile directives, thus ;

{$IFDEF debugging}
  DebugStr('A debug message');
{$ENDIF}

so you can use:
{$DEFINE debugging} to turn it on, or
{$UNDEF debugging} to turn it off

that way you can release code with no extra debug crap in it, but can turn it back on in an instant.

Mark (Mullet_Attack)


0
 

Author Comment

by:aztec
Comment Utility
Thanks Mullet!

Regards
   Shawn
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Suggested Solutions

This article explains how to create forms/units independent of other forms/units object names in a delphi project. Have you ever created a form for user input in a Delphi project and then had the need to have that same form in a other Delphi proj…
Introduction I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need. This article will address some of the concepts when dealing with a multithreaded delphi database…
This video discusses moving either the default database or any database to a new volume.
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

7 Experts available now in Live!

Get 1:1 Help Now