Start Free Trial

asked on

Threading in Delphi

Hi
I am using TThread Class in Delphi and I have created my own class called TMyThread adding properties and methods. An instance of another object let's name it "MyObject" spawn a single thread of TMyThread creating every 5 minutes. Once the thread is done I copy some data in MyObject properties and then I free it within the TMyThread class itself. I am new to multithreading so I have some doubts I hope you can clarify it:

A). Let's consider I have 400 different "MyObject" which spawn 400 threads every x minutes where x is variable. How I can monitor if the system is suffering of intensive utilization so I can reguate the modalities of spawing threads. Should I set the max number of threads running based on the cpu current utilization ? Should I extimate periodcally this value within my program? Tmythread is class for polling remote devices via snmp so the time consuming depends on the type of snmp request which can be a huge routing table or a simple value.

B) I store the snmp result in TMyThread propeprty then I copy this value in MyObject before free the thread. Is this a right way generally to exchange data among objects? MyObject updade a memo in VCL main thread for each result. Here my question.
In case I have 400 or more threads running simultaneously ( I am considering the wrost case, a peak) what will be the best way to store the data if I don't want to freeze the user interface? Storing data in TThreadList? In xml external file updated within
the threads? Should I use a Synvhronizer class that I declare fo the purpose of storing information and only when done I update a visual control in the main thread?

C) All thread are destroyed when done. Should I poll these threads or pause them instead of re-creating them every time?

(*) I already red all topic of threading in internet, I simple searching for any advices how you
would solve it.

Thank in advance.

SNMP Result? What exactly are you doing creating 400 SNMP threads? You building a DOD Attack program?

I meant DOS attack program.

ASKER

A network monitoring tool. I wrote 400, but it could be 30 every 3 minutes. By the way I am thinking to add a random sleep() among them.

Thx

in my opinion i think you just wrote a tool to bring down a computer
400 threads ?
ugh
what are you monitoring ?

ASKER

A multiplatform enviroment server, router, windows workstation....
how would u develop such tools? Have you ever seen such solution ? I was wondering how to pool many devices...

400 thread is an example!!
How to develop a NMS for monitoring a network with 300 nodes: 50 server, 10 routers...and so on..

ASKER

I will check it.
Thx

ASKER

Sorry for this post.
It was not releated to this question...

ASKER

Nobody can help me? :-(

SOLUTION

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

SOLUTION

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ASKER

I've appreciated a lot last two comments which i am reviewing right now.
I will give you a feedback ASAP.

ASKER

Basing on your replies I had to keep the number of threads at reasonable level. How to find out an optimal value? ;-)

Scenario: Let's assume that the network is composed of 300 nodes (server, routers, workstation, snmp general devices).

For instance my first goal is:

1. Ping the nodes for checking the network availability
2. Check if snmp is UP
3. If (1) and (2) are ok then I send an snmp getrequest (es. temperature value, tcp error ...)

In my tool Node is rappresented by a TNodeClass. An object of TNodeClass type has several properties, the most
important are IP Address and TTimer component. Every x minute (it's programmable by the user) OnTimer Event is fired for the given node and a procedure executes:

procedure TNodeBase.CheckNode(Sender: TObject);
begin
NewThread := TPrimeThrd.Create(True);
NewThread.FreeOnTerminate := True;
NewThread.OnTerminate := HandleTerminate_test;
try
NewThread.ip := IpAddress;
NewThread.Resume;
except
on E: Exception do
NewThread.Free;
end;

In Excute of TPrimeThrd Class:

procedure TPrimeThrd.Execute;
var
Ping : TPing;
snmp : TSNMP;
begin
Ping := TPing.Create(nil);
snmp := TSNMP.Create(nil);
with Ping do
begin
//
OnResponse := PingResponse;
OnError := Error;
end;

with snmp do
begin
RemoteHost := FIpAddress;
SNMPVersion := snmpverV2c;
OnResponse := Response;
Obj[0] := '1.3.6.1.2.1.2.2.1.8.1';
end;

try
begin
Ping.PingHost(FIpAddress);
snmp.SendGetRequest();
end;

finally
Ping.Free;
snmp.Free;
end;

end;

Here is an example so I've set a single item for the colletion Obj[]. Imagine that for a Node I would poll
more object values it depends what my customer wants to monitor (I would store these oid in a TStringList property for
the node). OnResponse events is fired for each snmp reply.( I assume for instance that only one value
is returned from the remote device so I do not care of overwriting problems..)

procedure TPrimeThrd.Response(Sender: TObject; RequestId,
SNMPVersion: Integer; const Community, User: string; SecurityLevel: Integer;
const SourceAddress: string; SourcePort, ErrorIndex, ErrorStatus: Integer;
const ErrorDescription: string);
begin
// Here I get my value and store it in a TPrimeThrd field.
end;

THe same for ping component:

procedure TPrimeThrd.PingResponse(Sender: TObject; RequestId: Integer;
const ResponseSource, ResponseStatus: string; ResponseTime: Integer);
begin
Status := ResponseStatus;
end;

What I would have done:

Set a max number of a running theads (es. 30), keep track of the current
number threads and run another only if NumCurrentTheads < 30.
This the firts think I thought.

Geert suggests to create a joblist of 10 jobs. Each job has several tasks..so these
tasks in my case I suppose are ping and getsnmp simple requests?. I will
be running 10 threads simultanesly , and for each job I need to have
10 different ping and snmp objects? So what if Node are 200? Could you
please post some code sample?

>> You need to consider the data the threads are bringing back needs to be processed too.

Here I am confused. You mean in case to avoid conflict when the loop
restars? If I had 20 nodes and I run 20 separate threads I would not have
such problem right? This happen only if the number of threads(jobs) is less
than the number of nodes?

DelphOwl

A:The reasonable number of threads for your application depends of how hard each thread utilize the cpu.
How I can check it?

B: In case your threads produces to much data to visialize in your GUI, you should consider a
producer/consumer approach (which could be a class based on TThreadList...
Can you post some code exmaple with TThreadList? :-(

And what if I define a TStringList property for each node where
store snmp value and ping result along with data and time? As per the code I posted above
I might copy the snmp value from the thread to Node's StringList....

C: I dont have any experience in what the penalty for spawning to many threads/sec. actually is, many someone else can comment on that..

I think this is an interesting subject thant could be helpful also in other situation.

Thx

ASKER

I have increased the point to 500.

ASKER

No reply for me? :-(

SOLUTION

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Or maybe you could just use the OnTerminate event and read the thread result directly from there...
What TPing and TSNMP components are you using?

ASKER

Hi DelphiOwl and Thanks a lot for your reply.
I printed your answer and i will give a look it today.
The components are IP Works, VCL version for Delphi 2006.
Thx

ASKER

Hi

Ping.PingHost(FIpAddress), and
snmp.SendGetRequest();

should be these methods at place of your comments in Execute or not?

...and at very end of Excute I call :
synchronize(DoCallback); ?

In the thread, events are fired when packets are received in :

procedure TPrimeThrd.Response(Sender: TObject; RequestId,
SNMPVersion: Integer; const Community, User: string; SecurityLevel: Integer;
const SourceAddress: string; SourcePort, ErrorIndex, ErrorStatus: Integer;
const ErrorDescription: string);
begin
// Here I get my value and store it in a TPrimeThrd field.
end;

THe same for ping component:

procedure TPrimeThrd.PingResponse(Sender: TObject; RequestId: Integer;
const ResponseSource, ResponseStatus: string; ResponseTime: Integer);
begin
Status := ResponseStatus;
end;

Componets works like that : I sent an snmp request snmp.SendGetRequest(); and TPrimeThrd.Response
is fired when packet is received. So I don't undestand the needed of TEvent :-(((((

Callback procedure is a method of TNodeBase which I call only when
jobs is done so information are avaiable for the TnodeBase?

Scenario

TBaseNode1 -> (after 2 minutes) Thread 1 - get data for TbaseNode1 -> Destroy Thred1
TBaseNode1 -> (after 3 minute) Thread 2 - get data fot TbaseNode2 -> Destroy thread2
(and so on...)

TBaseNode is inhereted from TComponent.

How i can undestand when Suspend the thread instead of destroing it?

Thx

ASKER

With regards to SNMP here are you see events, properties and methods:

http://www.nsoftware.com/products/component/snmpmgr.aspx

Data specified in ObjOid are then stored in:

ObjValue
Array of object values.

ASKER CERTIFIED SOLUTION

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ASKER

Thx. I have missed one important thing. I don't know but I was assuming that after sending pinghost and sendrequest the thread would have wait for response , and exit from Excute method only when event were fired. Got it. By the way you are very helpful to me and I thank you for your suggestion.

As per your last advice you mean using TPrimeThrd objects in the main thread? Will this frooze my GUI while pinging nodes?

Do you think that create just a few threads (10/20) and use them for doing more jobs would be better than the previous approach where for each TbaseNode i create a thread ?.

I am interested in multithreading so I kindly ask you if you could complete the code above with spleep in excute along with code on how wakeup the thread from Response and PnigResponse?

The callback procedure you pass as parameter in Thread costructor it's defined
in TBaseNode?

Can you suggest me a good book (I have already download lot of documentation from the web) for dephi multithreading programming?

Thanks a lot for you patience

SOLUTION

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

I have made this little example for you, using TIdIcmpClient to do the actual ping'ing. One thing I realized is that TIdIcmpClient blocks when the Ping method is called, and does not return until a response or a timeout occurs. If the IP Works TPing component also blocks, you will not have the need for making your thread sleep with TEvent (you have allready figured this out yourself).
In my example I ping a whole c-subnet by making all 255 threads at one time. It seems to be no problem for the cpu to handle, however, the Delphi debugger spends a lot of time on managing new threads - so, run the application from outside Delphi to evaluate the true performance.

unit Unit1;
 
interface
 
uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, IdBaseComponent, IdComponent, IdRawBase, IdRawClient, IdIcmpClient,
  StdCtrls;
 
type
  TForm1 = class(TForm)
    IdIcmpClient1: TIdIcmpClient;
    Button1: TButton;
    ListBox1: TListBox;
    procedure Button1Click(Sender: TObject);
  private
    FSeqID : word;
    procedure ThreadTerminate ( Sender : TObject );
  public
  end;
 
  TTask = class ( TThread )
  private
    FIdIcmpClient: TIdIcmpClient;
    FPingRecv : integer;
    FIP : string;
    FError: string;
    FSeqID : word;
    procedure IdIcmpClientReply(ASender: TComponent;
                                const AReplyStatus: TReplyStatus);
  protected
    procedure Execute; override;
  public
    constructor Create ( IP : string; seqID : word );
    destructor Destroy; override;
    property PingRecv : integer read FPingRecv;
    property IP : string read FIP;
    property Error : string read FError;
  end;
 
var
  Form1: TForm1;
 
implementation
 
{$R *.dfm}
 
procedure TForm1.Button1Click(Sender: TObject);
var
  t : TTask;
  n : integer;
begin
  FSeqID := 1;
  for n := 1 to 255 do
  begin
    t := TTask.Create ( '192.168.10.'+inttostr(n), FSeqID );
    t.OnTerminate := ThreadTerminate;
    t.Resume;
    inc(FSeqID);
  end;
end;
 
procedure TForm1.ThreadTerminate(Sender: TObject);
begin
  if TTask(Sender).Error = '' then
  begin
    if TTask(Sender).PingRecv <> 0 then
      ListBox1.Items.Add( 'Reply from ' + TTask(Sender).IP + ' '+inttostr(TTask(Sender).PingRecv)+' bytes' )
    else
      ListBox1.Items.Add( 'No reply from '+TTask(Sender).IP );
  end else
  begin
    ListBox1.Items.Add( 'Error on '+TTask(Sender).IP+':´'+TTask(Sender).Error );
  end;
end;
 
{ TTask }
 
constructor TTask.Create(IP: string; seqID : word );
begin
  inherited Create ( true );
  FreeOnTerminate := true;
  FIP := IP;
  FError := '';
  FSeqID := seqID;
end;
 
destructor TTask.Destroy;
begin
  FIdIcmpClient.Free;
  inherited;
end;
 
procedure TTask.Execute;
begin
  try
    FIdIcmpClient := TIdIcmpClient.Create ( nil );
    FIdIcmpClient.Host := FIP;
    FIdIcmpClient.ReceiveTimeout := 5000;
    FIdIcmpClient.OnReply := IdIcmpClientReply;
    FIdIcmpClient.Ping ( '', FSeqID );
  except
    on e: exception do
    begin
      FError := e.Message;
    end;
  end;
end;
 
procedure TTask.IdIcmpClientReply(ASender: TComponent;
  const AReplyStatus: TReplyStatus);
begin
  if AReplyStatus.SequenceId = FSeqID then
  begin
    FPingRecv := AReplyStatus.BytesReceived;
  end else
  begin
    FPingRecv := 0;
  end;
end;
 
end.

Open in new window

ASKER

Hi Guys.
Thanks a lot, give me a few days to study your code and I will back ASAP to u.
I am getting fond of multithreading in delphi, very cool. -))

<<...If the IP Works TPing component also blocks>>

Yes, Actually in my example with IP Works (using TPrimeThread) I get all response in main vcl thread in a MEMO components , I have tested it with 10 local snmp devices (using snmp simulator ) and with remote website like google.com...
The snmp query seems to work nice even with snmp bulk request on a large
IpRouteTable. But I had some doubts that's why i've started to post here.

If so I need only to find a good solution to exchange data among TbaseNode and threads (and limit the number of running thread to a max value).

I will also open a ticket support with the HQ asking for futher information on
this issue.

See ya.

ASKER

Hi DelphiOwl. Could you please send the project as attached file for delph 2006?
Please Geert could you do the same with yours?

Thanks a lot.

i don't have D2006, i uninstalled it...
i still have to install D2007 some time in the future, currently only D7 and D2009.

Its allready there in PingThread.txt... rename it to PingThread.rar and unpack it ;-)

ASKER

IP Works support:

As long as you set the Timeout property to a positive value the components
will behave synchronously. If the Timeout is set to 0, the operations will
be asynchronous (so you would need to wait).

ASKER

Below the description of the Timeout from help file:

Syntax
property Timeout: Integer;
Default Value
60

Remarks
If the Timeout property is set to 0, all operations return immediately, potentially failing with an error if they can't be completed immediately.

If Timeout is set to a positive value, the component will wait for the operation to complete before returning control.

The component will use DoEvents to enter an efficient wait loop during any potential waiting period, making sure that all system events are processed immediately as they arrive. This ensures that the host application does not "freeze" and remains responsive.

If Timeout expires, and the operation is not yet complete, the component raises an exception.

Please note that by default, all timeouts are inactivity timeouts, i.e. the timeout period is extended by Timeout seconds when any amount of data is successfully sent or received.

So setting the Timeout>0 will block the component, by the way I don't undestand if this implies that the thread fall asleep until responses from Ping and SNMP arrive.

ASKER

if Timeout > 0 the component will block.
The component won't be sleeping though, it will be internally doing events,
so you shouldn't see any increased CPU usage there, and events on the thread
will continue to work.

Guys, does it solve the problem?

DelphiOwl:
<<If you do not make your thread sleep in the execute method, the execute method will use 100% cpu power if you keep it running in a loop of some kind.>>

Thx

Well, it seems it all depends on how the "IP Works" components works. I dont have any experience with them, so I am not to very much help there.

ASKER

Thx.
Nobody had experiences with it?

Thx

i don't, just with indy and ye olde netmaster components (D5)

have you read the recommendations on grading a question in the help ?
when you don't give a comment, then it is recommended you give an A grade
if there is a B grade then something is obviously missing
but since no comment, we don't know what ...

ASKER

Hi. Sorry, I to turn in A?
Thx

i dunno, if you still have some issues that need clearing out, post them
if you find everything correct than the idea would be an A grade.
this seems to be a tendancy which everybody new to this site seems to be having.
i admit, the help is a lot to read and halfway through i gave up as well

this is a very advanced site on threading:
http://otl.17slon.com/