systan
asked on
Applying ThreadPool to my code
hi
I started again; Since I could not figure it out how to mix the threadPool unit to my app, I would like someone to assist me to mix the threadPool code to my code, put the threadPool to any location of my code.
https://www.experts-exchange.com/questions/24623451/TThreadPool-AV-multi-thread-beginner-here.html
Actually the important function to be threaded is this line;
ret := called_from_dll_IDENTIFY( PChar(Field.AsString),
Thanks
Please mix the threadPool to my code.
I would like to increase the points if success to my test.
I started again; Since I could not figure it out how to mix the threadPool unit to my app, I would like someone to assist me to mix the threadPool code to my code, put the threadPool to any location of my code.
https://www.experts-exchange.com/questions/24623451/TThreadPool-AV-multi-thread-beginner-here.html
Actually the important function to be threaded is this line;
ret := called_from_dll_IDENTIFY( PChar(Field.AsString),
Thanks
Please mix the threadPool to my code.
I would like to increase the points if success to my test.
///
This is in my Main_Form unit, that calls [identifying function] from unit [uUtil]
///
uses uUtil, ...
procedure TForm1.BUTTON_iDENtifyClick(Sender: TObject);
var
ret, score: Integer;
Ticks : Cardinal;
begin
score := 0;
ticks := GetTickCount();
ret := Identifying (score);
if ret > 0 then
begin
ShowMessage(' BlobField Identified_with_ID is '+ IntToStr(ret) + ' and the Score of the context is ' + IntToStr(score) );
end
else if ret = 0 then
ShowMessage('not_iDentified')
else
ShowMessage('Error');
ticks := GetTickCount - ticks;
ShowMessage('Number of Seconds found:' + IntToStr(ticks) );
end;
==============
///
This is the code of unit uDBClass that it to be called by [Main_Form] unit and unit [uUtil]
///
unit uDBClass;
interface
uses ....
TDBClass = class
private
// the connection object
mycon: TSQLite3Connection;
// SQL Transaction
transact: TSQLTransaction;
public
// a data set to maintain all records of database
ds0: TSQLQuery;
function openDB(): boolean;
procedure closeDB();
end;
implementation
...
...
// Open connection
function TDBClass.openDB(): boolean;
var i,i2,size:integer;
tmp:string;
begin
try
ds0 := TSQLQuery.Create(nil);
mycon := TSQLite3Connection.Create(nil);
transact := TSQLTransaction.Create(nil);
mycon.Directory := '';
mycon.DatabaseName := 'dbsqlite.db';
transact.DataBase := mycon;
transact.Action := caCommit;
transact.Active := True;
mycon.Transaction := transact;
mycon.Open;
ds0.DataBase := mycon;
ds0.DisableControls;
ds0.SQL.Text := 'select id, template from table1';
ds0.CreateDataSet;
ds0.Prepare;
ds0.Open;
ds0.ActiveBuffer;
openDB := true;
except
openDB := false;
end;
end;
// Close conection
procedure TDBClass.closeDB();
begin
mycon.Close;
mycon.Free;
mycon := nil;
transact.Free;
transact := nil;
end;
end.
====================
///
This code is unit [uUtil] that calls database unit [uDBClass]
///
unit uUtil;
interface
uses udBclass,...
Var
// Database class.
dB: TDBClass;
implementation
function first_called_from_main_form_during_load_for_opening_db
begin
...
// Opening database
dB := TDBClass.Create();
if not dB.openDB() then
begin
...
Exit;
end;
...
end;
//NEED to apply FIFO (first in first out) like ciuly did in his file search multithreading
function identified(var dsRec: TSQLQuery; out score:integer): integer;
var ret:integer;
FIeld: TField;
begin
Field := dsRec.FieldByName('template');
dsRec.DisableControls;
dsRec.First;
while not dsRec.EOF do
begin
ret := called_from_dll_IDENTIFY( PChar(Field.AsString), score, 0);
if (ret = 1) then
begin
Identified := dsRec.FieldByName('id').AsInteger;
exit;
end
else if (ret < 0) then
begin
Identified := ret;
exit;
end;
dsRec.Next;
end;
if (ret = 0) then
begin
Identified := 0;
end;
end;
//This is called in unit Main_Form [uMain]
function Identifying (var score: Integer): Integer;
Begin
Identifying := identified(db.ds0, score);
end;
end.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
I see I did a mistake copying the URL to get OmniThread
http://omnithreadlibrary.googlecode.com/files/OmniThreadLibrary-1.05a.zip
please let me know if you are prepared to work step by step on a fast & reliable solution with OmniThread, and stop trying new things every day ;-)
http://omnithreadlibrary.googlecode.com/files/OmniThreadLibrary-1.05a.zip
please let me know if you are prepared to work step by step on a fast & reliable solution with OmniThread, and stop trying new things every day ;-)
lol, epasquier, trying out that lib too ?
it's amazing, but i admit, not easy to grasp all the concepts :)
it's amazing, but i admit, not easy to grasp all the concepts :)
yep, the documentation is a bit lacking and the few there is on the website is out of date. You have to dif in project samples to get your bearing, but when you find how to do this or that, it works on first try !
Simple simple simple... (if you are fluent with threads vocabulary and concepts)
I played a bit with Core affinity, to see that heavy calculations on a i7 (4Cores+ HT = 8 logical cores) is optimum with 8 Threads, but only from a fraction compared to 4 threads, with a little loss between 5-7 threads due to Hyperthreading. That was to be expected, I was even surprised 4 Threads was not just plain better than 8. Still I would recommend using only the nb of physical cores when doing heavy calculations. Which means that a smart application using OmniThread would have to detect HT (or that number of Physical cores), and explicitly set the process affinity to only those cores (the odd ones, so 0,2,4,6 on my system)
Systan, just a question, how many cores does your celeron have ?
Simple simple simple... (if you are fluent with threads vocabulary and concepts)
I played a bit with Core affinity, to see that heavy calculations on a i7 (4Cores+ HT = 8 logical cores) is optimum with 8 Threads, but only from a fraction compared to 4 threads, with a little loss between 5-7 threads due to Hyperthreading. That was to be expected, I was even surprised 4 Threads was not just plain better than 8. Still I would recommend using only the nb of physical cores when doing heavy calculations. Which means that a smart application using OmniThread would have to detect HT (or that number of Physical cores), and explicitly set the process affinity to only those cores (the odd ones, so 0,2,4,6 on my system)
Systan, just a question, how many cores does your celeron have ?
he's got 1 core on that ... :)
already noted that threading isn't really the way to go with this topic
>fingerprint recognition as overal goal
already noted that threading isn't really the way to go with this topic
>fingerprint recognition as overal goal
sure, it's not gonna help with multithreading one 1 core. I think that by default OmniThread would limit the pool of worker threads that executes simultaneously to the number of logical cores. Or maybe not, it's just an impression I had reading some samples. I haven't try to launch more than 8 worker threads
but I understand the willingness to develop something that CAN be faster, provided the user has a better CPU than the developper... And if the application is done correctly, the potential benefit on multicore does not have to translate in an actual loss when used with only 1 core.
but I understand the willingness to develop something that CAN be faster, provided the user has a better CPU than the developper... And if the application is done correctly, the potential benefit on multicore does not have to translate in an actual loss when used with only 1 core.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
>>stop trying new things every day
Yes, I know, I spend this test for almost 3 years.
Got this in vb.net and c#, but I could not figure it out on delphi, I find delphi very hard, but I am also enjoying the hard time.
It's only my personal test, nothing matters, I also make some test for antivirus apps, and its 80% done, making an antivirus that would globally shield/protect the windows system with the minimum update of virus signatures, lol. But it is possible as I see it.
Anyway I test the code in quad core processor with 3gig memory free(friends cpu);
results;
5 to 6 seconds, its not good, I though it will be 1 to 2 seconds
>>how many cores does your celeron have ?
I don't know, but it is intel celeron (mobile) 380 1.6hz
>>overal goal
yes, but no clients involved, its only my personal interest.
>>but I understand the willingness to develop something that CAN be faster
Yes, as I said if it is possible
Do you remember epasquier, that you told me to stop in tmemorystream, I didn't, I dance with it,
So! I got it runnin it on tmemoryStream, but it only gain 4 seconds.
But the fact that I didn't stop, so I got it.
BUT again, not satisfied with the result, and it was tested on quad core 3gig free mem with 5 seconds.
Ok;
regarding to a new architecture you proposed, I though ThreadPool can help with a few workers.
Yes, I know, I spend this test for almost 3 years.
Got this in vb.net and c#, but I could not figure it out on delphi, I find delphi very hard, but I am also enjoying the hard time.
It's only my personal test, nothing matters, I also make some test for antivirus apps, and its 80% done, making an antivirus that would globally shield/protect the windows system with the minimum update of virus signatures, lol. But it is possible as I see it.
Anyway I test the code in quad core processor with 3gig memory free(friends cpu);
results;
5 to 6 seconds, its not good, I though it will be 1 to 2 seconds
>>how many cores does your celeron have ?
I don't know, but it is intel celeron (mobile) 380 1.6hz
>>overal goal
yes, but no clients involved, its only my personal interest.
>>but I understand the willingness to develop something that CAN be faster
Yes, as I said if it is possible
Do you remember epasquier, that you told me to stop in tmemorystream, I didn't, I dance with it,
So! I got it runnin it on tmemoryStream, but it only gain 4 seconds.
But the fact that I didn't stop, so I got it.
BUT again, not satisfied with the result, and it was tested on quad core 3gig free mem with 5 seconds.
Ok;
regarding to a new architecture you proposed, I though ThreadPool can help with a few workers.
Sure, a pool of worker threads is part of the architecture I told you about. But I don't know of a standard mechanism in Delphi for that, maybe in one of those newest versions ?
Which is why I proposed to use OmniThread library for that.
So my question is again : Are you OK with it , and start with the "Producer" = the thread-safe object that can give Workers job to do ( template data ) ?
Which is why I proposed to use OmniThread library for that.
So my question is again : Are you OK with it , and start with the "Producer" = the thread-safe object that can give Workers job to do ( template data ) ?
ASKER
hi Geert;
The things you've mentioned was applied to the identify function of the dll and uses the pthread lock and unlock.
First extract the image, copy bytes to memory, as array of bytes.
Second the identify function, that generates multithreading of the minutia.
Ok;
I think its time to close this post, since I've fly my code to the other hand, I just hope it gets better.(if possible to dance why not fly)
The things you've mentioned was applied to the identify function of the dll and uses the pthread lock and unlock.
First extract the image, copy bytes to memory, as array of bytes.
Second the identify function, that generates multithreading of the minutia.
Ok;
I think its time to close this post, since I've fly my code to the other hand, I just hope it gets better.(if possible to dance why not fly)
ASKER
OmniThread? Its Ok. As long it performs a very good gain of seconds.
But, if there is a solution without using OmniThread?, that only gain for 1 second?, much better.
Thanks
Oh, ThievingSix?, the Expert who answered about threadPool, Thanks also.
But, if there is a solution without using OmniThread?, that only gain for 1 second?, much better.
Thanks
Oh, ThievingSix?, the Expert who answered about threadPool, Thanks also.
ASKER
Oh about the OmniThreadLiB, I could not do it, I'm to newbie to do that kind of stuff, impossible, my mind is old, tired and abused. I better wait for a month or less.
Thanks a lot.
Thanks a lot.
you'll see it's easy TO USE once you have done it once. And besides, what are we there for if not helping you to sort it out ?
Of course trying to understand HOW it works is just another story, as Geert said. But that is so with every thread mechanism
From what I've seen, it works as advertised : easily, without visible overhead due to task synchronization, with loads of features yet to explore. But we won't need much exotic features for your problem, so you should get it quickly
Of course trying to understand HOW it works is just another story, as Geert said. But that is so with every thread mechanism
From what I've seen, it works as advertised : easily, without visible overhead due to task synchronization, with loads of features yet to explore. But we won't need much exotic features for your problem, so you should get it quickly
ASKER
Ok;
Here's my current test, I divide my app into 2,
a.exe has 1 to 50000 records
b.exe has 500001 to 100001 records
And you know what?, it hits 9 seconds.
The a.exe has 9 seconds, last record found. The b.exe has 11.500 seconds (not found)
I'll try to get back asap.