Applying ThreadPool to my code


I started again; Since I could not figure it out how to mix the threadPool unit to my app,  I would like someone to assist me to mix the threadPool code to my code,   put the threadPool to any location of my code.

Actually the important function to be threaded is this line;
ret := called_from_dll_IDENTIFY( PChar(Field.AsString),

Please mix the threadPool to my code.
I would like to increase the points if success to my test.
This is in my Main_Form unit, that calls [identifying function] from unit [uUtil]
uses uUtil, ...
procedure TForm1.BUTTON_iDENtifyClick(Sender: TObject);
  ret, score: Integer;
  Ticks : Cardinal;
  score := 0;
  ticks := GetTickCount();

  ret :=  Identifying (score);

  if ret > 0 then
    ShowMessage(' BlobField Identified_with_ID is  '+ IntToStr(ret) + '  and the Score of the context is  ' + IntToStr(score) );
  else if ret = 0 then

    ticks := GetTickCount - ticks;
    ShowMessage('Number of Seconds found:' + IntToStr(ticks) );


This is the code of unit uDBClass that it to be called by [Main_Form] unit and unit [uUtil]
unit uDBClass;


uses ....

  TDBClass = class
    // the connection object
    mycon: TSQLite3Connection;
    // SQL Transaction
    transact: TSQLTransaction;
    // a data set to maintain all records of database
    ds0: TSQLQuery;

    function openDB(): boolean;
    procedure closeDB();

// Open connection
function TDBClass.openDB(): boolean;
var i,i2,size:integer;
        ds0 := TSQLQuery.Create(nil);

        mycon := TSQLite3Connection.Create(nil);
        transact := TSQLTransaction.Create(nil);

        mycon.Directory := '';
        mycon.DatabaseName := 'dbsqlite.db';

        transact.DataBase := mycon;
        transact.Action := caCommit;
        transact.Active := True;

        mycon.Transaction := transact;

        ds0.DataBase := mycon;

        ds0.SQL.Text := 'select id, template from table1';

        openDB := true;
        openDB := false;

// Close conection
procedure TDBClass.closeDB();
  mycon := nil;
  transact := nil;



This code is unit [uUtil] that calls database unit [uDBClass]
unit uUtil;


uses udBclass,...

  // Database class.
  dB: TDBClass;


function first_called_from_main_form_during_load_for_opening_db
// Opening database
dB := TDBClass.Create();
if not dB.openDB() then

//NEED to apply FIFO (first in first out) like ciuly did in his file search multithreading
function identified(var dsRec: TSQLQuery; out score:integer): integer;
  var ret:integer;
  FIeld: TField;
  Field := dsRec.FieldByName('template');
while not dsRec.EOF do
ret := called_from_dll_IDENTIFY( PChar(Field.AsString), score, 0);
if (ret = 1) then
 Identified := dsRec.FieldByName('id').AsInteger;
 else if (ret < 0) then
 Identified := ret;
 if (ret = 0) then
 Identified := 0;

//This is called in unit Main_Form [uMain]
function Identifying (var score: Integer): Integer;
Identifying := identified(db.ds0,  score);


Open in new window

LVL 14
Who is Participating?
Emmanuel PASQUIERConnect With a Mentor Freelance Project ManagerCommented:
What about the new architecture I proposed in a former mail ? I have been doing some tests with OmniThread this week-end, that lib is outstanding for thread, so I might have something for you, and with a step by step method I'm sure we can finalize something and get done with that problem. What do you think ?

If you are OK with that, abandon this question and start a new one in Delphi Programming Zone, not IDE or Editors IDE that has nothing to do with that.
Let's says the first step is
Implementation of a threadsafe list of elements to process using OmniThread Lib (template data + ID)
systanAuthor Commented:
Oh, I forgot, I though negative.  Let me check it.

Here's my current test, I divide my app into 2,
a.exe has 1 to 50000 records
b.exe has 500001 to 100001 records
And you know what?, it hits 9 seconds.
The a.exe has 9 seconds, last record found.  The b.exe has 11.500 seconds (not found)

I'll try to get back asap.
Emmanuel PASQUIERFreelance Project ManagerCommented:
I see I did a mistake copying the URL to get OmniThread

please let me know if you are prepared to work step by step on a fast & reliable solution with OmniThread, and stop trying new things every day ;-)
Cloud Class® Course: C++ 11 Fundamentals

This course will introduce you to C++ 11 and teach you about syntax fundamentals.

Geert GOracle dbaCommented:
lol, epasquier, trying out that lib too ?
it's amazing, but i admit, not easy to grasp all the concepts :)
Emmanuel PASQUIERFreelance Project ManagerCommented:
yep, the documentation is a bit lacking and the few there is on the website is out of date. You have to dif in project samples to get your bearing, but when you find how to do this or that, it works on first try !
Simple simple simple... (if you are fluent with threads vocabulary and concepts)

I played a bit with Core affinity, to see that heavy calculations on a i7 (4Cores+ HT = 8 logical cores) is optimum with 8 Threads, but only from a fraction compared to 4 threads, with a little loss between 5-7 threads due to Hyperthreading. That was to be expected, I was even surprised 4 Threads was not just plain better than 8. Still I would recommend using only the nb of physical cores when doing heavy calculations. Which means that a smart application using OmniThread would have to detect HT (or that number of Physical cores), and explicitly set the process affinity to only those cores (the odd ones, so 0,2,4,6 on my system)

Systan, just a question, how many cores does your celeron have ?
Geert GOracle dbaCommented:
he's got 1 core on that ... :)
already noted that threading isn't really the way to go with this topic

>fingerprint recognition as overal goal
Emmanuel PASQUIERFreelance Project ManagerCommented:
sure, it's not gonna help with multithreading one 1 core. I think that by default OmniThread would limit the pool of worker threads that executes simultaneously to the number of logical cores. Or maybe not, it's just an impression I had reading some samples. I haven't try to launch more than 8 worker threads

but I understand the willingness to develop something that CAN be faster, provided the user has a better CPU than the developper... And if the application is done correctly, the potential benefit on multicore does not have to translate in an actual loss when used with only 1 core.
Geert GConnect With a Mentor Oracle dbaCommented:
i have been following the (well more than 1) questions about this problem

> fingerprint recognition
i have been reading up on this a little too
looks like a fingerprint matching technique first scans the fingerprint
then converts this fingerprint into minutia from a central location and orientation

matching 2 fingerprints consists of matching the central location, orientation and minutia

>> see the basics of what needs to be described systan ?

now if i would take all that into account for multithreading
thread 1:
> scan fingerprint
> find central point
> find oriantation
> find minutia relative to central point and orientation
> convert this into db tables/ data
>> this gives what is needed to compare to saved fingerprints

after thread 1 is finished, i could startup searching threads
looks like systan already cut up the dataset into several pieces
to try find a match  
with 1 core there is no use of multithreading (other then having a responsive UI)
the only faster way would be to have more cores
or different computers processing a piece of the table
> for this you need some kind of client-server system to know when you found something
> simple feedback of the record index to the client
> and then calling the db from the client to see the match

but what if systan is using a fingerprint recognition algorithm which gets stuck in a endless loop ?
systanAuthor Commented:
>>stop trying new things every day
Yes, I know, I spend this test for almost 3 years.
Got this in and c#, but I could not figure it out on delphi,  I find delphi very hard, but I am also enjoying the hard time.

It's only my personal test, nothing matters, I also make some test for antivirus apps, and its 80% done, making an antivirus that would globally shield/protect the windows system with the minimum update of virus signatures, lol.  But it is possible as I see it.

Anyway I test the code in quad core processor with 3gig memory free(friends cpu);
5 to 6 seconds, its not good, I though it will be 1 to 2 seconds

>>how many cores does your celeron have ?
I don't know,  but it is intel celeron (mobile) 380 1.6hz

>>overal goal
yes, but no clients involved, its only my personal interest.

>>but I understand the willingness to develop something that CAN be faster
Yes, as I said if it is possible

Do you remember epasquier, that you told me to stop in tmemorystream, I didn't, I dance with it,
So! I got it runnin it on tmemoryStream, but it only gain 4 seconds.
But the fact that I didn't stop, so I got it.
BUT again, not satisfied with the result,  and it was tested on quad core 3gig free mem with 5 seconds.

regarding to a new architecture you proposed, I though ThreadPool can help with a few workers.
Emmanuel PASQUIERFreelance Project ManagerCommented:
Sure, a pool of worker threads is part of the architecture I told you about. But I don't know of a standard mechanism in Delphi for that, maybe in one of those newest versions ?
Which is why I proposed to use OmniThread library for that.

So my question is again : Are you OK with it , and start with the "Producer" = the thread-safe object that can give Workers job to do ( template data ) ?
systanAuthor Commented:
hi Geert;

The things you've mentioned was applied to the identify function of the dll and uses the pthread lock and unlock.
First extract the image, copy bytes to memory, as array of bytes.
Second the identify function, that generates multithreading of the minutia.

I think its time to close this post, since I've fly my code to the other hand,  I just hope it gets better.(if possible to dance why not fly)
systanAuthor Commented:
OmniThread? Its Ok.  As long it performs a very good gain of seconds.
But, if there is a solution without using OmniThread?, that only gain for 1 second?, much better.


Oh, ThievingSix?,  the Expert who answered about threadPool,  Thanks also.
systanAuthor Commented:
Oh about the OmniThreadLiB, I could not do it, I'm to newbie to do that kind of stuff, impossible, my mind is old, tired and abused.  I better wait for a month or less.

Thanks a lot.
Emmanuel PASQUIERFreelance Project ManagerCommented:
you'll see it's easy TO USE once you have done it once. And besides, what are we there for if not helping you to sort it out ?
Of course trying to understand HOW it works is just another story, as Geert said. But that is so with every thread mechanism

From what I've seen, it works as advertised : easily, without visible overhead due to task synchronization, with loads of features yet to explore. But we won't need much exotic features for your problem, so you should get it quickly
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.