Solved

Applying ThreadPool to my code

Posted on 2010-11-22
14
594 Views
Last Modified: 2013-11-23
hi

I started again; Since I could not figure it out how to mix the threadPool unit to my app,  I would like someone to assist me to mix the threadPool code to my code,   put the threadPool to any location of my code.

http://www.experts-exchange.com/Programming/Languages/Pascal/Delphi/Q_24623451.html

Actually the important function to be threaded is this line;
ret := called_from_dll_IDENTIFY( PChar(Field.AsString),


Thanks
Please mix the threadPool to my code.
I would like to increase the points if success to my test.
///

This is in my Main_Form unit, that calls [identifying function] from unit [uUtil]

///

uses uUtil, ...

procedure TForm1.BUTTON_iDENtifyClick(Sender: TObject);

var

  ret, score: Integer;

  Ticks : Cardinal;

begin

  score := 0;

  ticks := GetTickCount();



  ret :=  Identifying (score);



  if ret > 0 then

  begin

    ShowMessage(' BlobField Identified_with_ID is  '+ IntToStr(ret) + '  and the Score of the context is  ' + IntToStr(score) );

  end

  else if ret = 0 then

     ShowMessage('not_iDentified')

  else

    ShowMessage('Error');



    ticks := GetTickCount - ticks;

    ShowMessage('Number of Seconds found:' + IntToStr(ticks) );

end;



==============







///

This is the code of unit uDBClass that it to be called by [Main_Form] unit and unit [uUtil]

///

unit uDBClass;



interface



uses ....



  TDBClass = class

  private

    // the connection object

    mycon: TSQLite3Connection;

    // SQL Transaction

    transact: TSQLTransaction;

  public

    // a data set to maintain all records of database

    ds0: TSQLQuery;



    function openDB(): boolean;

    procedure closeDB();

  end;



implementation

...

...

// Open connection

function TDBClass.openDB(): boolean;

var i,i2,size:integer;

tmp:string;

begin

   try

        ds0 := TSQLQuery.Create(nil);



        mycon := TSQLite3Connection.Create(nil);

        transact := TSQLTransaction.Create(nil);



        mycon.Directory := '';

        mycon.DatabaseName := 'dbsqlite.db';



        transact.DataBase := mycon;

        transact.Action := caCommit;

        transact.Active := True;



        mycon.Transaction := transact;

        mycon.Open;



        ds0.DataBase := mycon;

        ds0.DisableControls;



        ds0.SQL.Text := 'select id, template from table1';

        ds0.CreateDataSet;

        ds0.Prepare;

        ds0.Open;

        ds0.ActiveBuffer;



        openDB := true;

  except

        openDB := false;

  end;

end;



// Close conection

procedure TDBClass.closeDB();

begin

  mycon.Close;

  mycon.Free;

  mycon := nil;

  transact.Free;

  transact := nil;

end;



end.







====================









///

This code is unit [uUtil] that calls database unit [uDBClass]

///

unit uUtil;





interface



uses udBclass,...



Var

  // Database class.

  dB: TDBClass;



implementation



function first_called_from_main_form_during_load_for_opening_db

begin

...

// Opening database

dB := TDBClass.Create();

if not dB.openDB() then

 begin

...

    Exit;

  end;

...

end;



//NEED to apply FIFO (first in first out) like ciuly did in his file search multithreading

function identified(var dsRec: TSQLQuery; out score:integer): integer;

  var ret:integer;

  FIeld: TField;

begin

  Field := dsRec.FieldByName('template');

  dsRec.DisableControls;

dsRec.First;

while not dsRec.EOF do

begin

ret := called_from_dll_IDENTIFY( PChar(Field.AsString), score, 0);

if (ret = 1) then

 begin

 Identified := dsRec.FieldByName('id').AsInteger;

 exit;

 end

 else if (ret < 0) then

 begin

 Identified := ret;

 exit;

 end;

dsRec.Next;

end;

 if (ret = 0) then

 begin

 Identified := 0;

 end;

end;



//This is called in unit Main_Form [uMain]

function Identifying (var score: Integer): Integer;

Begin

Identifying := identified(db.ds0,  score);

end;



end.

Open in new window

0
Comment
Question by:systan
  • 6
  • 5
  • 3
14 Comments
 
LVL 25

Accepted Solution

by:
epasquier earned 150 total points
ID: 34187120
What about the new architecture I proposed in a former mail ? I have been doing some tests with OmniThread this week-end, that lib is outstanding for thread, so I might have something for you, and with a step by step method I'm sure we can finalize something and get done with that problem. What do you think ?

If you are OK with that, abandon this question and start a new one in Delphi Programming Zone, not IDE or Editors IDE that has nothing to do with that.
Let's says the first step is
Implementation of a threadsafe list of elements to process using OmniThread Lib (template data + ID)
file:///E:/Documents/Downloads/OmniThreadLibrary-1.05a.zip
0
 
LVL 14

Author Comment

by:systan
ID: 34187207
Oh, I forgot, I though negative.  Let me check it.

Ok;
Here's my current test, I divide my app into 2,
a.exe has 1 to 50000 records
b.exe has 500001 to 100001 records
And you know what?, it hits 9 seconds.
The a.exe has 9 seconds, last record found.  The b.exe has 11.500 seconds (not found)


I'll try to get back asap.
0
 
LVL 25

Expert Comment

by:epasquier
ID: 34187306
I see I did a mistake copying the URL to get OmniThread

http://omnithreadlibrary.googlecode.com/files/OmniThreadLibrary-1.05a.zip

please let me know if you are prepared to work step by step on a fast & reliable solution with OmniThread, and stop trying new things every day ;-)
0
 
LVL 36

Expert Comment

by:Geert Gruwez
ID: 34187358
lol, epasquier, trying out that lib too ?
it's amazing, but i admit, not easy to grasp all the concepts :)
0
 
LVL 25

Expert Comment

by:epasquier
ID: 34187414
yep, the documentation is a bit lacking and the few there is on the website is out of date. You have to dif in project samples to get your bearing, but when you find how to do this or that, it works on first try !
Simple simple simple... (if you are fluent with threads vocabulary and concepts)

I played a bit with Core affinity, to see that heavy calculations on a i7 (4Cores+ HT = 8 logical cores) is optimum with 8 Threads, but only from a fraction compared to 4 threads, with a little loss between 5-7 threads due to Hyperthreading. That was to be expected, I was even surprised 4 Threads was not just plain better than 8. Still I would recommend using only the nb of physical cores when doing heavy calculations. Which means that a smart application using OmniThread would have to detect HT (or that number of Physical cores), and explicitly set the process affinity to only those cores (the odd ones, so 0,2,4,6 on my system)

Systan, just a question, how many cores does your celeron have ?
0
 
LVL 36

Expert Comment

by:Geert Gruwez
ID: 34187427
he's got 1 core on that ... :)
already noted that threading isn't really the way to go with this topic

>fingerprint recognition as overal goal
0
 
LVL 25

Expert Comment

by:epasquier
ID: 34187466
sure, it's not gonna help with multithreading one 1 core. I think that by default OmniThread would limit the pool of worker threads that executes simultaneously to the number of logical cores. Or maybe not, it's just an impression I had reading some samples. I haven't try to launch more than 8 worker threads

but I understand the willingness to develop something that CAN be faster, provided the user has a better CPU than the developper... And if the application is done correctly, the potential benefit on multicore does not have to translate in an actual loss when used with only 1 core.
0
6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

 
LVL 36

Assisted Solution

by:Geert Gruwez
Geert Gruwez earned 100 total points
ID: 34187542
i have been following the (well more than 1) questions about this problem

> fingerprint recognition
i have been reading up on this a little too
looks like a fingerprint matching technique first scans the fingerprint
then converts this fingerprint into minutia from a central location and orientation

matching 2 fingerprints consists of matching the central location, orientation and minutia

>> see the basics of what needs to be described systan ?

now if i would take all that into account for multithreading
thread 1:
> scan fingerprint
> find central point
> find oriantation
> find minutia relative to central point and orientation
> convert this into db tables/ data
>> this gives what is needed to compare to saved fingerprints

after thread 1 is finished, i could startup searching threads
looks like systan already cut up the dataset into several pieces
to try find a match  
with 1 core there is no use of multithreading (other then having a responsive UI)
the only faster way would be to have more cores
or different computers processing a piece of the table
> for this you need some kind of client-server system to know when you found something
> simple feedback of the record index to the client
> and then calling the db from the client to see the match

but what if systan is using a fingerprint recognition algorithm which gets stuck in a endless loop ?
0
 
LVL 14

Author Comment

by:systan
ID: 34187649
>>stop trying new things every day
Yes, I know, I spend this test for almost 3 years.
Got this in vb.net and c#, but I could not figure it out on delphi,  I find delphi very hard, but I am also enjoying the hard time.

It's only my personal test, nothing matters, I also make some test for antivirus apps, and its 80% done, making an antivirus that would globally shield/protect the windows system with the minimum update of virus signatures, lol.  But it is possible as I see it.

Anyway I test the code in quad core processor with 3gig memory free(friends cpu);
results;
5 to 6 seconds, its not good, I though it will be 1 to 2 seconds

>>how many cores does your celeron have ?
I don't know,  but it is intel celeron (mobile) 380 1.6hz

>>overal goal
yes, but no clients involved, its only my personal interest.

>>but I understand the willingness to develop something that CAN be faster
Yes, as I said if it is possible

Do you remember epasquier, that you told me to stop in tmemorystream, I didn't, I dance with it,
So! I got it runnin it on tmemoryStream, but it only gain 4 seconds.
But the fact that I didn't stop, so I got it.
BUT again, not satisfied with the result,  and it was tested on quad core 3gig free mem with 5 seconds.

Ok;
regarding to a new architecture you proposed, I though ThreadPool can help with a few workers.
0
 
LVL 25

Expert Comment

by:epasquier
ID: 34187695
Sure, a pool of worker threads is part of the architecture I told you about. But I don't know of a standard mechanism in Delphi for that, maybe in one of those newest versions ?
Which is why I proposed to use OmniThread library for that.

So my question is again : Are you OK with it , and start with the "Producer" = the thread-safe object that can give Workers job to do ( template data ) ?
0
 
LVL 14

Author Comment

by:systan
ID: 34187699
hi Geert;

The things you've mentioned was applied to the identify function of the dll and uses the pthread lock and unlock.
First extract the image, copy bytes to memory, as array of bytes.
Second the identify function, that generates multithreading of the minutia.

Ok;
I think its time to close this post, since I've fly my code to the other hand,  I just hope it gets better.(if possible to dance why not fly)
0
 
LVL 14

Author Closing Comment

by:systan
ID: 34187747
OmniThread? Its Ok.  As long it performs a very good gain of seconds.
But, if there is a solution without using OmniThread?, that only gain for 1 second?, much better.

Thanks

Oh, ThievingSix?,  the Expert who answered about threadPool,  Thanks also.
0
 
LVL 14

Author Comment

by:systan
ID: 34187773
Oh about the OmniThreadLiB, I could not do it, I'm to newbie to do that kind of stuff, impossible, my mind is old, tired and abused.  I better wait for a month or less.


Thanks a lot.
0
 
LVL 25

Expert Comment

by:epasquier
ID: 34187805
you'll see it's easy TO USE once you have done it once. And besides, what are we there for if not helping you to sort it out ?
Of course trying to understand HOW it works is just another story, as Geert said. But that is so with every thread mechanism

From what I've seen, it works as advertised : easily, without visible overhead due to task synchronization, with loads of features yet to explore. But we won't need much exotic features for your problem, so you should get it quickly
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

In our object-oriented world the class is a minimal unit, a brick for constructing our applications. It is an abstraction and we know well how to use it. In well-designed software we are not usually interested in knowing how objects look in memory. …
Here is a helpful source code for C++ Builder programmers that allows you to manage and manipulate HTML content from C++ code, while also handling HTML events like onclick, onmouseover, ... Some objects defined and used in this source include: …
This tutorial covers a step-by-step guide to install VisualVM launcher in eclipse.
The viewer will learn how to synchronize PHP projects with a remote server in NetBeans IDE 8.0 for Windows.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now