Solved

Implementation of a thread-safe list of elements to process using OmniThread Lib

Posted on 2010-11-22
50
974 Views
Last Modified: 2012-05-10
Ok, that was the title of the question
How to Implement a thread-safe list of elements to process using OmniThread Lib?

The problem of me?, is my brain, its to old, dumb, used, confused.
But I'll try to understand whatever the multiple comments are;

The other problem, I could not get it to work, the package contains d2009 and d2010, I have d7


Thanks
0
Comment
Question by:systan
  • 29
  • 16
  • 3
  • +1
50 Comments
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
Ok, so you have downloaded OmniThread library and installed the package in Delphi

The main purpose here is to use TOmniMREW object : Multiple Read Exclusive Write synchro object
We will create an object that use it to protect the access to a list of data (string) + ID (integer).
That list can be implemented with a TStringList, but it could also use a DB directly. Or both, depending on a parameter.

unit MultiSearch;



interface



Uses Classes,OtlSync;



Type

 TLockedStringList=Class(TObject)

 private

  _LockList:TOmniMREW;

  _List:TStringList;

  _Index:Integer;

  

  function GetCount:Integer;

  function GetItem(i:integer):String;

  function GetID(i:integer):Integer;

 public

  Constructor Create;

  Destructor Destroy; override;



// loading functions : Thread-safe

  function SetCapacity(Cap:Integer):TLockedStringList;

  function Clear:TLockedStringList;

  procedure Add(S:String;ID:Integer=0);

  

// iteration functions : Thread-safe

  Procedure First;     // Goes to first of list (start iteration)

  Function GetNext(Var S:String;Var ID:Integer):Boolean;

  Procedure EndOfList; // Goes to end of list (stop iteration)



// direct access properties : Thread-safe (through thread safe getter functions)

  property Count:Integer read GetCount;

  property Item[i:integer]:String read GetItem; default;

  property ID[i:integer]:integer read GetID;

 End;



implementation



Constructor TLockedStringList.Create;

begin

 _List:=TStringList.Create;

end;



Destructor TLockedStringList.Destroy;

begin

 _List.Free;

end;



function TLockedStringList.GetCount:integer;

begin

 _LockList.EnterReadLock;

 Result:=_List.Count;

 _LockList.ExitReadLock;

end;



function TLockedStringList.GetItem(i:integer):String;

begin

 _LockList.EnterReadLock;

 Result:=_List[i];

 _LockList.ExitReadLock;

end;



function TLockedStringList.GetID(i:integer):Integer;

begin

 _LockList.EnterReadLock;

 Result:=Integer(_List.Objects[i]);

 _LockList.ExitReadLock;

end;



function TLockedStringList.SetCapacity(Cap:Integer):TLockedStringList;

begin

 _LockList.EnterWriteLock;

 try

  _List.Capacity:=Cap;

 except

 end;

 _LockList.ExitWriteLock;

 Result:=Self;

end;



function TLockedStringList.Clear:TLockedStringList;

begin

 _LockList.EnterWriteLock;

 try

  _List.Clear;

 except

 end;

 _LockList.ExitWriteLock;

end;



procedure TLockedStringList.Add(S:String;ID:Integer=0);

begin

 _LockList.EnterWriteLock;

 try

  _List.AddObject(S,Pointer(ID));

 except

 end;

 _LockList.ExitWriteLock;

end;



Procedure TLockedStringList.First;

begin

 _LockList.EnterWriteLock;

 _Index:=0;

 _LockList.ExitWriteLock;

end;



Procedure TLockedStringList.EndOfList;

begin

 _LockList.EnterWriteLock;

 _Index:=_List.Count;

 _LockList.ExitWriteLock;

end;



Function TLockedStringList.GetNext(Var S:String;Var ID:Integer):Boolean;

Var i:integer;

begin

 _LockList.EnterWriteLock;

 Result:=_Index<_List.Count;

 if Result then

  begin

   i:=_Index;

   Inc(_Index);

  end;

 _LockList.ExitWriteLock;

 if Not Result then Exit;

 _LockList.EnterReadLock;

 S:=_List[i];

 ID:=Integer(_List.Objects[i]);

 _LockList.ExitReadLock;

end;



end.

Open in new window

0
 
LVL 14

Author Comment

by:systan
Comment Utility
>>so you have downloaded OmniThread library and installed the package in Delphi
Yes, I've download it last Saturday, April 05, 2008, 1:08:42 AM,  But I can't install it because I'm only using d7.

I don't know this will gain a little second to my test app, but I'll try to understand what you have.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
Oh, where did you put the;
identify function;
ret := identify(pchar(string), score, 0); ?
It's easy to read, but hard to code, yeah for me(write) right.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
Oh, about this;?
_LockList.EnterReadLock;
 _LockList.ExitReadLock;

It's like TMultiReadExclusiveWrite
0
 
LVL 25

Accepted Solution

by:
epasquier earned 500 total points
Comment Utility
> It's like TMultiReadExclusiveWrite
Exactly

> Oh, where did you put the identify function
Not that part yet. We go step by step remember, the goal of this one is to
- generate
- load / save (from file)
- load from DB
a list of template and ID.

Here is the code of the little app I made this week end.

I tested with my own generated data (random). I added a DB Load for you. You have to set the _DataSet variable to the one you use. Field names might also require change (in btnDBLoadClick)

When you have done so, if you can put the saved file of your templates data somewhere we can download it, then we could all work on the same data.
unit MainSearch;



interface



uses

  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,

  Dialogs, StdCtrls, Buttons, Spin, ExtCtrls, MultiSearch,

  OtlSync, OtlTask, OtlTaskControl, OtlComm, OtlEventMonitor, DB;



type

  TForm1 = class(TForm)

    pnlList: TPanel;

    lbl1: TLabel;

    seListItems: TSpinEdit;

    lbl2: TLabel;

    seItemsSize: TSpinEdit;

    lbl3: TLabel;

    btnGenerateList: TBitBtn;

    btnLoad: TBitBtn;

    btnSave: TBitBtn;

    pnl2: TPanel;

    lbl4: TLabel;

    lblMemUsage: TLabel;

    dlgOpen: TOpenDialog;

    dlgSave: TSaveDialog;

    mmoLog: TMemo;

    btnDBLoad: TBitBtn;

    procedure FormCreate(Sender: TObject);

    procedure FormDestroy(Sender: TObject);

    procedure btnGenerateListClick(Sender: TObject);

    procedure btnDBLoadClick(Sender: TObject);

    procedure btnLoadClick(Sender: TObject);

    procedure btnSaveClick(Sender: TObject);

  private

    { Déclarations privées }

    RecCount,RecSize:integer;

    MemUsed:Cardinal;



    _DataSet:TDataSet;

    _List:TLockedStringList;



    procedure DisplayMemUsage;

  public

    { Déclarations publiques }

  end;



var

  Form1: TForm1;



implementation



{$R *.dfm}



Const

{$ifndef VER200}

 STRING_HEADER_SIZE=8;

{$else}

 STRING_HEADER_SIZE=12;

{$endif}

 MaxDist=4;



Var

 Compare:function (Const S1,S2:String):Boolean;



function FormatNbBytes(v:Cardinal):String;

Var

 e:byte;

 r:extended;

 s:string;

begin

 e:=0;

 r:=v;

 while r>=4096 do

  begin

   r:=r/1024;

   inc(e);

  end;

 case e of

  0 : s:='Oct';

  1 : s:='Ko';

  2 : s:='Mo';

  3 : s:='Go';

  4 : s:='To';

 end;

 Result:=Format('%f %s',[r,s]);

end;



procedure TForm1.FormCreate(Sender: TObject);

begin

 _List:=TLockedStringList.Create;

 dlgOpen.InitialDir:=ExtractFilePath(Application.ExeName);

 dlgSave.InitialDir:=dlgOpen.InitialDir;

 Randomize;

end;



procedure TForm1.FormDestroy(Sender: TObject);

begin

 _List.Free;

end;



// ===== LOAD / SAVE / GENERATE ITEM LIST =====



procedure TForm1.DisplayMemUsage;

begin

 MemUsed:=GetHeapStatus.TotalAllocated-MemUsed;

 lblMemUsage.Caption:=FormatNbBytes(MemUsed)+Format('  with %d items of %d bytes. Overhead of %f bytes per item',[RecCount,RecSize,MemUsed/RecCount-(RecSize+1+STRING_HEADER_SIZE)-sizeof(TStringItem)]);

end;



procedure TForm1.btnGenerateListClick(Sender: TObject);

Var

 i,j:integer;

 S:String;

begin

 _List.Clear;

 MemUsed:=GetHeapStatus.TotalAllocated;

 SetLength(S,seItemsSize.Value);

 RecCount:=seListItems.Value*1000;

 if RecCount<=0 then RecCount:=1;

 _List.SetCapacity(RecCount);

 RecSize:=seItemsSize.Value;

 for i := 0 to RecCount - 1 do

  begin

   for j:= 1 to RecSize do S[j]:=Char(Random(256));

   _List.Add(S, i);

  end;

 DisplayMemUsage;

end;



procedure TForm1.btnDBLoadClick(Sender: TObject);

Var

 DataField:TField;

 IDField:TField;

begin

 _List.Clear;

 // Here, or before in your onFormCreate event, set _DataSet to a valid open TDataSet

 if Not Assigned(_DataSet) then Raise Exception.Create('Dataset not assigned');

 DataField:=_DataSet.FieldByName('TEMPLATE_DATA');

 IDField:=_DataSet.FieldByName('ID');

 Assert(IDField.DataType=ftInteger,'Field '+IDField.FieldName+' not Integer');

 MemUsed:=GetHeapStatus.TotalAllocated;

 RecCount:=_DataSet.RecordCount;

 _DataSet.First;

 while Not _DataSet.Eof do

  begin

   _List.Add( DataField.AsString, IDField.AsInteger );

   _DataSet.Next;

  end;

 DisplayMemUsage;

end;



procedure TForm1.btnLoadClick(Sender: TObject);

Var

 i:integer;

 S:String;

 ID:integer;

 F:File;

begin

 _List.Clear;

 MemUsed:=GetHeapStatus.TotalAllocated;

 if dlgOpen.Execute then

  begin

   AssignFile(F,dlgOpen.FileName);

   Reset(F,1);

   BlockRead(F,RecCount,4);

   _List.SetCapacity(RecCount);

   for i := 0 to RecCount-1 do

    begin

     BlockRead(F,ID,4);

     BlockRead(F,RecSize,4);

     SetLength(S,RecSize);

     BlockRead(F,S[1],RecSize);

     _List.Add(S,ID);

    end;

   CloseFile(F);

  end;

 DisplayMemUsage;

end;



procedure TForm1.btnSaveClick(Sender: TObject);

Var

 i,RecCount,RecSize:integer;

 ID:integer;

 S:String;

 F:File;

begin

 if _List.Count=0 then Exit;

 if dlgSave.Execute then

  begin

   RecCount:=_List.Count;

   AssignFile(F,dlgSave.FileName);

   Rewrite(F,1);

   BlockWrite(F,RecCount,4);

   for i := 0 to RecCount-1 do

    begin

     ID:=_List.ID[i];

     BlockWrite(F,ID,4);

     S:=_List[i];

     RecSize:=Length(S);

     BlockWrite(F,RecSize,4);

     BlockWrite(F,S[1],RecSize);

    end;

   CloseFile(F);

  end;

end;



end.



//== DFM



object Form1: TForm1

  Left = 0

  Top = 0

  BorderWidth = 4

  Caption = 'Form1'

  ClientHeight = 460

  ClientWidth = 646

  Color = clBtnFace

  Font.Charset = ANSI_CHARSET

  Font.Color = clWindowText

  Font.Height = -13

  Font.Name = 'MS Sans Serif'

  Font.Style = []

  OldCreateOrder = False

  OnCreate = FormCreate

  OnDestroy = FormDestroy

  PixelsPerInch = 96

  TextHeight = 16

  object pnlList: TPanel

    Left = 0

    Top = 0

    Width = 646

    Height = 36

    Align = alTop

    BevelOuter = bvLowered

    Constraints.MinWidth = 552

    TabOrder = 0

    ExplicitWidth = 552

    object lbl1: TLabel

      Left = 98

      Top = 9

      Width = 52

      Height = 16

      Caption = 'Data List'

    end

    object lbl2: TLabel

      Left = 215

      Top = 9

      Width = 86

      Height = 16

      Caption = 'x1000 items of '

    end

    object lbl3: TLabel

      Left = 357

      Top = 9

      Width = 33

      Height = 16

      Caption = 'bytes'

    end

    object seListItems: TSpinEdit

      Left = 156

      Top = 6

      Width = 52

      Height = 26

      MaxValue = 9999

      MinValue = 0

      TabOrder = 0

      Value = 100

    end

    object seItemsSize: TSpinEdit

      Left = 303

      Top = 6

      Width = 48

      Height = 26

      MaxValue = 0

      MinValue = 0

      TabOrder = 1

      Value = 723

    end

    object btnGenerateList: TBitBtn

      Left = 4

      Top = 4

      Width = 88

      Height = 26

      Caption = 'Generate'

      TabOrder = 2

      OnClick = btnGenerateListClick

      Glyph.Data = {

        36030000424D3603000000000000360000002800000010000000100000000100

        18000000000000030000C40E0000C40E00000000000000000000F6F5F3F6F5F3

        F6F5F3F6F5F3F6F5F3F6F5F3004F66F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5

        F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F300819A00

        4F66F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3

        F6F5F3F6F5F3F6F5F3F6F5F300819A00C5EA004F66F6F5F3F6F5F3F6F5F3F6F5

        F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3004F66004F66004B7000

        819A00F3FF006081F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3

        F6F5F3F6F5F3F6F5F3004F6696FFFF00E9FF00DCFC00F1FF004F6DF6F5F3F6F5

        F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F300709C0089ACFF

        FFFF3BE6FC0CE4FF00FDFF004968F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3

        F6F5F3F6F5F3004F66004F66004F66004F6632EDFF004F66004F66004F660012

        32F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3004F6600B6DE89FEFF55F0FF43

        EAFF52EBFF0DE1FE004F66005E84F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3

        004E7E01B8DD00ACD23AEBFF89F0FE5EE9FC5DF5FF004F66004F66004F66F6F5

        F3F6F5F3F6F5F3F6F5F3F6F5F3FBF7F4004E7E009EC800B4D91DC7E7B1F9FF7F

        EEFE6BF1FF1CBFD7005372619DAAF6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F7F5F3

        F6F5F3006A9201BEE400B7E0ECFFFF77EEFE81EFFE6EFAFF004F66F6F5F3F6F5

        F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3004E7E00B9E100B4DD77F6FFFF

        FFFFABF7FF85EFFE64FEFF004F66F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3

        F6F5F3F6F5F3004E7E00C0E400A7D313B1D932D4F1CEFFFFC7FFFF85FFFF004F

        66F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3004E7E01B3E200AEDB00

        A6D5009ECC009ACB0097CA00A1D001B9F2000F2EF6F5F3F6F5F3F6F5F3F6F5F3

        F6F5F3F6F5F3F6F5F300648D006B8F008DBA009ED200A4D601ACDC01AAE30028

        44F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F3F6F5F300

        6B8F004E7E004E7E004E7E00435BF6F5F3F6F5F3F6F5F3F6F5F3}

    end

    object btnLoad: TBitBtn

      Left = 412

      Top = 4

      Width = 64

      Height = 26

      Caption = 'Load'

      TabOrder = 3

      OnClick = btnLoadClick

      Glyph.Data = {

        36030000424D3603000000000000360000002800000010000000100000000100

        18000000000000030000C40E0000C40E00000000000000000000D8E9ECD8E9EC

        D8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9EC8F6E1CD8E9ECD8E9ECD8E9

        ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8

        E9EC816100FF9F2A8F6F1DD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECA8A8A8737373

        553F2A553F2A3D3D3D3D3D3DB4B4B4806100FF9F2AD9A77DFF9F2A8F6F1C7373

        737474743D3D3DA7A7A7919191A9FFFF99F8FF99F8FF55DFFF55BFD3806100D9

        A77DD9A77DD9A77DD9A77DFF9F2A8F6F1D55DFFF55C0D43D3D3D919191A9FFFF

        A9FFFFA9FFFFA9FFFF806100D9A77DFFFFC3FFFFC3D9A77DD9A77DD9A77DFF9F

        2A8F6F1D55DFFF3D3D3D919191A9FFFFA9FFFFA9FFFFE1BE788F6F1C8F6F1C8F

        6E1CFFFFC3FFFFC4D9A77D8F6F1D8F6F1C8F6F1CE1BE78737373909090A9FFFF

        A9FFFFA9FFFFA9FFFFA9FFFFA9FFFF9D7C30FFFFC3FFFFC3FFF1AB9D7C30B4B4

        B499F8FF55DFFF3D3D3D919191D8E9ECA9FFFFA9FFFFA9FFFFA9FFFFA9FFFFAB

        8A40FFFFC3FFFFC3D9A77DFBD79198F7FF55DFFFAADFD53D3D3D919191A9FFFF

        A9FFFFA9FFFFA9FFFFA9FFFFB9974FD9A77DFFF0AAFFFFC3AB8A4099F7FF99F7

        FF99F8FF55DFFF595959A8A8A8D8E9ECA9FFFFD8E9ECA9FFFFA8FEFFB9974EFF

        E49EFFF0AAE1BE77FBD790A9FFFFA9FFFFA9FFFFA9FFFF666666B4B4B4A9FFFF

        D8E9ECA9FFFFA8FEFFAB893FD9A77DFBD790E1BE78A9FFFFA9FFFFA9FFFFA9FF

        FFA9FFFFA9FFFF737373C1C1C1D8E9ECA9FFFFD8E9ECD4B16AD4B26BD4B26BD4

        B26BEECB84C6C6C6C6C6C6AADFD555BFD354BFD355BFD3818181CCCCCCB5B5B5

        B4B4B4B4B4B4CBCBCBAADFD5AADFD555DFFF55DFFF55DFFF55DFFF55DFFF55DF

        FF55DFFF54BFD3C1C1C1D8E9ECDADADA00F2FF00F2FF00F1FF00F1FF55BFD3E6

        E6E68D8D8D8D8D8D8D8D8D8D8D8D8D8D8D8D8D8D8D8D8DD8E9ECD8E9EC9B9B9B

        54FFFF67F4FF67F4FF67F4FF00F1FFB4B4B4D8E9ECD8E9ECD8E9ECD8E9ECD8E9

        ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9EC9B9B9B8D8D8D8D8D8D8D8D8D9B9B9BD8

        E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9EC}

    end

    object btnSave: TBitBtn

      Left = 482

      Top = 4

      Width = 64

      Height = 26

      Caption = 'Save'

      TabOrder = 4

      OnClick = btnSaveClick

      Glyph.Data = {

        36030000424D3603000000000000360000002800000010000000100000000100

        18000000000000030000C40E0000C40E00000000000000000000D8E9ECD8E9EC

        D8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9

        ECD8E9ECD8E9ECD8E9ECD8E9ECD8E9ECA7A7A77373737373735959595959594B

        4B4B4B4B4B3D3D3D303030303030303030A7A7A7D8E9ECD8E9ECD8E9ECA7A7A7

        633600CDCDCDE6E6E6C1C1C1C1C1C1CDCDCDF0F0F0EDEDEDE6E6E6A7A7A73333

        33303030A7A7A7D8E9EC633600633600633600DADADAE6E6E6E6E6E6E6E6E6E6

        E6E6E6E6E6E6E6E6E6E6E6DADADACB8C44633600303030303030633600CB8C44

        633600D9A77DD9A77DD9A77DD9A77DD9A77DCB8C44CB8C44CB8C44CB8C44CB8C

        446336007F5B00303030633600D9A77D633600D9A77DD9A77DD9A77DD9A77DD9

        A77DD9A77DCB8C44CB8C44CB8C44CB8C44633600CB8C44303030633600D9A77D

        633600D9A77DD9A77DD9A77DD9A77DD9A77DD9A77DD9A77DCB8C44CB8C44CB8C

        44633600CB8C443D3D3D633600D9A77D633600AA3F2A63360063360063360063

        3600633600633600633600633600CB8C44633600CB8C444B4B4B633600D9A77D

        6336009A9A9AAAFFFF99F8FF99F8FF99F8FF99F8FF99F8FF99F8FF99F8FF6336

        00633600CB8C444B4B4B633600D9A77D633600AAFFFFCDCDCDA7A7A7A7A7A7A7

        A7A7A7A7A7A7A7A7A7A7A7C1C1C199F8FF633600CB8C444B4B4B633600D9A77D

        7F5B00AAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFF99F8

        FF7F5B00CB8C444B4B4B633600D9A77D7F5B00AAFFFFCDCDCDA7A7A7A7A7A7A7

        A7A7A7A7A7A7A7A7A7A7A7C1C1C199F8FF7F5B00CB8C444B4B4B633600D9A77D

        989898AAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFF99F8

        FF989898CB8C44595959633600D9A77DA6A6A6D8E9ECCDCDCDCB8C44CB8C44A7

        A7A7A7A7A7A7A7A7A7A7A7C1C1C199F8FFA6A6A6D9A77D666666A7A7A7633600

        7F5B00D8E9ECD8E9ECAAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFFFFAAFF

        FF7F5B00633600A7A7A7D8E9ECD8E9ECA7A7A763360063360063360063360063

        36006336006336006336006336006336009A9A9AD8E9ECD8E9EC}

    end

    object btnDBLoad: TBitBtn

      Left = 552

      Top = 4

      Width = 80

      Height = 26

      Caption = 'DB Load'

      TabOrder = 5

      OnClick = btnDBLoadClick

      Glyph.Data = {

        36030000424D3603000000000000360000002800000010000000100000000100

        18000000000000030000130B0000130B00000000000000000000FF00FFFF00FF

        FF00FFFF00FFFF00FFFF00FFFF00FFFF00FFFF00FFFF00FFFF00FFFF00FFFF00

        FFFF00FFFF00FFFF00FFFF00FF23738A327C914894A95A98AA66A9BB66A9BB5A

        98AA5993A43B7C8F20677B2A7186FF00FFFF00FFFF00FFFF00FF3F82963E95AE

        78B9CC4D95AA76BACE99CFDED4EBF1BCD5DB97BDC76DA2B144889C1C677D005D

        78FF00FFFF00FFFF00FF1C687E449EB882C1D34890A570B7CB98CEDDD7EEF5C1

        D8DE9CC0CA73A6B44A8C9F206E8417576AFF00FFFF00FFFF00FF1C687E449EB8

        82C1D34890A570B7CB98CEDDD7EEF5C1D8DE9CC0CA73A6B44A8C9F206E841757

        6AFF00FFFF00FFFF00FF1C687E197D99428FA54B93A869ACBF85BFCF95C4D281

        B2C0689EAD4C899A31738626983E17576AFF00FFFF00FFFF00FF005D784495AC

        82C1D34890A570B7CB98CEDDD7EEF5C1D8DE9CC0CA73A6B426983E4BCD6A1E78

        31FF00FFFF00FFFF00FF1C687E449EB882C1D34890A570B7CB98CEDDD7EEF5C1

        D8DE9CC0CA26983E4BCD6A32A7492DA0411E7831FF00FFFF00FF1C687E449EB8

        82C1D34890A570B7CB98CEDDD7EEF5C1D8DE26983E26983E26983E21AD3921AD

        3921AD3926983EFF00FF1C687E2488A4529BB04A92A76BAFC28AC3D3A6CFDB91

        BBC775A7B55690A126983E4BCD6A26983EFF00FFFF00FFFF00FF005D78398AA2

        72B5C84991A56FB4C893CADAC7E3ECB1CED68FB8C3699FAE26983E4BCD6A2698

        3EFF00FFFF00FFFF00FF1C687E449EB882C1D34890A570B7CB98CEDDD7EEF5C1

        D8DE9CC0CA73A6B426983E26983E17576AFF00FFFF00FFFF00FF1C687E449EB8

        82C1D34890A570B7CB98CEDDD7EEF5C1D8DE9CC0CA21AD3926983E206E841757

        6AFF00FFFF00FFFF00FF1C687E469FB973A6B44A8C9F2B819A2B819A2B819A2B

        819A26983E26983E387D911F6D8317576AFF00FFFF00FFFF00FF397F932B819A

        4890A570B7CB66A9BB9CC0CAC1D8DE9CC0CA70B7CB4890A524697D135C70005D

        78FF00FFFF00FFFF00FFFF00FF19728B196C833B879C3A8CA25A98AA5A98AA3A

        8CA24786993272851C5E723E899EFF00FFFF00FFFF00FFFF00FF}

    end

  end

  object pnl2: TPanel

    Left = 0

    Top = 36

    Width = 646

    Height = 41

    Align = alTop

    BevelOuter = bvLowered

    TabOrder = 1

    ExplicitWidth = 552

    object lbl4: TLabel

      Left = 8

      Top = 9

      Width = 77

      Height = 16

      Caption = 'List is using : '

    end

    object lblMemUsage: TLabel

      Left = 88

      Top = 9

      Width = 3

      Height = 16

    end

  end

  object mmoLog: TMemo

    Left = 0

    Top = 77

    Width = 646

    Height = 383

    Align = alClient

    TabOrder = 2

    ExplicitTop = 137

    ExplicitWidth = 552

    ExplicitHeight = 157

  end

  object dlgOpen: TOpenDialog

    DefaultExt = 'TPL'

    Filter = 'Template List|*.TPL'

    FilterIndex = 0

    Left = 384

    Top = 8

  end

  object dlgSave: TSaveDialog

    DefaultExt = 'TPL'

    Filter = 'Template List|*.TPL'

    FilterIndex = 0

    Left = 480

    Top = 24

  end

end

Open in new window

0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
> But I can't install it because I'm only using d7.
Shit ! I worked on D2007.... maybe there is an older version that could setup on D7 ? Or I'll see if I can make it compile under D7
0
 
LVL 5

Expert Comment

by:briangochnauer
Comment Utility
to epasquier,

systan is using FreePascal (Lazarus) which claims only language compatibility with Delphi 7.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
hi
I'm using Lazarus [also], but delphi and Lazarus are most exactly the same, 80% of it.

But don't worry, I'm go for delphi.  If I ask for Lazarus?, I ask to Lazarus forum.

>>I worked on D2007?
Yep, But mine is Delphi 7.3+

Anyway, just continue, I'm reading some ideas from you epasquier, thanks.

>>Or I'll see if I can make it compile under D7
I don't know if you can do that, its 4:20 PM
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
Ok, I've tried quickly to compile it with D7, it would require a bit of work to avoid code style that where not yet introduced with D7 ( for .. in .. )

before going to that , I tried to find other options, and found this that sums it up :
http://efreedom.com/Question/1-3419945/Choose-Various-Ways-Threading-Delphi

I also conclude that one should seriously think about upgrading if one want to deal with threads, because OmniThread is the best options. Or as suggested, and as I tried, fix that D7 incompatibility by altering the whole lib source.

I think it is the best option, but will not do it today. Stay tuned, I will try

While I do that, remove
OtlSync, OtlTask, OtlTaskControl, OtlComm, OtlEventMonitor,
in the uses of Main form (not used yet)

and in MultiSearch unit replace TOmniMREW with TMultiReadExclusiveWriteSynchronizer , and the methods names as well that are not the same
0
 
LVL 14

Author Comment

by:systan
Comment Utility
Hey epasquier;
How about sending me your compiled binary exe, just renamed it to jpg, I'll try to run your test app on this codes;
11/22/10 06:48 AM, ID: 34188304

Or, if is Ok; can you attach the zip file here, I'd like to test it. (wow the problem is I'm with d7)
Anyway, an .exe will do, just to look at it.

thanks

oh, hi briangochnauer;
0
 
LVL 14

Author Comment

by:systan
Comment Utility
Why OmniThreadLibrary is very good that borland TThread? hmp, don't answer, lol

>>OmniThread is the best options
But, OmniThread is using the Old threading like TMultiReadExclusiveWrite just renamed to TOmniMREW

Anyway I have to listen at you, and maybe
#2. AsyncCalls by Andreas Hausladen is not the option also.

Oh, let me have the .exe of  ID: 34188304

thanks
0
 
LVL 14

Author Comment

by:systan
Comment Utility
AsyncCalls by Andreas Hausladen uses also TThread
>>TThread = class(Classes.TThread)
Maybe I'll try this also.

OmniThread also use;
>> TOmniThread = class(TThread) // on OtlTaskControl.pas
>>TOTPWorkerThread = class(TThread) //on OtlThreadPool.pas

Ok, wheres the exe

0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
Exec for above code (Delphi 2007)
MultiThreadSearch.zip
0
 
LVL 14

Author Comment

by:systan
Comment Utility
Ok;
Thanks I've seen it.
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
well, of course they all use Thread. Nobody said TThread is not good, it is just too basically close to WinAPI for practical use at a high level. It needs too much code and there are so many ways of doing things wrong when left alone with TThread, basic system sync objects & messages and other peculiar aspects of MultiThreading
0
 
LVL 5

Expert Comment

by:briangochnauer
Comment Utility
Newbies and threads are like moths to a flame; with the same result.
0
 
LVL 36

Expert Comment

by:Geert Gruwez
Comment Utility
>> epasquier
your own TLockedStringList ???

nice work, but wouldn't you say that this sample would help more ?
http://17slon.com/blogs/gabr/2008/07/omnithreadlibrary-internals.html

OTL has it's own objects for adding/removing items from a list

i admit, it does add complexity but then it wouldn't attract moths if it didn't :)
0
 
LVL 14

Author Comment

by:systan
Comment Utility
Geert?
Have you test/used AsyncCalls by Andreas Hausladen
0
 
LVL 36

Expert Comment

by:Geert Gruwez
Comment Utility
By Who ?
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
Geert, you are right that we could use some of the Omni containers. First, I would say for my defence that I haven't looked through all of Omni, by far, I only read what was useful for a simple project as Systan's.

And second, the idea of the above list is only a first step along a way that will ultimately be neither a stack nor a queue, but a list that is scanned from first to last and can be either completely preloaded , completely uncached DB, or a mix of both world  (one thread preloads the data while the worker threads use those).

So quickly enough there will be no room for Omni containers, and showing how to use MREW locks to protect about anything is probably more useful to Systan in near future
0
 
LVL 14

Author Comment

by:systan
Comment Utility
>>By Who ?
By you Geert

hi epasquier;
Do you think its really possible to speed up using OTL?
How about AsyncCalls by Andreas Hausladen?
I hope Geert got the full code also, so he could understand all the way also.
0
 
LVL 36

Expert Comment

by:Geert Gruwez
Comment Utility
>>Have you test/used AsyncCalls by Andreas Hausladen
I didn't test or use that ... do i really have to ?
don't plan on using it

It is only a subset of what OTL is looking at the description
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
> Do you think its really possible to speed up using OTL?
I believe it is possible to speed up on MULTI CORE machines using Multi thread search.

Multi Threading could be done with many ways, some easy, some not easy. some that will work with all delphi versions, some that will work only on newest. That's a tough choice.

Omnithread has everything one could dream about threading. So it is a choice of future. You use Delphi 7 that is not supported, well that's a bad piece of news, because your project was a good example to test OmniThread capabilities. One word on my Core i7 & Delphi 2007 : Awesome !

But it is probably possible to do it also with TThreads and a few mechanisms picked from OmniThread, that will compile with D7. I'll have to think about the best option, but I don't let you down.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
ahah, so both of you have an eye of OTL.
>>TThreads and a few mechanisms picked from OmniThread,
I don't know how to do that, maybe Geert and Epasquier can do that, but me, nop, I have to study the codes first that is running than I could build my own.
Ok, I have to wait for the analysis of epasquier, I just hope there is a solution for that codes.
I guess no ones interested in the full source code, when no ones bothering to contact me in my private address.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
I wonder, whats the best option to deal with this identify function, is it OTL, AsyncCalls, ThreadPool?
Anyway, I have to wait for epasquier's answer upon his test.

Using it in intel quad core with 3gigfreemem 5 to 6 seconds for thousand records
Using it in dual core athlon ii 5200 with 1.5gigfreemem 6 to 8 seconds
Using it in my celeron 0core 380 with 620freemem 11 to 12 seconds
But thats using only the tthread with 2threads

I hope to got it better on any theadLibs.


thanks
0
Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

 
LVL 14

Author Comment

by:systan
Comment Utility
I just test AsyncCalls by Andreas Hausladen
Not good, it even pause/flickers on any of my application using it.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
The FasterThread by Jyrki Kyllönen, I did Test but I dont understand it well
OTL, I can't test, so I'll waif for epasquiers comment.
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
I'm looking at your full code right now, I will first "clean" it, or just maybe rewrite it...

As I said already, that is completely wrong, for a whole loads of reasons. You are leaking memory wildly (continuous realloc of StreamRec1), writing records in a stream that is inefficient compared to array of records, and those records have pointers in them, that points to the same memory zone pointed by AsString (if not worse)

as I said : TStringList is the thing to use. You'll probably understand once you see the fixed code
while not dsTemplates.EOF do

  begin

   New(StreamRec1);

   StreamRec1^.id := dsTemplates.FieldByName('id').AsInteger;

   StreamRec1^.blob :=dsTemplates.FieldByName('template').AsString;

   tptStream1.WriteBuffer(StreamRec1, sizeOF(StreamRec1));

   dsTemplates.Next;

  end;

Open in new window

0
 
LVL 14

Author Comment

by:systan
Comment Utility
Epasquier, that code is a msaccess db, I just use that db for quick reading of records and quick access.
If I will use SQLite just like the code below, maybe you don't have the component.
I started it on a standalone db, so anyone would try will directly know and see the result by just running the code in the designtime.

Here's my original code(SQLite)

So, whats you comment on that code? iS it the same?
What about applying Records from TmemoryStream on OTL?


Oh. I got delphi 2009, but problem using it,  because Image1 is cleared during load time.  Not on Delphi 7.+
/////////////////////

//in unit [udBclass]

type

PmyStreamRec = ^myStreamRec;

myStreamRec = packed record

id: Integer;

blob: String;

end;

...

...

type

  TMyThread = class(TThread)

  private

    xmycon: TSQLite3Connection;

    xtq: TSQLQuery;

    SQLString : string;

    limitfirst:integer;

    limitlast:integer;

    Priority: TThreadPriority;

    public

    RecStream: PmyStreamRec;

    MemStream: TMemoryStream;

  protected

    procedure Execute; override;

  end;

...

...

{ TMyThread }

procedure TMyThread.Execute;

begin

  inherited;

  xtq.DataBase := xmycon;

  xtq.DisableControls;

  xtq.ReadOnly := true;

  xtq.Params.CreateParam(ftUnknown,'aa',ptUnknown);

  xtq.Params.CreateParam(ftUnknown,'bb',ptUnknown);

  xtq.SQL.Text := SQLString;

  xtq.Params.ParamByName('aa').AsInteger := limitfirst;

  xtq.Params.ParamByName('bb').AsInteger := limitlast;

  xtq.CreateDataset;

  xtq.Prepare;

  xtq.Open;

  xtq.ActiveBuffer;

  while not xtq.EOF do

  begin

  New(RecStream);

  ///////////////////////////GetMem(RecStream, SizeOF(RecStream));

  ///////////////////////////RecStream := allocmem(SizeOf(RecStream));

  RecStream^.id := xtq.FieldByName('id').AsInteger;

  RecStream^.blob := xtq.FieldByName('template').AsString;

  MemStream.WriteBuffer(RecStream, sizeOF(RecStream));

  xtq.Next;

  end;



  xtq.Close;

  xtq.Free;

end;





function RunThread(PraMStream: TMemoryStream; RecTypeStream: PmyStreamRec; xmycon: TSQLite3Connection; tq: TSQLQuery; SQLString:string; xlf: integer; xll:integer; Priority: TThreadPriority):TMyThread;

var

  MyThread : TMyThread;

begin

  MyThread := TMyThread.Create(false);

  MyThread.Priority := Priority;

  MyThread.xmycon := xmycon;

  MyThread.xtq := tq;

  MyThread.SQLString := SQLString;

  MyThread.limitfirst := xlf;

  MyThread.limitlast := xll;

  MyThread.MemStream := PraMStream;

  MyThread.RecStream := RecTypeStream;

  MyThread.Resume;

  Result := MyThread;

end;

...

...

function TDBClass.openDB(): boolean;

var i,i2:integer;

begin

   try

        ds0 := TSQLQuery.Create(nil);

        ds1 := TSQLQuery.Create(nil);

        ds2 := TSQLQuery.Create(nil);



        mycon := TSQLite3Connection.Create(nil);

        transact := TSQLTransaction.Create(nil);

        mycon.Directory := '';



        mycon.DatabaseName := 'biofinger.db';



        transact.DataBase := mycon;

        transact.Action := caCommit;

        transact.Active := True;



        mycon.Transaction := transact;

        mycon.Open;



        ds0.DataBase := mycon;

        ds0.DisableControls;

        ds0.ReadOnly := true;

        ds0.SQL.Text := 'select count(id) as xcount from enroll';

        ds0.Prepare;

        ds0.Open;

        ds0.ActiveBuffer;



        i:=ds0.FieldByName('xcount').AsInteger;

        ds0.Close;

        ds0.Free;

        i2:= (i div 2);

        ticks := GetTickCount();

        tptStream1 := TMemoryStream.Create;

        tptStream2 := TMemoryStream.Create;

        ct1 := RunThread(tptStream1, StreamRec1, mycon, ds1,'select id, template from enroll limit :aa, :bb',0,i2+1,tpTimeCritical);

        ct2 := RunThread(tptStream2, StreamRec2, mycon, ds2,'select id, template from enroll limit :aa, :bb',i2-1,i,tpNormal);





        openDB := true;

  except

        openDB := false;

  end;

end;

...

end.

...

...

...

/////////////////////

/////////////////////

//in unit [uUtil]

....

procedure tmythread.showStatus;

begin

writeLog(inttostr(xscore) + ':' + inttostr(retvalue));

end;

...

procedure tmythread.execute;

var ret:integer;

score: integer;

i:integer;

begin

  MStream.Position := 0;

  while (MStream.Position < MStream.Size) do

  begin

  MStream.ReadBuffer( streamrec, SizeOf(streamrec) );

  ret := MyIdentify( PChar( streamrec.blob ), score, 0);

    if (ret = 1) then

    begin

      xscore:=score;

      RetValue := streamrec.id;

      synchronize(showStatus);

      exit;

    end

    else if (ret<0) then

    begin

    xscore:=score;

    RetValue := ret;

    synchronize(showStatus);

    exit;

    end;

  end;

  if (ret=0) then

  begin

    xscore:=score;

    RetValue := ret;

    synchronize(showStatus);

  end;

end;

...

...

function MyIdentify(var score: integer): integer;

const max=2;

var

  mythreads: array[0..max-1] of tmythread;

begin

   mythreads[0] := tmythread.Create(true);

   mythreads[0].MStream := db.tptStream1;

   mythreads[0].StreamRec := db.StreamRec1;

   mythreads[0].ftemplate := raw.template;

   mythreads[0].Resume;



   mythreads[1] := tmythread.Create(true);

   mythreads[1].MStream := db.tptStream2;

   mythreads[1].StreamRec := db.StreamRec2;

   mythreads[1].ftemplate := raw.template;

   mythreads[1].Resume;



   for i:=1 to Max-1 do if not myThreads[i].Terminated then Sleep(200);



   for i:= 1 to max do

   begin

      mythreads[i-1].Terminate;

   end;



end;

...

end.

Open in new window

0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
I wonder if I have made a mistake using your dll, I have a strange behaviour of the Identify function.
Here is what I do :

I load a bitmap and create its template. For your sample image, that template returned by the dll is 554 bytes.
I generate a list of random strings (templates) of that same size  : 100.000 items, with an ID=its position in my list, but it could well be otherwise.
I start search of the template, first inserting the one loaded from the bitmap in my list at the test position (here, end of list)

Here is the app screen :

 FingerSearch App
The time it takes to compare with 100.000 items preloaded in a TStringList (55,3 Mo used) is extremely short. In fact, so short that it is better run in 1 thread only.

If I compare 100.000 times the template loaded to itself, it takes ~15s and seems limited to 1 core only. I am absolutely sure the framework is OK, as I can change the Compare function (that calls Identify) and replace it with 2 functions : one that compares by string equality, and the other by "distance"=Sum(Sqr(S1[n]-S2[n])) + a loop to consume time by doing heavy calculations.

When I run it with that distance comparison, I can see depending on the nb of task I launched the CPU/Core usage going up, on the exact Cores I assigned the threads to.
And I can see here the benefits of multi-core , even of the hyper-threaded cores.

I don't know what to think of this, except that this dll does not allow multi-threading for an unknown reason. Or that my random data are so completely wrong that the Identify function does not spend much time on it.

I will finish tomorrow the application DB load part, so that you can try with your own DB, not my random data.
By the way, how did you get your DB template set ? you scanned a 100.000 fingers ?
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
> So, whats you comment on that code? iS it the same?
Yes, completely the same mistake.

> What about applying Records from TmemoryStream on OTL?
since Records from TMemoryStream is an absolute horror to me, no problem with those.

Here are the stats of TStringList usage :
- 55,3 Mo used for 100.000 strings of 554 bytes => almost 0 bytes lost
- the full search using a simple string comparison function takes 0,031s to find the last element

That is purely unbeatable, no need to try to find other ways to load/store your data.
The problems of long times can only occur in Identify function, and apparently only when the template compared are close enough.
Search of 100.000 random templates + last one = same as searched takes 0,063s ; so as a mean, Identify does twice as much simple comparisons as string equality to reject a bad template.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
>>that template returned by the dll is 554 bytes.
Yes, that's the last record, record id:100,007

>>In fact, so short that it is better run in 1 thread only.
>>If I compare 100.000 times the template loaded to itself, it takes ~15s and seems limited to 1 core only
>>I don't know what to think of this, except that this dll does not allow multi-threading for an unknown reason.
This answers possibly;
Yes, I don't why also,  but the identify function from a dll is also threaded
When the [raw image] is extracted, prepare_to_identify function loads it in the memory as a thread also
So, identify function depends on prepared [raw image].
This is what I am concerned about threads get mix up in the memory?

>>By the way, how did you get your DB template set ? you scanned a 100.000 fingers ?
I'll explain on this way;
As you notice there are records starting from 3 to 100,007(i deleted 1 and 2 for testing)
With my 10 fingers?,
First Set of template is my Right hand;
1: record 3 only
2: record 4 to 10,000
3: record 10,001 to 20,000
4: record 20,001 to 30,000
5: record 30,001 to 40,000
Second Set of template is my Left hand;
1: record 40,001 to 50,000
2: record 50,001 to 60,000
3: record 60,001 to 70,000
4: record 70,001 to 100,006
5: record 100,007 [which is the last record finger] that you can see you the screen

>>I will finish tomorrow the application DB load part, so that you can try with your own DB, not my random data.
I just hope it does gain some seconds than the way my code does.
But I can see OTL is hard to mix up on the way this code behaves, because I can't?

Thanks,
Well see that soon.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
>>the full search using a simple string comparison function takes 0,031s to find the last element
Oh, surprising.
>>That is purely unbeatable, no need to try to find other ways to load/store your data.
lol, I know, I believed you.
But, I doubt a little, that's to impossible.  But I'm shock with the results.
Can you set it to 4 threads only(if will not degrade speed in ms).

Thanks a lot
Now you dance with it  and seems  no impossible when Good Delphi Experts move.
Not like me, I'm a bad Expert,  still a newbie for a year and a quarter.  Isn't the reason that I have no books of Delphi?   I just research on the internet.   My eyes get tired directly, my brain drain when I see a lot of codes that I can't understand.  lol,  I'm striving hard to learn delphi as an expert.

Ok;
I think I have to close this post now.
0
 
LVL 14

Author Closing Comment

by:systan
Comment Utility
.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
Oh, not all templates have 544 bytes, average is 600 bytes and it also depends on the users fingers pattern, probably the exact average is 1k(but unusual)
0
 
LVL 14

Author Comment

by:systan
Comment Utility
Again;
>>the full search using a simple string comparison function takes 0,031s to find the last element
31 seconds? in my celeron 380 1.6mhz 620 free memory is only 11 seconds.
or
31 milliseconds
Sorry, my bad, can't understand 0,031s

>>That is purely unbeatable, no need to try to find other ways to load/store your data.
Sure, if it's 31 milliseconds per 100thousand records, this is very impossible.
Try to read my post above.
>>By the way, how did you get your DB template set ? you scanned a 100.000 fingers ?
0
 
LVL 14

Author Comment

by:systan
Comment Utility
hello epasquier;
Where are you?,  your speechless about my last 4 comments.
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
sorry, I had work to do (yes, that happens)
I'll release you my application compiled so that you can try it, and export data for me to try also with the same conditions
I also have an idea to test, to circumvent the fact that this bloody dll is limiting its usage to one thread. I'll tell you more when I have made research on it
0
 
LVL 14

Author Comment

by:systan
Comment Utility
ok;
can you just attached the binary compiled exe here, not including the dlls.
So I can try, if it gain a little or not.

Thanks
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
Here is the app. You can check the INI file for the connexion parameters on your DB file

Click the "DB Load only selected rec." button, it will ask you a list of records to load in memory, that list is 0-based (the list is defaulted to what I think should be the record index of your 10 fingers)
if it loaded 10 records (see status line below the buttons), you can use the "Save" button to save that list in a file, and send me that file. I will then reconstruct a DB similar to yours.
MultiThreadSearch.zip
0
 
LVL 14

Author Comment

by:systan
Comment Utility
ok;

When I open the compiled, error message;
EAssertionFailed:Field ID not integer (ligne 148)

>>it will ask you a list of records to load in memory
It only ask for 3,4,10001,20001,30001,40001,50001,60001,70001,100007

>>the list is defaulted to what I think should be the record index of your 10 fingers
Oh, we suppose to get only the last record Only, which is 100,007
OR, we suppose to get a [record not found] and see the number of seconds it scan's for the whole records.  
I'll attached a bmp that does not belong to the whole records.
And that's the time when it reach to the end of records, and has not found?,  a result time is outputted.

>>you can use the "Save" button to save that list in a file
it's not saving, maybe because of the error on startup
0
 
LVL 14

Author Comment

by:systan
Comment Utility
The error on startup
 the error
The new bmp (that has no record)
 no record finger
0
 
LVL 14

Author Comment

by:systan
Comment Utility
And I'm sorry if I say it wrong and you got it wrong,
the ID has nothing to do with the identification, the template itself is the important.
the ID only appears when the template is found.

if I have miscalculated the fingers that I inputed, maybe 5 to 55,500 is my left thumb finger,  so it doesn't make sense, the id here has nothing to do with the identification.  It's the blobfield.

if I say 3 to 100,006 records is my left thumb, then 100,007 is my right thumb which is the last record,  that makes 100,004 records in a file.  So where scanning the whole file, and identifying my right thumb without knowing the ID, if right thumb is found then record is match with the ID.
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
> the ID only appears when the template is found.
Yes I know that. But is it not an integer ? I can remove that assertion and treat it like a string, if that is the case

look in the INI file, you can setup things (the field names).

> It only ask for 3,4,10001,20001,30001,40001,50001,60001,70001,100007
that is the list of rec# that needs to be loaded in memory. So that you can save to a file all your fingers template. You must change that list if needed so that each and every one of your 10 fingers templates goes in that file. Remember that this list start with item n° 0 (so if I understood correctly you have 3 records with unknown data before your first finger)
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
here is a new app, please send me back the log line that starts with :
"Connected to DB Table"
MultiThreadSearch.zip
0
 
LVL 14

Author Comment

by:systan
Comment Utility
Ok;
>>that is the list of rec# that needs to be loaded in memory. So that you can save to a file all your fingers template.

Why Load only 10 templates in memory?,  While we have a 100 thousands
What if every template has assigned to different people?
Please just don't mine the template records from 3 to 100,006

"
I say 3 to 100,006 records is my left thumb, then 100,007 is my right thumb(which is shown in the display.
"

I only assigned the templates to get the number of records high up to 100,005
And I'm not sure if my fingers are designated on template record id: 3,4,10001,20001,30001,40001,50001,60001,70001


1 thing for sure that I know, that the [last record] is my right thumb, in which is the main point of getting the identification up to the [last record] by a few seconds.



>>(so if I understood correctly you have 3 records with unknown data before your first finger)
1 and 2, I forgot the finger number, but for sure I deleted it because I want to assign the new finger template to a many template, maybe template 1 was 4 to 25,000, 2 was 25001 to 55,000.

I really forgot the arrangement of my fingers during I input as a test,   I did do that to reach the record up to 100,005,  and the last record that I'm sure is my right thumb, which is the point of reaching this template by a very few seconds.


About the new compiled you've send, there is an error after inputting a line on button saved.
Honestly, I don't understand your techniques.  But I'm trying too.

About the NoRecordFinger.bmp ? yes that finger was not inputted in the ms access sample db.
Which is important, when is not found?, then we can determined the number of seconds reach.

epasquier please don't get offended,  I hope you have understand what I mean.
0
 
LVL 14

Author Comment

by:systan
Comment Utility
>>please send me back the log line that starts with :
"Connected to DB Table"

oh, this one?
BMP 355x390 3
Template loaded 554 bytes
Connected to DB Table ENROLL(ID:14,template) 100005 Records
0
 
LVL 14

Author Comment

by:systan
Comment Utility
And oh, about the score, it should be 68, not 288
because that's the real score of that last record template I know.
0
 
LVL 25

Expert Comment

by:epasquier
Comment Utility
> Why Load only 10 templates in memory?,  While we have a 100 thousands
To send ME a file with REAL templates, that I can load and save to DB to reproduce something like your environment....
Because it is obvious that with random data, the templates are quickly discarded by the dll when I compare, so I have search times that are not relevant.
Just look in your .MDB some templates that are unique and load them with the button that asks you the record index, and save in my file format.

Or you can zip your .MDB that will have something like 200Mb and put it somewhere for download... but a few kb file is more appealing to me

By the way : my file format loads 100.000 records in a second or so, while access file via ADO takes more than a minute.. Something to say against DB compared to flat files... And it takes a fifth of the place in harddrive. Maybe you should consider getting rid of MDB completely

> About the new compiled you've send, there is an error after inputting a line on button saved.
that I don't understand... Care to rephrase ?

> And oh, about the score, it should be 68, not 288
I compared the template from the image with ITSELF, not the template made from a different image of the same finger. That is probably why I have a greater score
0
 
LVL 14

Author Comment

by:systan
Comment Utility
I have attached myfingers1to10.zip with 10 bmp's

Ok;
I run the MultiThreadSearch, then click the [dB Load] button.
Click Method: Identify, then start
this is the result on my computer;
///////////
---------------------------- Lines below, NoRecordFinger.bmp was already loaded in the image
BMP 355x390 3
Template loaded 554 bytes
Connected to DB Table ENROLL(ID:14,template) 100005 Records
196594 ms to perform action
=== Looking for ID 100004, 1 Thread(s)
Started Task 1 : 0
Task 1 : Found Element ID 100005 Score=288
Task 1 : Finished, tested 100005 records In 15.281 s
---------------------------- Lines below when I Load the NoRecordFinger.bmp
BMP 355x390 3
Template loaded 535 bytes
=== Looking for ID 100004, 1 Thread(s)
Started Task 2 : 0
Task 2 : Found Element ID 100005 Score=278
Task 2 : Finished, tested 100005 records In 10.204 s
=== Looking for ID 100004, 1 Thread(s)
Started Task 3 : 0
Task 3 : Found Element ID 100005 Score=278
Task 3 : Finished, tested 100005 records In 10.234 s
=== Looking for ID 100004, 1 Thread(s)
Started Task 4 : 0
Task 4 : Found Element ID 100005 Score=278
Task 4 : Finished, tested 100005 records In 10.234 s
---------------------------- Lines below when I Load AGAIN the MyFinger.bmp
BMP 355x390 3
Template loaded 554 bytes
=== Looking for ID 100004, 1 Thread(s)
Started Task 5 : 0
Task 5 : Found Element ID 100005 Score=288
Task 5 : Finished, tested 100005 records In 15.672 s
=== Looking for ID 100004, 1 Thread(s)
Started Task 6 : 0
Task 6 : Found Element ID 100005 Score=288
Task 6 : Finished, tested 100005 records In 15.906 s
---------------------------- Lines below when I Load AGAIN the NoRecordFinger.bmp
BMP 355x390 3
Template loaded 535 bytes
=== Looking for ID 100004, 1 Thread(s)
Started Task 7 : 0
Task 7 : Found Element ID 100005 Score=278
Task 7 : Finished, tested 100005 records In 10.204 s
////////////

While if I test it using the original code that I send, was only 11.500 s, but with the [last record] and if [no record found]
I wonder how short the time in your i7

You, in the [last record] found is 15.0+s, in [no record found] its 10.0+s


ok;
when I click [dB Save] button, errors;
dbsaveclick
when I click [Save] button, errors;
 saveclick

>>That is probably why I have a greater score
I have to see about this, why.

About the 10fingersbmp that I've attached;
Your way will be implemented

>>Maybe you should consider getting rid of MDB completely
ah, probably not this time, its just a test, the dataset will be no longer used when copied to another set.

Actually in my idea, I think 3 to 100,007 recorded template(fingers) is not a big deal, the important is?, how to reach at the end of the records on a few seconds.


Thanks
myfingers1to10.zip
0

Featured Post

Maximize Your Threat Intelligence Reporting

Reporting is one of the most important and least talked about aspects of a world-class threat intelligence program. Here’s how to do it right.

Join & Write a Comment

Introduction The parallel port is a very commonly known port, it was widely used to connect a printer to the PC, if you look at the back of your computer, for those who don't have newer computers, there will be a port with 25 pins and a small print…
Introduction I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need. This article will address some of the concepts when dealing with a multithreaded delphi database…
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
This video explains how to create simple products associated to Magento configurable product and offers fast way of their generation with Store Manager for Magento tool.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

7 Experts available now in Live!

Get 1:1 Help Now