Solved

Fast Strings , Fast Memory assignments .....

Posted on 2002-06-17
21
579 Views
Last Modified: 2010-04-04
Hi all,

I involved in a project where the, total work depends on string functions, assigning memory, copying memory,deleting memory and so on.

Actually the project was completed. But when i compared the similar applications which were developed using c++, my application is working dead slow.Even I tested my application in c++ code.(it is working fine there).  But client requires the source code of my project. I optimized the code also, by removing unneccessary loops and memory assignments.But still it is working slow,


When i read different articles provided on net. Most of the people are saying that delphi memory assignments and delphi strings are slow when compared to other language string functions and memory assignment functions.

what they are saying is hyper strings and fast strings( i don't know what these are) will work fast when compared to delphi strings.

******************
In this case i want explaination of hyper strings and fast strings. Please provide some source codes, if anybody have. According to one article fast strings are much faster than hyper strings.

*******************

 
similarly delphi memory assignments, and memory copy functions are slow b'coz the assembeled code uses 16 bit instructions instead of 32 bit instructions.

Here also i require some source code. If any body have the above source codes please post me.

If any body have detailed information on this  please post ur valuable answers

thanks in advance

Koundinya
0
Comment
Question by:sudhakar_koundinya
  • 8
  • 7
  • 2
  • +4
21 Comments
 
LVL 27

Expert Comment

by:kretzschmar
ID: 7083334
listening . . .
0
 
LVL 22

Expert Comment

by:mnasman
ID: 7083354
Listening, and waiting madshi :-)
0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 7083362
yep,
madshi has the most background for this
from my point of view

look at its madBasic-unit at
http://www.madshi.net/

meikl ;-)
0
 
LVL 20

Expert Comment

by:Madshi
ID: 7083399
:-)   Thanx, guys...

hyper strings and fast strings have some nice routines. The biggest improvements can be found when comparing strings or the like. E.g. if you want to find out whether two strings are identical, while ignoring the case, with only Delphi you would do:

if CompareText(str1, str2) = 0 then

Or if you want to find out whether one string is a part of another, again ignoring the case, you would have to do something like that:

if Pos(AnsiUpperCase(str1), AnsiUpperCase(str2)) <> 0 then

Such things are really slow, if you're calling them quite often. There are much faster functions for such stuff available. I think both hyper strings and fast strings have nice functions there. My package madBasic also contains some fast string functions, as meikl already pointed out. Best would be to download all 3 and compare yourself. Doing a little performance test is really no problem. Just do the things you normally do in a big loop and check how long it takes with different functions.

Memory assignment should not be too bad with Delphi. And especially strings in general are NO WAY slow. Quite the other way round! In Delphi strings are reference counted. That can save both memory and time. Copy actions are done by Delphi automatically in the background, only if it is needed. However, if you're writing "unclever" code, it might be very slow. Just a tight change might make it multiple times faster, simply by avoiding some allocation actions.

Please show us some string/memory actions you're often doing. Then we might find suggestions to make it faster. But have no fear, you can write top performing programs with Delphi string, no problem, if you're doing it right.

Regards, Madshi.
0
 
LVL 11

Expert Comment

by:robert_marquardt
ID: 7083411
I would guess that a memory manager targeted at small memory blocks (which strings usually are) could help.
0
 

Expert Comment

by:lingababu
ID: 7086374
Thanks all,
 for immediate response. May be i am late to react for ur responses :-( . I feel sorry for that.

Thanks Madshi, I will check ur project. If it is useful for me i will try to use that


Pos, Compare,CompareNoCase,StringReplace are the frequent methods I have to use in the project. I read that these methods are slow when compared to other language functions.


When coming to memory management, Move is the method , i am using frequently. This is also problem for me as per articles . So I got one method from some site. In my case it is not helpful for me though it is faster


//My move is not as fast as MOVE when source and destination are both
//DWord aligned, but certainly faster when they are not.
//As we are moving characters in a string, it is not very likely at all that
//both source and destination are DWord aligned, so moving bytes avoids the
//cycle penality of reading/writing DWords across physical boundaries
procedure PMove(const Source; var Dest; Count : Integer);
asm
//Note:  When this function is called, delphi passes the parameters as follows
//ECX = Count
//EAX = Const Source
//EDX = Var Dest

        //If no bytes to copy, just quit altogether, no point pushing registers
        cmp   ECX,0
        Je    @JustQuit

        //Preserve the critical delphi registers
        push  ESI
        push  EDI

        //move Source into ESI  (generally the SOURCE register)
        //move Dest into EDI (generally the DEST register for string commands)
        //This may not actually be neccessary, as I am not using MOVsb etc
        //I may be able just to use EAX and EDX, there may be a penalty for
        //not using ESI, EDI but I doubt it, this is another thing worth trying !
        mov   ESI, EAX
        mov   EDI, EDX

        //The following loop is the same as repNZ MovSB, but oddly quicker !
    @Loop:
        //Get the source byte
        Mov   AL, [ESI]
        //Point to next byte
        Inc   ESI
        //Put it into the Dest
        mov   [EDI], AL
        //Point dest to next position
        Inc   EDI
        //Dec ECX to note how many we have left to copy
        Dec   ECX
        //If ECX <> 0 then loop
        Jnz   @Loop

        //Another optimization note.
        //Many people like to do this

        //Mov AL, [ESI]
        //Mov [EDI], Al
        //Inc ESI
        //Inc ESI

        //There is a hidden problem here, I wont go into too much detail, but
        //the pentium can continue processing instructions while it is still
        //working out the desult of INC ESI or INC EDI
        //(almost like a multithreaded CPU)
        //if, however, you go to use them while they are still being calculated
        //the processor will stop until they are calculated (a penalty)
        //Therefore I alter ESI and EDI as far in advance as possible of using them

        //Pop the critical Delphi registers that we have altered
        pop   EDI
        pop   ESI
    @JustQuit:
end;



****************


One more point i have to share with you guys is

in one article i read that

instead of following like this

while(1): do
begin
a[x]:=b[x]
x:=x+1;
end;

he is suggesting some thin like this for code optimization. I don't know why it should be like that.

x=len(b);
while(1): do
begin
if(x<=0) then break;
a[x]:=b[x];
x:=x-1;
end;

Author of the article is saying that, this will work fast when compared to previous one. Are there any particular reasons for this.


Waiting for responses

thanks
Koundinya
0
 
LVL 20

Accepted Solution

by:
Madshi earned 50 total points
ID: 7086442
Here is a fast Move function. It uses Delphi's move, but it makes sure that all actions are aligned:

procedure MoveAligned(var src, dst; size: integer);
var c1, c2 : cardinal;
begin
  c1 := 4 - dword(@src) mod 4;
  if c1 < 4 then begin
    c2 := (dword(size) - c1) and $FFFFFFFC;
    if dword(@src) > dword(@dst) then begin
      Move(src, dst, c1);
      Move(pointer(dword(@src) + c1)^, pointer(dword(@dst) + c1)^, c2);
      if c1 + c2 < dword(size) then
        Move(pointer(dword(@src) + c1 + c2)^, pointer(dword(@dst) + c1 + c2)^, dword(size) - c1 - c2);
    end else begin
      if c1 + c2 < dword(size) then
        Move(pointer(dword(@src) + c1 + c2)^, pointer(dword(@dst) + c1 + c2)^, dword(size) - c1 - c2);
      Move(pointer(dword(@src) + c1)^, pointer(dword(@dst) + c1)^, c2);
      Move(src, dst, c1);
    end;
  end else begin
    c2 := size and $FFFFFFFC;
    if dword(@src) > dword(@dst) then begin
      Move(src, dst, c2);
      if c2 < dword(size) then
        Move(pointer(dword(@src) + c2)^, pointer(dword(@dst) + c2)^, dword(size) - c2);
    end else begin
      if c2 < dword(size) then
        Move(pointer(dword(@src) + c2)^, pointer(dword(@dst) + c2)^, dword(size) - c2);
      Move(src, dst, c2);
    end;
  end;
end;

Pos is okay, as long as you don't combine it with UpperCase or something like that. Compare is okay, if you really need it this way. If you only want to check, whether 2 strings are identical (case sensitive oder case insensitive), there are faster ways. CompareNoCase? What's that? That's no Delphi function. StringReplace looks complicated/slow. For what purpose exactly do you need that? There are surely faster ways, but depending on what you really want to do...

Regards, Madshi.
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 7086498
Madshi,

You are talking abt aligned actions

What are that?
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 7086507
Lingababu,

Can u explain how ur move works.

thanks
0
 
LVL 20

Expert Comment

by:Madshi
ID: 7086516
If you move from or to an odd physical RAM address, the move often is slower. E.g. if you do this:

Move(pointer(1)^, pointer(5)^, 1000);

Of course this is nonsense. But let's do as if would be real. You're moving from an odd address to an odd address. That's probably slow. Better would be to split that line into those:

Move(pointer(   1)^, pointer(   5)^,   3);
Move(pointer(   4)^, pointer(   8)^, 996);
Move(pointer(1000)^, pointer(1004)^,   1);

This way we're doing 2 slow Moves (the first and third), but both Moves do only copy <4 bytes, so it's no big problem. But the big chunk of the Move is done from an even address to an even address (even is a multiple of 4). These 3 Moves are supposed to be faster than the one from above.

Now my MoveAligned function basically checks the Move addresses and splits one Move call into several, if it makes sense.

Regards, Madshi.
0
Highfive + Dolby Voice = No More Audio Complaints!

Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

 
LVL 20

Expert Comment

by:Madshi
ID: 7086519
Compare the two Moves yourself. I'm quite sure mine is faster.
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 7086533
Thanks Madshi,

I will test that code. DO u have any article regarding fast strings and hyper strings?


0
 
LVL 20

Expert Comment

by:Madshi
ID: 7086543
No.
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 7086576
In MadBasic (your project),
What is the the approach you are following to make the functions work faster.

In my project, in some areas i am following this idea for traversing into the data. But as per lingababu comment, will it work slow

while(1): do
begin
a[x]:=b[x]
x:=x+1;
end;

thanks
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 7086598
Madshi,

Just now I saw this on net


procedure MoveMem32(Src,Dest:Pointer;Size:integer);
// Size=number of dword elements to fill
// assumes that Size>4
asm
  push edi
  push esi
  push ebx
  mov ebx,[eax]
  mov [eax],ebx
  mov ebx,[eax+4]
  mov [eax+4],ebx
  mov ebx,[eax+8]
  mov [eax+8],ebx
  mov ebx,[eax+12]
  mov [eax+12],ebx
  mov ebx,edx
  add ebx,15
  and ebx,-16
  mov edi,ebx
  sub ebx,edx
  shr ebx,2
  sub ecx,ebx
  lea esi,[eax+4*ebx]
  rep movsd
  pop ebx
  pop esi
  pop edi
end;

http://www.optimalcode.com/general.htm


0
 
LVL 20

Expert Comment

by:Madshi
ID: 7086666
>> Just now I saw this on net

Well, compare the speed yourself. Maybe you get the fastest solution, if you combine that MoveMem32 with my MoveAligned. Just change my MoveAligned function in that way, that it called MoveMem32 instead of Move.

>> In MadBasic (your project), what is the the approach you are following to make the functions work faster.

There are several approaches. Do you really need to understand the internals? Why not just trying Fast strings, Hyper strings and madBasic and take the best/fastest from each package?

>> In my project, in some areas i am following this idea for traversing into the data. But as per lingababu
comment, will it work slow

Yes, such a loop will be slow, because it does byte per byte. Copying memory dword by dword is usually way faster, well, if the dword move is aligned (even). Thus I wrote MoveAligned.

Regards, Madshi.
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 7087325
And I have found following code on net. When I tested giving errors.


procedure
   movelong(    fromp   : Pchar;
                var top     : pchar;
                len     : longint);
  {--------
   long mover
   - assumes from and to do not overlap (to not in from) }
type
   longtype        = array[1 .. 63 * 1024] of char;
   longtypeptr     = ^ longtype;
   ptrrec          = record
      ofs, seg     : word; end;
const
   longtypelen     = sizeof(longtype);
begin
   { fix the pointers: offsets between 0 and 15 }
   inc(ptrrec(fromp).seg, ptrrec(fromp).ofs div 16);
   ptrrec(fromp).ofs := ptrrec(fromp).ofs and 15;
   inc(ptrrec(top).seg, ptrrec(top).ofs div 16);
   ptrrec(top).ofs := ptrrec(top).ofs and 15;

   { move pieces }
   while len > sizeof(longtype) do begin
      { faster than: move(fromp^, top^, sizeof(longtype)); }
      asm
         push    ds
         lds     si,fromp
         les     di,top
         mov     cx,(longtypelen / 2)
         cld
         rep     movsw
         pop     ds
      end;
      dec(len, sizeof(longtype));
      inc(ptrrec(fromp).seg, sizeof(longtype) div 16);
      inc(ptrrec(top).seg, sizeof(longtype) div 16);
   end;
   if len <> 0 then
      { faster than: move(fromp^, top^, len); }
      asm
         push    ds
         lds     si,fromp
         les     di,top
         mov     cx,word(len)
         shr     cx, 1
         cld
         jnc     @wordmove
         movsb
      @wordmove:
         rep     movsw
         pop     ds
      end;
end;
0
 
LVL 20

Expert Comment

by:Madshi
ID: 7088038
Looks like 16bit code to me...
0
 
LVL 14

Expert Comment

by:DragonSlayer
ID: 7091609
madshi, your code is making me feel dizzy :)
0
 
LVL 20

Expert Comment

by:Madshi
ID: 7091964
:-)
0
 
LVL 14

Author Comment

by:sudhakar_koundinya
ID: 7239968
thanks  and sorry for late responding
:)
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Creating an auto free TStringList The TStringList is a basic and frequently used object in Delphi. On many occasions, you may want to create a temporary list, process some items in the list and be done with the list. In such cases, you have to…
Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
Access reports are powerful and flexible. Learn how to create a query and then a grouped report using the wizard. Modify the report design after the wizard is done to make it look better. There will be another video to explain how to put the final p…
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now