Solved

Fastest Way to Extract Record from a Long Semicolon Delimited WideString?

Posted on 2004-04-07
15
382 Views
Last Modified: 2010-04-05
I am writing a function to get data from server and process accordingly. The data receive from server is in a semi colon delimited widestring format.

For example,

var
  Input: WideString;
  i, TotalRec: Integer;

begin
  TotalRec := GetData(Input);
  // Input will have values like 'leon,24,89,100;jimmy,30,77,56;'
  for i := 0 to TotalRec do
  // TotalRec could range from 5,000 - 20,000
  // I need to extract each record to process
  // What's the fastest way to do this?
end;

0
Comment
Question by:coole
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 3
  • 3
  • +4
15 Comments
 
LVL 12

Expert Comment

by:esoftbg
ID: 10773125
unit Unit_Q_20946183;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, StdCtrls, Buttons;

type
  TForm1 = class(TForm)
      SpeedButton1: TSpeedButton;
      ListBox1: TListBox;
      procedure SpeedButton1Click(Sender: TObject);
    private   { Private declarations }
    public    { Public declarations }
      procedure ExtractFromString(Input: WideString);
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.ExtractFromString(Input: WideString);
var
//  i, TotalRec: Integer;
  P:      Integer;
  S:      string;
  procedure Extract(S: string);
  begin
    ListBox1.Items.Add(S);
    Delete(Input, 1, P);
    P := Pos(';',Input);
  end;
begin
  {
  TotalRec := GetData(Input);
  // Input will have values like 'leon,24,89,100;jimmy,30,77,56;'
  for i := 0 to TotalRec do
  // TotalRec could range from 5,000 - 20,000
  // I need to extract each record to process
  // What's the fastest way to do this?
  }
  P := Pos(';',Input);
  while (P>0) do
  begin
    S := Copy(Input, 1, P-1);
    Extract(S);
  end;
  if (Input<>'') then
    Extract(Input);
end;

procedure TForm1.SpeedButton1Click(Sender: TObject);
begin
  ExtractFromString('Leon,24,89,100;Jimmy,30,77,56');
end;

end.

emil
0
 
LVL 12

Expert Comment

by:esoftbg
ID: 10773130
object Form1: TForm1
  Left = 199
  Top = 114
  Width = 696
  Height = 480
  Caption = 'Form1'
  Color = clBtnFace
  Font.Charset = DEFAULT_CHARSET
  Font.Color = clWindowText
  Font.Height = -11
  Font.Name = 'MS Sans Serif'
  Font.Style = []
  OldCreateOrder = False
  PixelsPerInch = 96
  TextHeight = 13
  object SpeedButton1: TSpeedButton
    Left = 36
    Top = 22
    Width = 103
    Height = 22
    Caption = 'Extract'
    OnClick = SpeedButton1Click
  end
  object ListBox1: TListBox
    Left = 150
    Top = 2
    Width = 531
    Height = 441
    ItemHeight = 13
    TabOrder = 0
  end
end
0
 
LVL 2

Expert Comment

by:xxflip
ID: 10773154
You can do as follows:

procedure ParseRecData;
var
 ml:TStrings;
 Input:WideString;
begin
  // Input:=  ----> Method to get Input from server
  ml:=TStringList.Create;
  try
    if Input[1]=';' then Delete(Input,1,1);
    while Pos(';',Input) <> 0 do begin
      ml.Add(copy(Input,1,Pos(';',Input) - 1));
      Delete(Input,1,Pos(';',Input));
    end;
    if length(Input) <> 0 then ml.Add(Input);
    Memo1.Lines.Assign(ml);
  finally
    ml.Free;
  end;
end;
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 2

Expert Comment

by:xxflip
ID: 10773166
boy, one has to be quick if he wishes to be the first :-)
0
 
LVL 12

Expert Comment

by:esoftbg
ID: 10773212
Next code is better:

procedure TForm1.ExtractFromString(Input: WideString);
var
  P:      Integer;
  S:      string;
  procedure Extract(S: string);
  begin
    ListBox1.Items.Add(S);
    Delete(Input, 1, P);
    P := Pos(';',Input);
  end;
begin
  P := Pos(';',Input);
  while (P>0) do
  begin
    S := Copy(Input, 1, P-1);
    Extract(S);
  end;
  if (Input<>'') then
    ListBox1.Items.Add(S);
end;

emil
0
 
LVL 12

Expert Comment

by:esoftbg
ID: 10773218
 if (Input<>'') then
    ListBox1.Items.Add(Input);
0
 
LVL 3

Expert Comment

by:SuperUt
ID: 10773321
Forget any solution using Delete on the very large 'Input' string or Input:= copy( Input, xxx).
This will take a huge amount of time since the string is copied in memory over and over.

Point is to keep the Input string intact so that it isn't moved around in memory.
You could scan the string for the position of the delimiter and take the data out of it.
You can do this using AnsiStrPos(Str, SubStr: PChar) where you move the begin pointer in the string until you reach the end of the string.

Another solution could be to replace the delimiter by a CRLF and paste that string in the text property of a stringlist.
Then you can directly browse the stringlist.
0
 
LVL 22

Expert Comment

by:Ferruccio Accalai
ID: 10773361
what about this?

procedure TForm1.Button1Click(Sender: TObject);
procedure GetRecords(var Records: TStrings;Input: WideString);
  begin
    Records.Clear;
    Records.Delimiter := ';';
    Records.DelimitedText := Input;
  end;
var
List: TStrings;
begin
  List := TStringList.Create;
  try
    GetRecords(List,'leon,24,89,100;jimmy,30,77,56;');
    ListBox1.Items.Assign((list));//do here whatever you want with your record list
  finally
    List.Free;
  end;
end;
0
 
LVL 17

Expert Comment

by:mokule
ID: 10773385
uses
  StrUtils;

procedure TForm1.Extract(S: WideString);
var
  P,P1: integer;
begin
  P1 := 0;
  P := Pos(';',S);
  Memo1.Lines.BeginUpdate;
  while P > 0 do
    begin
    Memo1.Lines.Add(Copy(S,P1+1,P-P1-1));
    P1 := P;
    P := PosEx(';',S,P+1);
    end;
  if P1 < Length(S) then
    Memo1.Lines.Add(Copy(S,P1+1,Length(S)-P1));
  Memo1.Lines.EndUpdate;
end;
0
 
LVL 17

Expert Comment

by:mokule
ID: 10773500
And here is some test

my solution: 17665 ms

and

Feruccio: 671 ms

Well done.
0
 
LVL 2

Expert Comment

by:xxflip
ID: 10774349
This is why I love to read all the questions, once in a while I get my eyes opened ...

The best solution in my opinion: Ferruccio68
0
 
LVL 9

Accepted Solution

by:
mocarts earned 125 total points
ID: 10774891
Does TStrings (TStringList) supports widestring? if that is not so important anyway here is my solution ;)

const
  Semicolon: WideChar = ';';

function GetTotalRec(Input: WideString): integer;
var
  i: integer;
begin
  Result := 0;
  for i := 1 to Length(Input) do
    if Input[i] = Semicolon then
       inc(Result);
end;

function GetTotalRecPos(Input: WideString): TList;
var
  i: integer;
begin
  Result := TList.Create;
  try
    for i := 1 to Length(Input) do
      if Input[i] = Semicolon then
        Result.Add(pointer(i));
  except
    Result.Free;
    raise;
  end;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
  p, i, TotalRec: integer;
  Input: WideString;
begin
  Input := WideString('leon,24,89,100;jimmy,30,77,56;');
  TotalRec := GetTotalRec(Input);
  with GetTotalRecPos(Input) do
  begin
    TotalRec := Count;
    p := 1;
    for i := 0 to TotalRec -1 do
    begin
      ShowMessage(Copy(Input, p, integer(List^[i]) - p));
      p := integer(List^[i])+1;
    end;
    Free;
  end;
end;

also you can split your string in records already in GetTotalRecXXX function
wbr, mo.
0
 
LVL 17

Expert Comment

by:mokule
ID: 10775174
New unofficial test results
-----------------------------

my solution: 17665 ms
Feruccio: 671 ms
mocarts: 30ms

Well done.

It looks like I must improve time resolution (10ms now) or change tested data. ;-)
0
 
LVL 12

Expert Comment

by:esoftbg
ID: 10778958
I think Feruccio's and mocarts's solutions are equivalent each-other. They are the best !!!!

emil
0
 

Author Comment

by:coole
ID: 10789067
Thank you guys. Excellent work!
0

Featured Post

On Demand Webinar: Networking for the Cloud Era

Did you know SD-WANs can improve network connectivity? Check out this webinar to learn how an SD-WAN simplified, one-click tool can help you migrate and manage data in the cloud.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Objective: - This article will help user in how to convert their numeric value become words. How to use 1. You can copy this code in your Unit as function 2. than you can perform your function by type this code The Code   (CODE) The Im…
Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
In this video we outline the Physical Segments view of NetCrunch network monitor. By following this brief how-to video, you will be able to learn how NetCrunch visualizes your network, how granular is the information collected, as well as where to f…
Michael from AdRem Software explains how to view the most utilized and worst performing nodes in your network, by accessing the Top Charts view in NetCrunch network monitor (https://www.adremsoft.com/). Top Charts is a view in which you can set seve…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question