Solved

stringtokenizer in Delphi/Object Pascal

Posted on 2001-06-05
15
1,302 Views
Last Modified: 2012-06-22
hi there,

I was wondering if there is a sort of StringTokenizer in Delphi. I know that Java has a StringTokenizer, but I can't find it in Delphi, can anyone help me?

A SringTokenizer is an object which divides a string into parts. You can give up a delimiter like:

String deString = "hi,there,this,are,commas";
StringTokenizer someName = new StringTokenizer(deString, ",");

And if i want the next token/element i simply do:
someName.nexttoken();

But this is JAVA, but to implement this in Delphi? Plz hELP!!!

Regards and thanks in advance,
Ruben Bouman,
The Netherlands
0
Comment
Question by:rubenbouman
  • 5
  • 3
  • 2
  • +4
15 Comments
 
LVL 13

Expert Comment

by:Epsylon
ID: 6156387
Check out TStringList and its CommaText property....
0
 
LVL 13

Expert Comment

by:Epsylon
ID: 6156454
Here is an example:

var s: String;
    sl: TStringList;
    i: Integer;
begin
  sl := TStringList.Create;
  s := 'hi,there,this,are,commas';
  sl.CommaText := s;
  for i := 0 to sl.Count -1 do
    ShowMessage(sl[i]);
  sl.Free;
end;
0
 
LVL 3

Expert Comment

by:VSF
ID: 6156596
This is a simple example...
Add:
1 Editbox - Tedit
1 Button - TButton
1 Listbox - TlistBox

//Add this procedure
function sBreakApart(BaseString, BreakString: string; StringList: TStringList): TStringList;
var
EndOfCurrentString: byte;
TempStr: string;
begin
repeat
EndOfCurrentString := Pos(BreakString, BaseString);
if EndOfCurrentString = 0 then
StringList.add(BaseString)
else
StringList.add(Copy(BaseString, 1, EndOfCurrentString - 1));
BaseString := Copy(BaseString, EndOfCurrentString + length(BreakString), length(BaseString) - EndOfCurrentString);

until EndOfCurrentString = 0;
result := StringList;
end;

//Add this code to the OnClick event of the button
procedure TForm1.Button1Click(Sender: TObject);
var
t: TStringList;
begin
t := TStringList.create;
ListBox1.Items.Assign(sBreakApart(Edit1.Text, ' ', t));
t.free;
end;

Run your program and type your name in the Editbox
Press the button
and the list box should show the your name Tokenized using the blank spaces as the string separator.

Adjust this function to any string u want!
0
Free Tool: Site Down Detector

Helpful to verify reports of your own downtime, or to double check a downed website you are trying to access.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 3

Expert Comment

by:VSF
ID: 6156610
To adjust the function just send to the
BreakString: string
any separator u want!

Hope this helps!

E-mail me if u want a compiled sample or leave your e-mail and I'll send it to you!

VSF
www.enge.cjb.net
www.victory.hpg.com.br
UIN:14016999
0
 
LVL 8

Expert Comment

by:Cesario
ID: 6156761
Hi rubenbouman,

you can use the TPARSER-Object from Delphi. You will find it in the CLASSES Unit.

Best Regards

Cesario
0
 
LVL 25

Accepted Solution

by:
Tomas Helgi Johannsson earned 100 total points
ID: 6156833
Hi!

You can create your own Tokenizer easelly. This is how I did it (coded in 3 minutes in D5 ;)


unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  StdCtrls;

type
  TMyClass = class
  private
    TokList: TStrings;
  public
    constructor Create(const s, t: string);
    destructor Destroy;
    procedure AddLists(s: TStrings);
  published
    property thelist: TStrings read TokList write AddLists;
  end;

type
  TForm1 = class(TForm)
    ListBox1: TListBox;
    Edit1: TEdit;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.DFM}

{ TMyClass }

procedure TMyClass.AddLists(s: TStrings);
begin
  TokList.Assign(s);
end;

constructor TMyClass.Create(const s, t: string);
var
  i, j: Integer;
  temp : String;
begin
  temp := s;
  TokList := TStringList.Create;
  i := 1;
  j := 1;
  if Pos(t, temp) > 0 then
    while Pos(t, temp) > 0 do
    begin
      i := Pos(t, temp);
      Toklist.add(Copy(temp, j, i - j));
      delete(temp,j,i);
    end;
    Toklist.Add(Copy(temp, j, i - j));
end;

destructor TMyClass.Destroy;
begin
  TokList.Free;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
  myTokl: TMyClass;
begin
  myTokl := TMyClass.Create(Edit1.Text, '/');
  ListBox1.Items.Assign(myTokl.thelist);
  myTokl.Free;
end;

Regards,
Tomas Helgi
0
 
LVL 25

Expert Comment

by:Tomas Helgi Johannsson
ID: 6156859
Hi again!

I forgot to put in the "NextToken" function.
In TMyClass
under private write:
ix : Integer;
under public write :

function NextToken : String;

and the implementation is

function NextToken : String;
begin
    if ix < TokList.Count then
    begin
       result := TokList[ix];
       Inc(ix);
    end
    else
      result := '';
end;

add to the Constructor

ix := 0;

To prevent the user of this class to access the TokList outside the class you simply delete the property line. ;)

Regards,
Tomas Helgi
0
 
LVL 3

Expert Comment

by:VSF
ID: 6157132
No need to post answers! : )

Just lock his question!

VSF
0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6157305
//this simple function, does what you need
//it returns the first part as result, and shorten the given string
//for next item simple call again
//if there are no items, then the given string becomes empty
function SplitStr(Var AString : String; ADelim : String) : String;
var APos : Integer;
begin
  Result := '';
  APos := Pos(ADelim,AString);
  if APos > 0 then
  begin
    Result := Copy(AString,1,APos-1);
    AString := copy(Astring,APos+length(Adelim),MaxLongInt);
  end
  else
  begin
    Result := AString;
    AString := '';
  end;
end;

//a usage sample
procedure TForm1.Button1Click(Sender: TObject);
var s : String;
begin
 ListBox1.Clear;
 s := edit1.text;
 while s <> '' do
   listbox1.items.add(SplitStr(s,edit2.text));
end;

meikl ;-)
0
 
LVL 3

Expert Comment

by:VSF
ID: 6157763
so rubenbouman... have u tested any code?!
Feel free to ask if u have any doubts!
0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6158777
hi vsf,

who are you, the boy or the girl ?  :-))

sorry to be off topic

meikl ;-)
0
 
LVL 17

Expert Comment

by:geobul
ID: 6158801
Hi,

I've written a small piece of code about dividing a sentence into words. The sentence has to be entered
in the Edit1. Pressing the button passes it as a string to the Parser function and the result is returned
in SList. You can add additional symbols as word delimiters (space, tab and comma here) if you want.
A symbol for the end of the sentence is not mandatory because it ends at the end of the passed string
by default. If such a symbol exists, the rest of the string will be ignored.

unit Unit1;

interface

uses
 Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
 StdCtrls;

type
 TForm1 = class(TForm)
   Button1: TButton;
   Edit1: TEdit;
   Memo1: TMemo;
   procedure Button1Click(Sender: TObject);
   procedure FormCreate(Sender: TObject);
   procedure FormDestroy(Sender: TObject);
 private
   { Private declarations }
 public
   { Public declarations }
 end;

var
 Form1: TForm1;
 SList : TStringList;

implementation

{$R *.DFM}

procedure Parser(Sent: string);
const
 delimiters = [CHR(32),CHR(8),','];
 FullStops = ['.','!','?'];
var
 ch : char;
 i, cnt : integer;
 token : string;
begin
 SList.Clear;
 cnt := Length(Sent);
 token := '';
 if (cnt > 0) then begin
   for i := 1 to cnt do begin
     // Next character
     ch := Sent[i];
     // new token = word
     if ch in Delimiters then begin
       if length(token) > 0 then SList.Add(token);
       token := '';
     end else begin
       // end of sentence
       if ch in FullStops then begin
         if length(token) > 0 then SList.Add(token);
         token := '';
         break;
       end else begin
         // regular character
         token := token + ch;
       end;
     end;
   end;
   // last token
   if length(token) > 0 then SList.Add(token);
 end;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
 i: integer;
begin
 Parser(Edit1.Text);
 // you can analize the words here:
 // for i := 0 to SList.Count - 1 do begin
 //   ...
 // end;  
 // The next line is for visualisation only.
 Memo1.Lines.Assign(SList);
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
 SList := TStringList.Create;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
 SList.Free;
end;

end.

Regards, Geo
0
 

Author Comment

by:rubenbouman
ID: 6159797
Perfect solution! I already used it many many many times!!!

Regards,
Ruben Bouman
0
 
LVL 3

Expert Comment

by:VSF
ID: 6160647
Hi kretzschmar,

I'm the Boy... the girl is my girlfriend Carol. We're togather for almost 3 years now!


0
 
LVL 27

Expert Comment

by:kretzschmar
ID: 6160811
I guessed that, but ... i wasn't sure :-))

nice foto
0

Featured Post

Free Tool: IP Lookup

Get more info about an IP address or domain name, such as organization, abuse contacts and geolocation.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Hello everybody This Article will show you how to validate number with TEdit control, What's the TEdit control? TEdit is a standard Windows edit control on a form, it allows to user to write, read and copy/paste single line of text. Usua…
In my programming career I have only very rarely run into situations where operator overloading would be of any use in my work.  Normally those situations involved math with either overly large numbers (hundreds of thousands of digits or accuracy re…
Microsoft Active Directory, the widely used IT infrastructure, is known for its high risk of credential theft. The best way to test your Active Directory’s vulnerabilities to pass-the-ticket, pass-the-hash, privilege escalation, and malware attacks …

856 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question